Skip to content

misterpilou/appengine-python3

 
 

Repository files navigation

Google App Engine Python 3 SDK (Unofficial, Experimental)

This project is a hacky port of the Google App Engine SDK to Python 3.4, and can be deployed to App Engine as a Managed VMs Custom Runtime. It's an experimental side project (done for fun and for the educational experience), and there are no specific plans to maintain or support it (but forks and pull requests are welcome).

What's in the box?

  • The App Engine SDK, as of December 2014 (grabbed from a Python 2.7 managed VM), partially ported to Python 3. The port includes the following:
    • The GAE client libraries (datastore, log service, task queue service, user service, etc.).
    • dev_appserver.py, which can be used to run your app locally without using the Managed VMs environment (although the sandbox code was removed rather than ported).
    • The stub service implementations (e.g. datastore_sqlite_stub) that ship with the regular Python 2.7 SDK.
    • Many of the third-party libraries were replaced with Python 3 versions.
  • A dockerfile that can be used to build a base Docker image that is mostly interchangeable with the official google/appengine-python27 image.

How stable is the project?

Highly unstable! The SDK can run a basic demo, but it is not at all ready for production use. It was not developed by auditing the code, it was developed by repeatedly running the code and seeing what crashed, then fixing that. This means that the code is particularly vulnerable to security holes and race conditions, and if you use the SDK in non-standard ways, you're likely to run into bugs.

What's the status?

Currently, enough code has been ported to run the guestbook demo in the tutorial, but not much more than that.

  • There are lots of tests, and most of them fail.
  • Since the guestbook demo only uses NDB, the original DB library has not been ported at all.
  • Many little pieces are missing.
  • Much of the documentation is out of date now. The word "string" is used all over the place in cases where the type is now bytes.

How might I find this port useful?

As I mentioned above, this project isn't specifically meant to be useful; I did if for my own education. That said, there are a number of ways that you might find it useful anyway:

  • You can use the SDK for small projects and side projects where stability and security aren't as much of a concern.
  • If you're more serious about Python 3 App Engine support than I am, you're welcome to fork the project or submit pull requests to put it into a more stable state. You could also start from scratch and use this project as an example. However, keep in mind that I think the ideal approach is for Google to develop a higher-quality port (see below for some reasons).
  • You can try out an existing App Engine Python 2.7 app on this SDK to see how it fares when run under Python 3. This may help you prepare for the future when a more stable Python 3 App Engine SDK exists (if that ever happens).
  • If you want a real-world example of what it's like to port a project to Python 3, you can look through the commits. I tried to make the commit messages reasonably high-quality.

Also, keep in mind that there are some inherent tradeoffs between Managed VMs and normal sandboxed App Engine hosting, and the benefits of Python 3 may not outweigh the drawbacks of Managed VMs (e.g. slower instance startup time and added development complexity). See the Managed VM Considerations section of the docs for details.

Getting started

See the sample app README for setup instructions.

FAQ

How is this project possible? I thought App Engine's sandbox forced you to only use their built-in runtime and libraries.

There's a recent feature called Managed VMs that allows your app to run inside a Docker container instead of Google's custom sandbox. This makes instances a little more "heavyweight", both in development and production, but the advantage is that you get full control over the Docker image, so you can define whatever runtime you want.

Why is Google in a better position to write a high-quality Python 3 runtime?

A number of reasons:

  • Obviously, they wrote the original SDK, so they're the most qualified to make changes to it.
  • The SDK that I started with is the version that's publicly available, which seems to have almost all comments removed (although the docstrings are still there). There are also tens of thousands of lines of autogenerated protocol buffer code, but the source .proto files are missing. There's obviously a "real" repository out there, which would be the right place to make a big change like a port to Python 3.
  • The responsible way to port a library to Python 3 is to do a careful audit of all of the code and transition it to a state where it can run as Python 2 or Python 3 in the same codebase. This avoids the cost of maintaining two separate codebases, but only if you're porting the original codebase, not a fork.

What would I need to do to run an existing Python 2.7 GAE app in this runtime?

Obviously, you'll need to port your code to Python 3. After that, you'll need to change some settings in app.yaml to indicate that it's a custom runtime. You'll also need to remove your library dependencies from app.yaml and add them to your Dockerfile as pip install commands.

Is the Docker image hosted anywhere?

No. At this point, if you decide to use the SDK, you should build the Docker image yourself, since you'll probably need to iterate on the SDK anyway.

Why did you port dev_appserver and related code if it's not needed by Managed VMs?

It seems easier to debug and iterate on the runtime itself when working with the regular dev_appserver process. Also, it makes it possible to run existing unit tests that depend on the test stubs for the datastore and other services. That said, it seems like a reasonable simplification to just focus on the client libraries.

About

An unofficial experimental port of the Google App Engine SDK to Python 3, using Managed VMs

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 86.0%
  • PHP 11.2%
  • JavaScript 1.6%
  • HTML 0.9%
  • GAP 0.2%
  • CSS 0.1%