Google AppEngine

Categories: Cloud

A previous article on this site gave an overview of the Google Cloud Platform (GCP); this article looks more closely at the AppEngine service.

Introduction

Google Cloud Platform (GCP) provides several options for executing custom code on the platform. From low-level (concrete) to high-level (abstract), they are:

  • Compute Engine
  • Kubernetes Container Engine
  • App Engine Flexible
  • App Engine Standard
  • Cloud Functions

Compute Engine provides traditional virtual machines (VMs). Each VM acts like a physical server - you must boot an operating system image on it (which must match the hardware architecture of the VM - currently all x86-64). You are then responsible for applying kernel patches (and rebooting if necessary), installing and configuring software, configuring networking, etc. If you wish to provide highly-available or horizontally-scaled software over multiple VMs then you are responsible for configuring load balancers appropriately, etc. In short, nearly full control but also full responsibility. VM images are reasonably portable; the base OS is typically a google-provided OS image, but all the software installed on top is under your control and can be “rebased” onto a different OS image for use with a different cloud (or in-house datacenter) with little effort.

With Kubernetes Container Engine, software to be executed must be packaged as linux container images, and a configuration file then defines groups of images which should be deployed together to form a service (a “pod”), together with required network configuration, storage requirements, and other settings. Because each container is a complete operating-system user-space, a single container image can contain applications of almost any type. The resulting system can be horizontally scaled by simply specifying that multiple instances of a “pod” be started. Kubernetes takes care of configuring load-balancers so that incoming load is distributed across the available instances, and handles crashes of running images by restarting them. Kernel-level security patches are handled automatically by the cloud platform, but applying fixes to user-space operating system components are the responsibility of the application owner (rebuilding container images). It is because containers are “abstracted” from the real hardware somewhat that Google can provide the helpful features (kernel patching, scaling, load-balancing), but the abstractions also limit the options of the developer somewhat.

App Engine Flexible is a container-based version of App Engine Standard; it has some properties of both the Container Engine and App Engine options. App Engine Flexible is discussed later in more detail.

With App Engine Standard, software to be executed must be packaged in one of a small number of supported formats, including:

  • Java web archive (“.war”) files
  • PHP standard applications
  • Python and Ruby applications
  • Node.js applications
  • C# and Go applications

An App Engine Standard application is a single application in a single language; no operating-system components may be deployed with them, and no “native code”. In return, Google can scale such applications horizontally very fast (ie start and stop instances), and handles load-balancing automatically. The rest of this article discusses App Engine advantages and limitations in more detail.

Cloud Functions are even more fine-grained than App Engine; each “deployable unit” is a single code function. There are many things that simply cannot be implemented with such an extremely decoupled system - but it scales extremely rapidly.

In addition, the cost-per-unit-of-useful-work generally decreases as the platform becomes more abstract; more abstract systems are “more transparent” to Google, allowing Google to optimise their execution better. This in turn allows Google to deliver more useful work on the same hardware, wasting fewer CPU cycles and memory, which implies that Google can offer the service at a lower cost. Executing a basic webapp on AppEngine is far cheaper than delivering the same functionality on a dedicated VM - particularly when the total load varies over time.

App Engine Standard Deployment

To deploy an application into the App Engine runtime environment, a developer builds an application locally (or via an automated build-system). The application is then sent to the google AppEngine build-servers which post-processes the application then builds a “package” and stores it in a google repository.

When an instance of that application needs to be deployed, Google allocates a container, deploys the package into the container, and updates load-balancer settings. As an example, a java application is uploaded as a “.war” file, which is packaged and cached by google. To run the app, Google starts a container with Java and a ServletEngine environment, and then deploys the war-file to that server.

When load on existing (running) instances of an AppEngine application becomes too high, Google simply starts more instances. When load drops, those unneeded instances are stopped. When load is particularly low, the number of instances can be scaled down to zero - particularly useful for apps that are only accessed during business hours, as there are no charges when it is not being used! When a request is received and no instances are running, one is started - within a handful of seconds.

App Engine Standard Limitations

In order to provide cheap and quickly-scalable applications, Google puts quite a lot of limitations on AppEngine Standard applications. The most critical are:

  • a maximum of 1GB ram per instance (and the app runtime, eg JVM, is at least partially included in that)
  • no threads (“background processing” is possible but requires using a special API)
  • each http request has max limits on data uploaded in the request, downloaded in the response, and the total duration of the request.

And due to the fact that the developer provides “only the app”, no os-level external tools can be bundled with the app.

App Engine Flexible as Container

With AppEngine Standard apps, the developer provides just the application code (eg PHP’-package or java-war-file). With AppEngine Flexible, there are several options:

  • the developer can provide an “executable app” (eg an executable java jarfile) and specify an appropriate AppEngine base container image to wrap it in;
  • or the developer can provide a complete container image.

The AppEngine Standard limitations on threads, ram, http-request upload/download sizes, and http-request duration do not apply to the Flexible environment. However flexible-environment apps will never be auto-scaled down to zero instances, and it takes longer to start new instances (1-2 minutes vs 5-10 seconds for AppEngine Standard).

When providing a complete app, it should open TCP port 80 for incoming HTTP traffic after starting. Whatever else the app wants to do is up to it.

The AppEngine-provided containers are generally preferable to rolling your own - there is quite a lot of useful stuff in the standard containers, eg nginx and a log-forwarding agent that sends STDOUT from your app to StackDriver.

Network Routing

All AppEngine instances sit behind a common load-balancer. This load-balancer is not directly configurable.

The project gets a base domain-name, .appspot.com and each application gets a subdomain.

Only one AppEngine instance in a project is the “default app”.

Multiple versions can be running at the same time, with load distributed across them, eg 10% new, 90% old. However only basic control possible - kubernetes or VMs give far more control.

Scaling is integrated with networking - distribution assumes stateless back end AFAIK.

Scaling and Startup

AppEngine Standard instances start very fast. It is possible to set scaling to zero, and let GCP start an instance when the first request arrives.

AppEngine Flexible instances need a minute or so to start; they have a minimum of one active instance.

Scaling controls are done with appengine-specific settings in the “app.yaml” file.

AppEngine Flexible vs Google Container Engine (ie Kubernetes)

Flexible environment is like Google Container Engine except:

  • automatic load-balancing, versioning, load-splitting, rolling upgrades
  • autoscaling is simpler to configure
  • google provides tools to build the container image based on a yaml file as with java standard environment
  • other appengine google services are available via the google appengine libraries rather than exclusively via REST (because the container runs within the appengine cluster)
  • AppEngine SDK can be used to run the app during development, eg emulates DataStore, email and login…
  • background tasks available

Security

Usually, AppEngine applications which access other GCP services (eg reading from a database as a result of receiving an incoming http request) run as a project-specific “service account”; the IAM system can then be used to define what that application can and cannot access.

Google provides two main tools for securing an app against incoming HTTP requests:

  • Cloud Endpoints
  • IAP

Cloud Endpoints is a framework intended for securing REST servers (not ones serving HTML). In short, the developer defines a configuration-file that defines the REST endpoints of the app, and whether they should be secured or not. When using Cloud Endpoints with AppEngine Flexible, the AppEngine container used to host the application includes an NGINX instance which validates all incoming requests before forwarding them to the actual application. When using Cloud Endpoints with AppEngine Standard, the application itself must include the Cloud Endpoints library in its code-base instead. When using Cloud Endpoints with Kubernetes, an extra container must be added to each “pod”. In general, Cloud Endpoints appears to be aimed at securing the “back end” part of an app which implements its user-facing UI as a mobile-app or using a javascript-rich-client.

Because Cloud Endpoints is aware of the full set of entry-points to the app (via the Swagger-based declarations needed), it can (and does) generate good statistics on the usage of the various endpoints it is protecting.

IAP (Identity-aware Proxy) is instead functionality that is built in to the AppEngine network-load-balancer infrastructure. When IAP is enabled for a project, then no HTTP request is allowed through the load-balancers unless the request includes a suitable OAuth ticket issued by the Google auth-servers - ie the requesting user must have “logged on” as some account registered with Google. The identity of the user is then provided as an HTTP header; the application itself is responsible for determining whether that user is allowed to access a specific url or not. Unfortunately, IAP:

  • is a per-project switch, ie is enabled for all AppEngine apps or none,
  • can only authenticate Google accounts, and
  • does not really provide authorization - just a choice of allowing:
    • all requests
    • all users logged-in to Google
    • all users logged-in to a specific domain
    • all users in a specific group

On the positive side, IAP is probably very robust (as it is part of the load-balancer infrastructure).

Configuration

The GCP metadata-service provides a way to manage central settings for deployed AppEngine instances. However the application must include code to look up such configuration explicitly. In general, apps are deployed with their config built-in rather than being externally configurable (ie to change config, redeploy).

Deployment and Rollback

Deployment of a new version is done in an atomic way - first N instances of the new version are started, and then the “loadbalancer” is reconfigured to point to the new version, and the old containers are stopped. AppEngine keeps the most recent N deployed versions of an application, and a web ui can be used to “roll back” to a previous version if desired.

References and Useful Links