Terraform for IaaS
Terraform is a tool for configuring Infrastructure-as-a-Service (IaaS) cloud environments. The core of Terraform is open-source; its primary developer (Hashicorp) also offers proprietary components on top but the core (free) components are useful on their own.
A recent article here discussed what is a cloud, and described how an IaaS cloud environment provides a REST API for managing resources such as networks, storage volumes and virtual machines. As well as the API, a cloud provider usually also provides a web-based GUI for managing resources. However defining large numbers of such resources interactively over a UI is slow, error-prone, and not repeatable; it is better to automate such work via the cloud-provider’s API.
I just recently attended a presentation on Terraform (thanks Eric), and this is a brief summary of what I learned from that presentation. Obviously I am no expert - this is just an overview.
Terraform is a simple application which is run manually on a laptop or server, or executed from a CI tool like Jenkins. When executed, it:
- reads an “infrastructure definition” file which defines what resources (VMs, volumes, and networks should exist);
- makes calls to a cloud provider API in order to determine what resources already exist;
- generates a human-readable “plan” describing the changes it intends to apply in order to bring the existing cloud configuration “in-sync” with the definition;
- optionally waits for the “plan” to be approved by an operator; then
- makes calls to the cloud provider API to make the modifications described in the plan (ie creates/deletes/modifies resources as needed).
The result is that the cloud resources are brought into the state described by the input infrastructure definition. These definitions can (and should) be stored in a version-control system. A basic JSON syntax is supported, and a more advanced syntax supporting multiple files, templates, and other useful features.
Unfortunately, there are many different cloud-management APIs - Google GCP, Amazon AWS, OpenStack, CloudStack, and more. Terraform is therefore divided into a “core” module, and an adapter (called a “provider” by terraform) per supported API.
Obviously, different cloud providers also have different features. Terraform’s config syntax does make some effort to standardize definitions of features that are common across multiple cloud providers. However Terraform does allow provider-specific entries in its config-files, and in fact for non-trivial environments it is likely that significant amounts of provider-specific syntax will be needed. In other words, an infrastructure-definition file is typically not portable across providers.
Note that it it not part of Terraform’s feature-set to install software on virtual machines; it simply defines virtual machines (referencing an existing VM image file) and starts/stops them. While slightly off-topic, good options for building VM images are packer (also from Hashicorp), Chef and Puppet.
Terraform for non-IaaS environments
As can be seen from the list of providers, Terraform can also be used to automate configuration of things other than IaaS clouds. In fact, anything that offers a management API for creating/deleting resources can be “automated” via Terraform. Interesting examples are:
- a provider for Docker (local) and Docker-Swarm (distributed); logical as Docker is in some respects similar to an IaaS cloud.
- a provider for Kubernetes (similar to Docker case)
- a provider for Heroku (Platform-as-a-Service ie PaaS),
- providers for Bitbucket/GitHub/GitLab (Git repository hosting) which allows Terraform to define git repositories, webhooks, and assign default reviewers for commits.
- a provider for Chef Server (OS configuration management) which pushes configuration-data (variable-definitions) from Terraform into Chef, allowing Chef to then use these values when pushing software configuration into VM images.
- a provider for Cloudflare (Content Distribution Network) to push configuration data into Cloudflare
- a provider for DNS (Domain Name Service) which allows domain-name records to be pushed to a domain name server.
- a provider for Graphana, allowing Terraform to configure Graphana “dashboards” in an automated way
- providers for MySQL and Postgres (SQL database), allowing Terraform to define databases, users and permissions.