Manage infrastructure with Terraform

Manage infrastructure with Terraform

Hashicorp's Terraform is a tool designed for creating, managing, updating, and versioning reproducible application infrastructure. Application infrastructure is composed of all physical and virtual resources (including compute resources and upstack services) which support the flow, storage, processing, and analysis of data.

Terraform can manage all three types of Triton compute resources as first order objects:

1. Bare metal Docker containers. These run the Docker images you expect, but without complication of having to run them in a virtual machine or prepare the infrastructure first.

2. Infrastructure containers. These work like hardware virtual machines, but perform like the bare metal containers they are.

3. Hardware virtual machines. These allow flexibility to run Windows or other non-Linux operating systems.

Installing Terraform

Follow Terraform's installation instructions.

Verify the installation by opening a new terminal session. Execute terraform and you should see a help output similar to this:

$ terraform
Usage: terraform [--version] [--help] <command> [args]

The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
# ...

If you receive an error that terraform could not be found, PATH was not properly set up.

Set PATH on macOS or Linux

Open your terminal and run the following command:

export PATH=$PATH:/path/to/dir

You can also symlink to terraform:

cd /usr/bin
sudo ln -s </path/to/dir> terraform

Set PATH on Windows

Go to: Control Panel -> System -> Advanced System settings* -> Environment Variables.

Scroll down in system variables until you find PATH. Click edit and change accordingly. You will need to launch a new console for the settings to take effect.

Terraform input variables

To ensure your application is shareable and version controlled, it's important to include input variables in your Terraform configuration.

Create a variables file. This file can be named anything, since Terraform loads all files ending in .tf in a directory. The most common file name is variables.tf.

Variables may include login credentials or secret keys. Variables can also include image names, package names, version numbers, counts, and more.

Here is an example of variables including descriptions for each:

variable "service_name" {
  type        = "string"
  description = "The name of the service for Triton CNS."
  default     = "my-application"
}

variable "service_networks" {
  type        = "list"
  description = "Triton networks connected to your instance."
  default     = ["Joyent-SDC-Public"]
}

variable "image_name" {
  type        = "string"
  description = "The name of the image for deployment."
  default     = "nginx-1"
}

variable "image_type" {
  type        = "string"
  description = "Is this image a Linux image or a HVM image."
  default     = "lx-dataset"
}

variable "image_version" {
  type        = "string"
  description = "The version number for this image."
  default     = "20170404"
}

variable "package_name" {
  type        = "string"
  description = "The package to use when making a deployment."
  default     = "g4-highcpu-128M"
}

Terraform modules

Modules are self-contained packages of Terraform configuration. Modules can be used to create reusable components and to organize code. Think of modules like functions: modules have input variables and output variables.

The only required piece of information in a module is the source, which tells Terraform where to download the data sources and resources which in turn tell Terraform what to use.

For example, here is a module for deploying an application to us-sw-1:

module "east" {
  source      = "./modules/service"
  region_name = "us-sw-1"

  instance_count  = 3

  service_production = "${var.service_production}"
  service_name       = "${var.service_name}"
  service_networks   = "${var.service_networks}"

  image_name     = "${var.blue_image_name}"
  image_type     = "${var.blue_image_type}"
  image_version  = "${var.blue_image_version}"
  package_name   = "${var.blue_package_name}"
}

Terraform configuration file

The confirmation file will declare the provider, data sources, resources, and outputs after running Terraform. The minimum version of Terraform required is 0.10.x.

To ensure the correct version is being used, include the following at the top of your configuration file:

terraform {
  required_version = ">= 0.10.0"
}

Define the Triton provider

Providers are the underlying platforms which support Terraform. Providers are responsible for managing the lifecycle of a resource: create, read, update, delete. Triton is a Terraform provider.

provider "triton" {
  # The provider takes the following environment variables:
  # TRITON_URL, TRITON_ACCOUNT, and TRITON_KEY_ID
}

The "triton" provider uses Triton environment variables including your Triton username, SSH fingerprint, and the CloudAPI endpoint.

NOTE: Though it is possible to proceed without setting up environment variables by replacing the contents with the corresponding information, we do not advise you do so. It is a best practice to store all important keys locally instead of tying it to your application files.

Define data sources

Data sources allow data to be fetched or computed for use within Terraform configuration, allowing Terraform to build infrastructure based on information from outside of Terraform (or from a separate Terraform configuration file). Providers are responsible for defining and implementing data sources, which present read-only views of pre-existing data or compute new values on the fly.

Common data sources for Terraform include triton_image and triton_network. Below is an example of data sources which refer to Terraform variables.

data "triton_image" "my_image" {
  name        = "${var.image_name}"
  version     = "${var.image_version}"
  type        = "${var.image_type}"
  most_recent = true
}

data "triton_network" "service_networks" {
  count = "${length(var.service_networks)}"
  name  = "${element(var.service_networks, count.index)}"
}

Define resources

Resource blocks define components of your infrastructure. This could be a VM or container on Triton, or it could be an email provider, DNS record, or database provider.

Below us an example using variables and data sources to build a Triton machine:

resource "triton_machine" "my_new_machine" {
  name     = "my_new_application"
  package  = "${var.package_name}"
  image    = "${data.triton_image.my_image.id}"
  networks = ["${data.triton_network.service_networks.*.id}"]
  cns {
    services = ["${var.service_name}"]
  }
}

Let's break down this block further:

• The name of our container will be my_new_application

• The package assigned to our container is g4-highcpu-128M

• The image uses the ID of the previously defined data source, my_image

• The networks include the IDs taken from the data source service_networks

• We've added tags for Triton CNS, so our DNS names will start with our Triton CNS service name

Add outputs for Terraform data

Terraform creates and stores attribute values for all created resources. Not all of this information may be immediately valuable. Some may have immediate importance such as IP addresses or domain names. Outputs are a way for Terraform to query and display the necessary values after implementing application infrastructure.

Here are some examples for getting the primaryIP address and Triton CNS powered domain names:

output "primaryIp" {
  value = ["${triton_machine.my_new_machine.*.primaryip}"]
}

output "dns_names" {
  value = ["${triton_machine.my_new_machine.*.domain_names}"]
}

Download the Terraform provider

Downloading the Terraform provider is a critical step in determining how Terraform will work going forward. Execute terraform init to download the Triton provider in the background into the local application directory.

   $ terraform init

   Initializing provider plugins...
   - Checking for available provider plugins on https://releases.hashicorp.com...
   - Downloading plugin for provider "triton" (0.4.1)...

   The following providers do not have any version constraints in configuration,
   so the latest version was installed.

   To prevent automatic upgrades to new major versions that may contain breaking
   changes, it is recommended to add version = "..." constraints to the
   corresponding provider blocks in configuration, with the constraint strings
   suggested below.

   * provider.triton: version = "~> 0.4"

   Terraform has been successfully initialized!

   You may now begin working with Terraform. Try running "terraform plan" to see
   any changes that are required for your infrastructure. All Terraform commands
   should now work.

   If you ever set or change modules or backend configuration for Terraform,
   rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

This output informs us that version 0.4 of Triton has been installed. If you require a different version of a provider, you can specify it within the configuration file.

Creating, modifying, and updating application infrastructure

Below are some important terms going forward.

Plan: the plan is the first of two steps required for Terraform to make changes to infrastructure. Using terraform plan determines what changes need to be made and outputs what will be done before it's done.

Apply: the second of two steps required to make changes to the infrastructure. With terraform apply, Terraform communicates with external APIs (i.e. the providers) to make changes.

State: the Terraform state is the state of your infrastructure stored from the last time Terraform was run or applied. By default, this is stored in a local file named terraform.tfstate.

Planning the application infrastructure

Run terraform plan -out application.plan to review what Terraform will be building based on your configuration file. The -out parameter saves the plan to applican.plan to ensure you know exactly what's going to happen when you're ready to deploy.

The result should look similar the following:

   $ terraform plan -out application.plan
   Refreshing Terraform state in-memory prior to plan...
   The refreshed state will be used to calculate this plan, but will not be
   persisted to local or remote state storage.

   data.triton_network.service_networks: Refreshing state...
   data.triton_image.my_image: Refreshing state...

   An execution plan has been generated and is shown below.
   Resource actions are indicated with the following symbols:
     + create

   Terraform will perform the following actions:

     + triton_machine.my_new_machine
         id:                   <computed>
         cns.#:                "1"
         cns.0.services.#:     "1"
         cns.0.services.0:     "my-application"
         created:              <computed>
         dataset:              <computed>
         disk:                 <computed>
         domain_names.#:       <computed>
         firewall_enabled:     "false"
         image:                "45dff701-ce98-481d-94d3-ab0e66fbb8b6"
         ips.#:                <computed>
         memory:               <computed>
         name:                 "my_new_application"
         networks.#:           "1"
         networks.0:           "31428241-4878-47d6-9fba-9a8436b596a4"
         nic.#:                <computed>
         package:              "g4-highcpu-128M"
         primaryip:            <computed>
         root_authorized_keys: <computed>
         type:                 <computed>
         updated:              <computed>

   Plan: 1 to add, 0 to change, 0 to destroy.

   This plan was saved to: application.plan

   To perform exactly these actions, run the following command to apply:
       terraform apply "application.plan"

If there have been any errors, you may have to go back and modify the configuration file before proceeding.

Applying the Terraform plan

Once you know what Terraform will do, use terraform apply to build your infrastructure.

$ terraform apply application.plan

If you've included outputs in your configuration file, you will see those at the end.

Community and support

• The Terraform mailing list and IRC channel (#terraform-tool on Freenode) are great for general questions about Terraform.

• The sdc-discuss mailing list is a great starting point for questions about how to use and manage Triton, including with Terraform: Subscribe, archives.

We have several demos available on Joyent's blog for using Terraform.

• Step-by-step walkthrough building simple application infrastructure

• Blue-green deployment with Terraform

• Deploy to multiple data centers