Templates play an important role in automation. This fact is also applicable to Terraform workflows as Terraform is a tool to automate infrastructure lifecycle management. Infrastructure as code (IaC) has allowed us to leverage programming principles in cloud infrastructure maintenance. IaC has introduced consistency, improved speed of provisioning, reduced manual maintenance efforts and thus minimized risks.
In this post, we go through the basics of Terraform Templates and discuss probable use cases and examples. In short, if the applications hosted on your infrastructure depend on certain configuration files – which are usually not part of the application repository – Terraform templates help manage those files in dynamic ways.
Terraform provisions infrastructure resources. It helps create virtual machines, network components, databases, etc., to support the application architecture. Virtual resources often need additional configuration files – in various formats – to function.
Terraform templates provide a way to create these files in the desired format on the target resource. Virtual resources are purpose-driven. To accomplish a certain task, they are configured in a certain way. A slight difference in a given configuration may mean a world of difference in their purpose.
For example, server blocks in NGINX configuration file define which port to listen on for incoming requests. Keeping the security aspect aside, assigning this value as 80 instead of 443 – can be interpreted in many ways. Let us not go too deep into this, but this is a good reminder of how important configuration files are.
Managing the infrastructure as code also includes managing these core configurations on the resources. Otherwise, management of these configurations happens manually. When the infrastructure scales, it is desirable to rely on certain template files that help us configure the target resource correctly. Terraform templates implement just this.
If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box.
Terraform offers a way to package a group of Terraform scripts as modules. Modules are reusable infrastructure components based on which additional customized infrastructure components are built.
Modules offer a way to customize the included components using input variables. Input variables provide a way to control aspects like scale, range, type, etc., for infrastructure to be provisioned.
Terraform Templates are a great way to extend further the flexibility provided by modules. Templates manage the configuration and data files. This enables granular control and makes the module more flexible and reusable.
Like modules, usage of input variables drives template files to shape the actual config or data files on the target resource.
Terraform template combines a few features – templatefile function, string literals, and template files themselves.
templatefile function
The templatefile() function accepts couple of inputs:
- A template file (
*.tftpl) - Variable – which could be a list or map.
The template file contains text in UTF-8 encoded text, representing the desired format data. For example, configuration files required by applications have different formats. These files support Terraform’s string literals, which help replace the actual values supplied in variables.
For the final configuration, variables provide a way to set values in these template files dynamically. Before runtime, make sure that the template files are available on the disk.
File provisioned
File provisioners provide a way to copy the files from the machine where Terraform code executes to the target resource. The source attribute specifies the file’s source path on the host machine, and the destination attribute specifies the desired path where the file needs to be copied on target.
Learn more about file provisioners: Terraform Provisioners : Why You Should Avoid Them
provisioner "file" {
source = "./app.conf"
destination = "/etc/app.conf"
}jsonencode and yamlencode functions
If the string being generated from the template file is a JSON or YAML file, it could become quite tedious to format it for its validity. The chances of error are high if these files are not formatted properly.
Use jsonencode and yamlencode functions in the template file to produce an output file in respective formats easily.
AWS EC2 instances offer support to run shell/bash scripts when the instances are booted. These scripts perform essential tasks like updating the packages, creating environment variables, installing patches, etc. These scripts are provided for EC2 creation in the form of user_data.
Given their nature, these scripts can get very complex. However, once the script is tested it may be used multiple times with slightly different values. As an example, we may need to provide a different set of environment variables for two different sets of EC2 instances.
Using Terraform, we can create a template for this script by using string literals to provide variables’ values dynamically. In the example below, the script creates a directory, cds into that directory, and creates a file within that with some name.
#!/bin/sh
sudo mkdir ${request_id}
cd ${request_id}
sudo touch ${name}.txtLet the name of above template file be script.tftpl. String literals (${ … }) are used to represent variables. To add this script as user_data for EC2 instance, we use the templatefile() function as below.
resource "aws_instance" "demo_vm" {
ami = var.ami
instance_type = var.type
user_data = templatefile("script.tftpl", { request_id = "REQ000129834", name = "John" })
key_name = "tftemplate"
tags = {
name = "Demo VM"
type = "Templated"
}
}We have passed the template file name (script.tftpl) as first parameter, and a map object with request_id and name key-value pairs for substitution. Creating an EC2 instance with a given user_data script runs as expected when Terraform code is applied.
Please note that both template and terraform code files are located in the same directory. Further, Terraform variables are used to create larger map objects for easier management of supplied values.
Template file used in previous section is a bash script. Similarly, strings generated by shell or any other type of script are non-repetitive in their formats. Certain file formats are repetitive in nature. I.e. certain lines may have same format, with different values.
As an example, DNS resolution configurations are maintained in resolv.conf file, that lists the name servers in below format.
nameserver x.x.x.x
nameserver x.x.x.x
nameserver x.x.x.x
As we can see, the expected format of this file is repetitive. Use for loop expressions to create a template file for the same as below.
Filename: resolv.conf.tftpl
%{ for addr in ip_addrs ~}
nameserver ${addr}
%{ endfor ~}The corresponding terraform file would have file provisioner with a list type variable as second parameter.
provisioner "file" {
source = templatefile("resolv.conf.tftpl", {ip_addrs = ["192.168.0.100", "8.8.8.8", "8.8.4.4"]})
destination = "/etc/resolv.conf"
}Applying above Terraform config results in creating the expected resolve.conf file in target EC2 instance as below.
nameserver 192.168.0.100
nameserver 8.8.8.8
nameserver 8.8.4.4There may be times when applications depend on externally provided configuration files in JSON format. Since the application logic depends on JSON format, it becomes imperative for Terraform to format the configuration file accordingly.
Creating a valid JSON object using string operations can get tricky. A slight mistake in indentation, or mistyped : or " – in template files – can cause errors. Even if we successfully create a script to create a valid JSON object, there is always a risk of unhandled escape sequences from incoming data.
Terraform provides a function to create valid JSON files from the given template, without worrying about valid formatting. Let us consider that we want to host a product based on micro service architecture. Our micro services are developed in NodeJS.
For the sake of this example, it is critical that the application nodes in our micro service architecture depend on slightly different versions of dependencies. In that case, we create a template file as below – dependencies.tftpl.
${
jsonencode("dependencies": {
"cradle": ${cradle_v},
"jade": ${jade_v},
"redis": ${redis_v},
"socket.io": ${socket_v},
"twitter-node": ${twitter_v},
"express": ${express_v} })
}The corresponding Terraform code looks like below. Actual version values are passed via a variable – which would generate the desired dependencies.json file in target path.
variable dep_vers {
default = {
"cradle_v": "0.5.5",
"jade_v": "0.10.4",
"redis_v": "0.5.11",
"socket_v": "0.6.16",
"twitter_v": "0.0.2",
"express_v": "2.2.0"
}
}
provisioner "file" {
source = templatefile("dependencies.tftpl", var.dep_vers)
destination = "/desired/path/dependencies.json"
}Extending the above example, let us assume certain applications do not need certain dependencies to be installed. In that case, the developers may choose to just not supply the corresponding version values.
The current template file (dependencies.tftpl) throws an error in that case. If conditions provide this flexibility and improve the reusability of a given template file. Make the dependencies depend on the whether the corresponding information is supplied or not – using if conditions as below.
${
jsonencode("dependencies": {
%{if cradle_v != "" }
"cradle": ${cradle_v},
%{ endif }
%{if jade_v != "" }
"jade": ${jade_v},
%{ endif }
%{if redis_v != "" }
"redis": ${redis_v},
%{ endif }
%{if socket_v != "" }
"socket.io": ${socket_v},
%{ endif }
%{if twitter_v != "" }
"twitter-node": ${twitter_v},
%{ endif }
%{if express_v != "" }
"express": ${express_v}
%{ endif }
})
}The summary of the above file is that if no version information for the given dependencies is provided, the dependencies will not be included in the dependencies.conf file. This makes this template file reusable with any micro service application and can be controlled by the dep_vers variable values in Terraform code.
Terraform’s string literals are a great asset of the HCL language. Terraform offers a whole range of functions to perform string templating. The combination of string literals, templatefile() function, and file provisioner can prove to be of huge advantage when triggering the configuration management workflows on Day 0. Terraform templates offer flexibility around the file formats – thus not limiting to specific ones. Additionally, filesystem functions are used to perform validation tasks.
If you need more help with Terraform, I encourage you to check the following blog posts: How to Automate Terraform Deployments, and 12 Terraform Best Practices.
Manage Terraform Better and Faster
If you are struggling with Terraform automation and management, check out Spacelift. It helps you manage Terraform state, build more complex workflows, and adds several must-have capabilities for end-to-end infrastructure management.
Terraform Functions Cheatsheet
Grab our ultimate cheat sheet PDF for all the Terraform functions you need on hand.
