Templates play an important role in automation. This lso applies to Terraform workflows as Terraform is a tool to automate infrastructure lifecycle management. Infrastructure as code (IaC) has allowed us to leverage programming principles in cloud infrastructure maintenance, introducing consistency, faster provisioning, reduced manual maintenance efforts, and, consequently, minimized risks.
In this post, we go through the basics of Terraform Templates and discuss probable use cases and examples. In short, if the applications hosted on your infrastructure depend on certain configuration files – which are usually not part of the application repository – Terraform templates help manage those files in dynamic ways.
Terraform templates are configuration files that define and describe the infrastructure resources required for a particular application or environment using a declarative configuration language called Hashicorp Configuration Language (HCL).
These templates have a “.tf” extension, and they enable users to code infrastructure, ensuring consistent and reproducible deployments.
Inside a template, users can specify different resources such as virtual machines, IAM components, storage resources, or networking components and configure their relationships and properties. When executed using the Terraform CLI, these templates are translated into a set of actions that realize the described infrastructure on the targeted provider.
If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box.
Terraform offers a way to package a group of Terraform scripts as modules. Modules are reusable infrastructure components based on which additional customized infrastructure components are built.
Modules offer a way to customize the included components using input variables. Input variables provide a way to control aspects like scale, range, type, etc., for infrastructure to be provisioned.
Terraform Templates are a great way to further extend the flexibility provided by modules. Templates manage the configuration and data files, enabling granular control and making the module more flexible and reusable.
Like modules, usage of input variables drives template files to shape the actual config or data files on the target resource.
Terraform templates combine a few features – templatefile function, string literals, and template files themselves.
templatefile function
The templatefile() function accepts a couple of inputs:
- A template file (
*.tftpl) - Variable – which could be a list or map.
The template file contains text in UTF-8 encoded text, representing the desired format data. For example, configuration files required by applications have different formats. These files support Terraform’s string literals, which help replace the actual values supplied in variables.
For the final configuration, variables provide a way to set values in these template files dynamically. Before runtime, make sure that the template files are available on the disk.
Terraform template_file
For Terraform versions 0.12 and later, the template_file data source has been superseded by the templatefile function, that can be used directly in expressions without creating a separate data source.
File provisioners
File provisioners provide a way to copy the files from the machine where Terraform code executes to the target resource. The source attribute specifies the file’s source path on the host machine, and the destination attribute specifies the desired path where the file needs to be copied on target.
Learn more about file provisioners: Terraform Provisioners : Why You Should Avoid Them
provisioner "file" {
source = "./app.conf"
destination = "/etc/app.conf"
}jsonencode and yamlencode functions
If the string being generated from the template file is a JSON or YAML file, it could become quite tedious to format it for its validity. The chances of error are high if these files are not formatted properly.
Use jsonencode function and yamlencode function in the template file to produce an output file in respective formats easily.
Example 1 – Create user_data with Terraform
AWS EC2 instances offer support to run shell/bash scripts when the instances are booted. These scripts perform essential tasks like updating the packages, creating environment variables, installing patches, etc. These scripts are provided for EC2 creation in the form of user_data.
Given their nature, these scripts can get very complex. However, once the script is tested it may be used multiple times with slightly different values. As an example, we may need to provide a different set of environment variables for two different sets of EC2 instances.
Using Terraform, we can create a template for this script by using string literals to provide variables’ values dynamically. In the example below, the script creates a directory, cds into that directory, and creates a file within that with some name.
#!/bin/sh
sudo mkdir ${request_id}
cd ${request_id}
sudo touch ${name}.txtLet the name of above template file be script.tftpl. String literals (${ … }) are used to represent variables. To add this script as user_data for EC2 instance, we use the templatefile() function as below.
resource "aws_instance" "demo_vm" {
ami = var.ami
instance_type = var.type
user_data = templatefile("script.tftpl", { request_id = "REQ000129834", name = "John" })
key_name = "tftemplate"
tags = {
name = "Demo VM"
type = "Templated"
}
}We have passed the template file name (script.tftpl) as first parameter, and a map object with request_id and name key-value pairs for substitution. Creating an EC2 instance with a given user_data script runs as expected when Terraform code is applied.
Please note that both template and terraform code files are located in the same directory. Further, Terraform variables are used to create larger map objects for easier management of supplied values.
Example 2 – Template file with for loop
Template file used in previous section is a bash script. Similarly, strings generated by shell or any other type of script are non-repetitive in their formats. Certain file formats are repetitive in nature, i.e., certain lines may have the same format, with different values.
As an example, DNS resolution configurations are maintained in a resolv.conf file, that lists the name servers in below format.
nameserver x.x.x.x
nameserver x.x.x.x
nameserver x.x.x.x
As we can see, the expected format of this file is repetitive. Use for loop expressions to create a template file for what appears as below.
Filename: resolv.conf.tftpl
%{ for addr in ip_addrs ~}
nameserver ${addr}
%{ endfor ~}The corresponding terraform file would have file provisioner with a list type variable as the second parameter.
provisioner "file" {
source = templatefile("resolv.conf.tftpl", {ip_addrs = ["192.168.0.100", "8.8.8.8", "8.8.4.4"]})
destination = "/etc/resolv.conf"
}Applying above Terraform config results in creating the expected resolve.conf file in the target EC2 instance as below.
nameserver 192.168.0.100
nameserver 8.8.8.8
nameserver 8.8.4.4Example 3 – Creating JSON files using templatefile()
Occasionally, when applications depend on externally provided configuration files in JSON format. Since the application logic depends on JSON format, it becomes imperative for Terraform to format the configuration file accordingly.
Creating a valid JSON object using string operations can get tricky. A slight mistake in indentation, or mistyping : or " – in template files – can cause errors. Even if we successfully create a script to create a valid JSON object, there is always a risk of unhandled escape sequences from incoming data.
Terraform provides a function to create valid JSON files from the given template, without worrying about valid formatting. Let us consider that we want to host a product based on microservice architecture. Our microservices are developed in NodeJS.
For the sake of this example, it is critical that the application nodes in our microservice architecture depend on slightly different versions of dependencies. In that case, we create a template file as below – dependencies.tftpl.
${
jsonencode("dependencies": {
"cradle": ${cradle_v},
"jade": ${jade_v},
"redis": ${redis_v},
"socket.io": ${socket_v},
"twitter-node": ${twitter_v},
"express": ${express_v} })
}The corresponding Terraform code appears below. Actual version values are passed via a variable – which would generate the desired dependencies.json file in target path.
variable dep_vers {
default = {
"cradle_v": "0.5.5",
"jade_v": "0.10.4",
"redis_v": "0.5.11",
"socket_v": "0.6.16",
"twitter_v": "0.0.2",
"express_v": "2.2.0"
}
}
provisioner "file" {
source = templatefile("dependencies.tftpl", var.dep_vers)
destination = "/desired/path/dependencies.json"
}Example 4 – Conditions in Terraform templates
Extending the above example, let us assume certain applications do not need certain dependencies to be installed. In that case, developers may choose simply not to supply the corresponding version values.
The current template file (dependencies.tftpl) throws an error in that case. If conditions provide this flexibility and improve the reusability of a given template file, make the dependencies depend on whether the corresponding information is supplied or not – using if conditions as below.
${
jsonencode("dependencies": {
%{if cradle_v != "" }
"cradle": ${cradle_v},
%{ endif }
%{if jade_v != "" }
"jade": ${jade_v},
%{ endif }
%{if redis_v != "" }
"redis": ${redis_v},
%{ endif }
%{if socket_v != "" }
"socket.io": ${socket_v},
%{ endif }
%{if twitter_v != "" }
"twitter-node": ${twitter_v},
%{ endif }
%{if express_v != "" }
"express": ${express_v}
%{ endif }
})
}The summary of the above file is that if no version information for the given dependencies is provided, the dependencies will not be included in the dependencies.conf file. This makes this template file reusable with any microservice application and can be controlled by the dep_vers variable values in Terraform code.
Terraform’s string literals are a great asset of the HCL language. Terraform offers a whole range of functions to perform string templating. The combination of string literals, templatefile() function, and file provisioner can be a huge advantage when triggering the configuration management workflows on Day 0. Terraform templates offer flexibility around the file formats – thus not limiting to specific ones. Additionally, filesystem functions are used to perform validation tasks.
If you need more help with Terraform, I encourage you to check the following blog posts: How to Automate Terraform Deployments, and 12 Terraform Best Practices.
Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.
Manage Terraform Better and Faster
If you are struggling with Terraform automation and management, check out Spacelift. It helps you manage Terraform state, build more complex workflows, and adds several must-have capabilities for end-to-end infrastructure management.
