Using Terraform & Ansible Together

01 Nov 2023·9 min read

In the worlds of infrastructure as code (IaC) and configuration management, Terraform and Ansible have emerged as two key players. Both of these tools are powerful on their own, but you can use them together to achieve end-to-end elevated workflows.

Terraform in a nutshell

Terraform is an IaC tool developed by HashiCorp that focuses on managing and provisioning infrastructure across a wide array of cloud platforms and other platforms such as Kubernetes, and RabbitMQ.

It uses a declarative language called HashiCorp Configuration Language (HCL) that describes what the desired state should be. This contrasts with imperative approaches, where commands are given to reach the desired state. The language is developed for reusability, enabling engineers to implement modules (reusable IaC components), use loops and conditionals, and define different input variables for the configurations.

Terraform maintains a state file to keep track of the resources it manages. This enables it to identify the difference between the current state of the infrastructure and the desired state defined in the configuration files.

Ansible in a nutshell

Ansible is an agentless configuration management tool that relies on SSH for Linux hosts or WinRM for Windows hosts to communicate and execute commands directly, eliminating the need for a separate agent to be installed. Sometimes you will see it called ssh on steroids, which is really accurate.

Ansible configurations are written in playbooks using YAML, and these playbooks describe automation jobs. A single playbook can consist of one or multiple plays, each targeting a set of hosts.

One of Ansible’s core features is idempotence, which means that you can run the same command any number of times and the result will always be the same. This is critical for configuration management and automation, ensuring consistent results.

Ansible can also be used for infrastructure provisioning, but it is not on par with Terraform’s capabilities.

Terraform and Ansible - Why do you need both?

Terraform is used for managing your infrastructure, but if you want to install and configure software on your compute instances, Ansible is the way to go. You could use Terraform’s provisioners to achieve Ansible’s configuration management capabilities, but they are unreliable, and even HashiCorp recommends using them as the last resort.

Alternatively, you could use Ansible to provision your infrastructure via collections implemented for cloud providers, but it is much harder to have a highly customized infrastructure without writing a ton of code.

Although they can sometimes overlap in functionality, they excel in different areas.

💡 You might also like:

Example setup - How to use Ansible with Terraform

A common pattern is to use Terraform to set up base infrastructure, including networking, VM instances, and other foundational resources. Once that’s done, Ansible can be invoked (either manually or via Terraform) to configure those instances, set up necessary software, and deploy applications.

Let’s look at an example that will provision a couple of EC2 instances in AWS and after that will configure them with Ansible:

provider "aws" {
  region = "eu-west-1"
}

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  filter {
    name   = "architecture"
    values = ["x86_64"]
  }
  owners = ["099720109477"] #canonical
}

locals {
  instances = {
    instance1 = {
      ami           = data.aws_ami.ubuntu.id
      instance_type = "t2.micro"
    }
    instance2 = {
      ami           = data.aws_ami.ubuntu.id
      instance_type = "t2.micro"
    }
    instance3 = {
      ami           = data.aws_ami.ubuntu.id
      instance_type = "t2.micro"
    }
  }
}

resource "aws_key_pair" "ssh_key" {
  key_name   = "ec2"
  public_key = file(var.public_key)
}

resource "aws_instance" "this" {
  for_each                    = local.instances
  ami                         = each.value.ami
  instance_type               = each.value.instance_type
  key_name                    = aws_key_pair.ssh_key.key_name
  associate_public_ip_address = true

  tags = {
    Name = each.key
  }
}

In the above example, we are creating three EC2 instances, and adding an ssh key to all of them.

We are assigning public ips to these instances, to speed up the process of configuring them via Ansible. In the real world, usually, all these instances will have only private ips, and they will be configured through a bastion host.

As the code was written with a for_each loop, we can easily extend the number of instances, just by modifying the local Terraform variable.

We’ve also created an output containing all the public ips of the instances:

output "aws_instances" {
  value = [for instance in aws_instance.this : instance.public_ip]
}

After running an apply, this is what we observe:

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

aws_instances = [
  "54.74.244.166",
  "34.254.227.95",
  "63.33.71.25",
]

We can now prepare an inventory file based on the above servers for Ansible. The inventory file in Ansible is a configuration file for your hosts and groups and hosts that tells Ansible where to execute tasks, allowing users to define specific variables and configurations for different hosts.

This is how I created my inventory file:

inventory.ini

[all]
54.74.244.166
34.254.227.95
63.33.71.25

We will create a simple ansible playbook that installs htop.

install_htop.yaml

---
- name: Install htop
  hosts: all
  become: yes
  tasks:
    - name: Update apt cache
      apt:
        update_cache: yes

    - name: Install htop
      apt:
        name: htop
        state: present

Now we can run Ansible and install htop on all of the servers:

ansible-playbook -i inventory -u ubuntu install_htop.yaml

PLAY [Install htop] ************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************
ok: [34.254.227.95]
ok: [54.74.244.166]
ok: [63.33.71.25]

TASK [Update apt cache] ********************************************************************************************************************
changed: [63.33.71.25]
changed: [34.254.227.95]
changed: [54.74.244.166]

TASK [Install htop] ************************************************************************************************************************
ok: [34.254.227.95]
ok: [54.74.244.166]
ok: [63.33.71.25]

PLAY RECAP *********************************************************************************************************************************
34.254.227.95              : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
54.74.244.166              : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
63.33.71.25                : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Now, if we add another EC2 instance in the Terraform configuration, we will have to manually re-run both Terraform and Ansible, which can sometimes be problematic. Whenever you have multiple things to do after you commit your code, the chances for errors increase.

This is where Spacelift comes to the rescue.

How Spacelift can help with your Terraform and Ansible workflow

Stack dependencies v2 is a game changer when it comes to deploying multiple configurations in the correct order, without having to do manual interventions.

Stack dependencies are Directed Acyclic Graphs (DAG) of dependencies that can nest on how many levels you want. This allows you to create powerful end-to-end workflows that can even combine multiple IaC tools such as Terraform, OpenTofu, Ansible, CloudFormation, Kubernetes, Terragrunt, or even Pulumi.

We will create two stacks (one for Terraform and one for Ansible) and a dependency between them.

We will start by creating a context so we can share the ssh keys to both stacks. This is done by going to Contexts, creating a new context and adding the public and private keys as mounted files inside of it:

Next, as we don’t want to use static credentials, we are going to leverage a cloud Integration to AWS. This is easily done by following the guide from here.

In the end, your cloud integration will be similar to this:

I’ve added the code I’ve shown before in a repository:

Next, let’s create two stacks based on the mentioned configurations.

Terraform stack

First go to Stacks and select Create Stack.

Add a name for your stack and select a space. Next, put in the details related to your VCS configuration:

In the next two screens accept the defaults and click on Save Stack.

We need to attach the previously created context and cloud integration to this stack.

Let’s start with the context. Go to the stack settings and select Contexts, choose the context from the drop down list, and click on Attach.

spacelift context terraform with ansible

Similarly, for the cloud integration, go to the stack settings, select Integrations and then AWS.

Now let’s create the Ansible stack.

Ansible stack

The process of creating the Ansible stack is similar to the one for creating the Terraform stack. We will add a name for our stack and select the repository, but we will have to make some changes to the Configure Backend and Define Behaviour Tabs.

In the Configure Backend tab, we will select Ansible and specify the playbook file:

In the Define Behavior tab we will make some changes in the before init runner phase:

chmod 600 /mnt/workspace/id_rsa
echo [all] > /mnt/workspace/inventory.ini
for instance in $(echo "$instances" | jq -r '.[]'); do echo $instance >> /mnt/workspace/inventory.ini; done

We need to ensure the ssh permissions for the private key are 600 (read and write for the owner and no permissions for the group or others).

To ensure there are no problems with the ssh key permissions, add the chmod 600 before the apply phase as well.

With the following two commands, we are building the inventory dynamically. The output of the Terraform stack will be similar to this:

aws_instances = [
  "54.74.244.166",
  "34.254.227.95",
  "63.33.71.25",
]

As my plan is to create a stack dependency between these two stacks and pass this output from the Terraform stack to the Ansible one, I want to ensure these ips are going in the correct format in the inventory file. Based on the configuration above, I will have to name this output “instances”.

Before jumping in and creating the dependency per se, let’s do a couple more configurations to the Ansible stack.

First, let’s attach the context in the same way we did for the Terraform one. Next, go to the Environment tab of the Ansible stack and add the following environment variables:

ANSIBLE_PRIVATE_KEY_FILE – this is the path to the ssh private key used to connect to the ec2 instances
ANSIBLE_INVENTORY – path to the inventory file
ANSIBLE_REMOTE_USER – the remote user used for the ssh connection

Creating the stack dependency

Go to the Terraform stack, select Dependencies, click on Add Dependencies, and select the Ansible stack:

Next, click on Add output reference, select the output and call it instances to ensure our Ansible configuration will work properly:

Click on Add. Now we can run our configuration.

After the run finishes successfully on the Terraform stack, it will automatically trigger a run on the Ansible stack:

Now, if I add a new instance to my Terraform configuration by adding a new map to the local variable, this will be picked up by my Terraform stack and it will get configured with my Ansible one after that.

As you can see, there will be another instance created after I accept the plan. In the meantime, the Ansible stack reached a queued state until the EC2 instances stack finishes the run. If that is successful, it will start the run on the Ansible one.

As soon as it finished creating the EC2 instance, the Ansible stack started the run to configure it. Now that’s finished, all EC2 instances are configured successfully.

Key Points

Despite activating in the same areas for some of their use cases, Terraform and Ansible complement each other. Using them together, can greatly enhance your workflows and simplify the tasks around managing, provisioning, and configuring your infrastructure.

Spacelift greatly simplifies and elevates your workflow for both of these tools, and the capability of creating dependencies between stacks and passing outputs gives you the possibility to make end-to-end deployments while just making a small change.

If you want to learn more about Spacelift, create a free account today or book a demo with one of our engineers.

The Most Flexible CI/CD Automation Tool

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Start free trial

Written by

Flavius Dinu

Flavius is a passionate Developer Advocate with an Infrastructure as Code mindset and expertise in DevOps & Cloud Engineering. He holds ITIL Foundation Certificate in IT Service Management and Hashicorp Terraform Associate Certification. He currently works at Spacelift, and in his free time, he blogs at techblog.flaviusdinu.com, where he provides tutorials, tips, and tricks for all levels of experience based on his exposure.