In this article, we will look at how to run Terraform in an Azure DevOps pipeline step by step. We will go from the start of the process, showing how to create an Azure DevOps instance and project, how to set up Terraform in Azure DevOps, and how to create Terraform configuration files for the infrastructure and pipelines using YAML, sharing some examples and best practices along the way.
Before we get started, make sure you have the following in place:
- Azure Subscription: To host your resources provisioned by Terraform. If you don’t have one, you can sign up for a free trial.
- Azure DevOps Account: To create CI/CD pipelines. If you don’t have one, you can sign up here.
- Service Principal (SP): Setup a Service Principal in Azure Entra ID (Formally known as Azure Active Directory (AAD) with Contributor access to the Azure subscription or Resource Group where you want to deploy resources. Terraform will use the SP to authenticate and interact with Azure. Instructions on how to do that can be found on the Microsoft docs pages here.
Follow these steps to run Terraform in an Azure DevOps pipeline:
- Set up an Azure DevOps project
- Create Terraform configuration files
- Define CI/CD steps in YAML
- Configure the Azure DevOps pipeline
- Run the pipeline
1. Set up an Azure DevOps project
With your account set up, you next need to sign into Azure DevOps and create a project where you’ll be setting up your CI/CD pipelines.
- Go to the Azure DevOps website and sign in with your Microsoft account or organizational account.
- Navigate to Organization: If you’re part of multiple organizations, select the organization where you want to create the project from the top right corner of the page.
- Click on the “New Project” button.
- In the “Create a new project” form, fill in the Project Name and visibility (whether the project should be public or private) and optionally, provide a description for your project, then click on the “Create” button.
- Once the project is created, you’ll be redirected to the project’s dashboard. Depending on your requirements, you may want to customize your project further by adding team members, configuring permissions, setting up repositories, boards, and pipelines, and integrating with other services.
2. Create Terraform configuration files
Create your Terraform configuration files and store them in a repository. This repo could be in your Azure DevOps project, or hosted elsewhere. In this example, we define a configuration file to provision a resource group containing a VNET and subnet.
network.tf
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "example" {
name = "example-resources"
location = "UK South"
}
resource "azurerm_virtual_network" "example" {
name = "example-vnet"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
address_space = ["172.16.0.0/16"]
}
resource "azurerm_subnet" "example" {
name = "example-subnet"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = ["172.16.1.0/24"]
}3. Define CI/CD steps in YAML
As a best practice, your pipelines should be created in code. Create the pipeline in code in YAML format and store it in the same repository:
azure-pipelines.yml
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Plan
displayName: 'Terraform Plan'
jobs:
- job: TerraformPlan
displayName: 'Terraform Plan'
steps:
- task: TerraformInstaller@0
inputs:
terraformVersion: 'latest'
- script: |
terraform --version
terraform init -backend-config="storage_account_name=$(storageAccountName)" -backend-config="container_name=$(containerName)" -backend-config="key=$(key)" -backend-config="access_key=$(accessKey)"
terraform plan -out=tfplan
displayName: 'Terraform Init and Plan'
- stage: Deploy
displayName: 'Terraform Apply'
jobs:
- job: TerraformApply
displayName: 'Terraform Apply'
steps:
- task: TerraformInstaller@0
inputs:
terraformVersion: 'latest'
- script: |
terraform --version
terraform init -backend-config="storage_account_name=$(storageAccountName)" -backend-config="container_name=$(containerName)" -backend-config="key=$(key)" -backend-config="access_key=$(accessKey)"
terraform apply -auto-approve tfplan
displayName: 'Terraform Init and Apply'- The pipeline is triggered whenever changes are pushed to the
mainbranch. - The pipeline runs on an Ubuntu agent (
ubuntu-latest). - TerraformInstaller: Installs the latest version of Terraform on the build agent.
- Script: Executes Terraform commands to initialize the Terraform working directory, generate a plan, and apply changes to Azure. It initializes Terraform with a backend configuration to store the state in an Azure Storage Account (replace
<your-azure-service-connection>with your actual Azure service connection details).
TerraformTaskV1: Uses the Terraform task to apply changes to the infrastructure. It first specifies the plan command in the first stage, and then an apply stage, utilizing the Azure service connection for authentication (environmentServiceNameAzureRM and backendServiceArm), and the working directory where the Terraform files are located. The commandOptions: '-auto-approve' automatically approves the changes without prompting for confirmation.
Download The Practitioner’s Guide to Scaling Infrastructure as Code
4. Configure the Azure DevOps pipeline
To use the pipeline:
- In Azure DevOps, go to “Pipelines” > “Pipelines” and click on “New Pipeline”.
- Point your pipeline to the Git repository containing the
azure-pipelines.ymlfile. - Azure DevOps will automatically detect the YAML configuration file. Review and confirm the configuration, then save.
5. Run the pipeline
To run your new pipeline, first, navigate to Pipelines in the left sidebar. Choose the pipeline that you want to run from the list of available pipelines. If you have multiple pipelines, you’ll see them listed here.
Once you’ve selected the pipeline, you’ll see a “Run pipeline” button at the top right corner. Click on this button to initiate a manual run of the pipeline. If your pipeline is set up to trigger on specific branches, you may be prompted to select the branch you want to run the pipeline for. If not, you can skip this step.
Review the configuration and settings for the pipeline run, and then click on the “Run” button to confirm and start the pipeline execution.
Once the pipeline run is started, you’ll be taken to the pipeline run page, where you can monitor the progress of the pipeline execution. You’ll see the various stages and tasks being executed in real-time. As the pipeline runs, you can view detailed logs for each task to see the output and any error messages. You can also access any artifacts generated by the pipeline, such as build outputs or deployment packages.
After the pipeline run completes, you can review the results to see if the pipeline executed successfully or if there were any failures. You can view the overall status, individual task status, and any associated warnings or errors.
Here are 10 best practices you’ll want to think about when writing your Terraform pipelines in Azure DevOps:
- Define your Azure DevOps pipelines using YAML as code (e.g.,
azure-pipelines.yml) alongside your Terraform configurations. This practice enables versioning, review, and audit of pipeline changes. Store your Terraform configurations in version control (e.g., Git) to track changes, collaborate with team members, and maintain a history of infrastructure modifications. - Configure pipeline triggers to automatically execute pipelines upon code changes (e.g., commits to specific branches or pull requests, or on an overnight schedule to run a plan). This ensures timely validation and deployment of infrastructure changes.
- Optimize pipeline performance by parallelizing tasks whenever possible. For example, run Terraform plan and apply stages concurrently for different environments or resource groups.
- Maintain separate pipelines and environments (e.g., Dev, QA, Prod) to isolate infrastructure changes and avoid unintended impacts across environments. These can be set up in the Environments section of Azure DevOps.
- Integrate automated testing into your pipelines for Terraform configurations using tools like Terratest to validate infrastructure changes before deployment.
- Implement robust error handling and logging mechanisms within pipelines to capture and report errors effectively. Utilize Azure DevOps pipeline features like logging commands and error-handling tasks.
- Include validation steps in your pipelines to verify Terraform configurations against best practices, policies, and compliance requirements using tools like
terraform validateto confirm your syntax is correct,terraform fmtto validate your formatting or call static analysis tools. - Monitor pipeline execution, performance metrics, and resource usage to identify bottlenecks, optimize workflows, and ensure the reliability of infrastructure deployments.
- Apply security best practices to your pipelines, including access control, role-based permissions, and regular security reviews to mitigate risks associated with pipeline configurations and execution.
- Document pipeline configurations, deployment processes, and infrastructure designs to facilitate collaboration, knowledge sharing, and onboarding of team members.
Terraform tasks in the Terraform workflow including deployments can be automated with Azure DevOps CI/CD Pipelines. Azure DevOps is a good choice to seamlessly integrate with Azure and integrate with the Azure ecosystem in particular. Using pipelines means you can repeat the workflow tasks defined in your pipeline in a repeatable and controlled manner, giving you consistency and removing manual effort.
You should also consider the using a Continuous Integration/Continuous Delivery (CI/CD) platform. The most flexible platforms, such as Spacelift, support multiple tools, including Terraform, OpenTofu Ansible, Pulumi, and AWS CloudFormation. See our documentation to find out more about how Azure integrates with Spacelift.
Create a free account today, or book a demo with one of our engineers!
Manage Terraform Better with Spacelift
Build more complex workflows based on Terraform using policy as code, programmatic configuration, context sharing, drift detection, resource visualization and many more.
