In this article, we will take a look at Terraform testing strategies using different types of commonly used testing, tools and platforms. Let’s jump straight in!
We will cover:
As with any code, you should implement some testing strategies to help validate your code and ensure that your changes do not cause any unexpected issues or break existing functionality.
Common Terraform testing strategies include:
- Integration testing
- Unit testing
- End-to-end (E2E) testing
- Linting
- Compliance Testing
- Drift Testing
In this article, we will take a look at each and how to use them with Terraform code.
Integration testing involves testing your entire infrastructure configuration, including any dependencies on other resources, to ensure that they work together correctly. Terraform has built-in dependency mapping and management that can be utilized to make sure the changes being made are as expected.
You can use tools like Terratest or Kitchen-Terraform for integration testing.
The following tools can also be used in the workflow, and including them in the CI pipeline will form Integration Testing:
- Terraform fmt — to format the code correctly.
- Terraform validate — to verify the syntax.
- Terraform plan — to verify the config file will work as expected.
- TFLint — to verify the contents of the configuration as well as the syntax and structure, also checks account limits (e.g. that a VM instance type is valid, and that the limit on the number of VMs in Azure has not been reached).
- Kitchen-Terraform — Kitchen-Terraform is an open-source tool that provides a framework for writing automated tests that validate the configuration and behavior of Terraform code, including testing for errors, resource creation, destruction, and validation of outputs. Kitchen-Terraform uses a combination of Ruby, Terraform, and Test Kitchen to spin up infrastructure in a sandbox environment, run tests, and destroy the infrastructure once testing is complete.
- Terratest — Terratest is an open-source testing framework for testing Terraform that can also be used to test Kubernetes, Docker, and Packer, amongst others. Terratest enables automated testing of infrastructure code in a sandbox environment to validate that it meets requirements, functions as expected, and is reliable. Tests are written in Go, and it provides a rich set of testing functions that allow the user to automate the entire testing process, including provisioning resources, running tests, and cleaning up resources. Terratest also provides built-in support for popular testing frameworks like Ginkgo and Gomega.
How to Use Terratest
In the Terratest example below, a function is created to test the creation of an Azure storage account in a resource group, and a container in the storage account, then verifies that the storage account and container exist. The resources are then cleaned up after the test is complete.
package test
import (
"context"
"fmt"
"testing"
"time"
"github.com/Azure/azure-sdk-for-go/storage"
"github.com/gruntwork-io/terratest/modules/azure"
"github.com/gruntwork-io/terratest/modules/random"
"github.com/gruntwork-io/terratest/modules/testing"
)
func TestAzureStorageAccount(t *testing.T) {
t.Parallel()
// Set the Azure subscription ID and resource group where the Storage Account will be created
uniqueID := random.UniqueId()
subscriptionID := "YOUR_AZURE_SUBSCRIPTION_ID"
resourceGroupName := fmt.Sprintf("test-rg-%s", uniqueID)
// Create the resource group
azure.CreateResourceGroup(t, resourceGroupName, "uksouth")
// Set the Storage Account name and create the account
accountName := fmt.Sprintf("teststorage%s", uniqueID)
accountType := storage.StandardZRS
location := "uksouth"
account := azure.CreateStorageAccount(t, subscriptionID, resourceGroupName, accountName, accountType, location)
// Create a container in the Storage Account
ctx := context.Background()
client, err := storage.NewBasicClient(accountName, azure.GenerateAccountKey(t, accountName))
if err != nil {
t.Fatalf("Failed to create Storage Account client: %v", err)
}
service := client.GetBlobService()
containerName := fmt.Sprintf("testcontainer%s", uniqueID)
if _, err := service.CreateContainer(ctx, containerName, nil); err != nil {
t.Fatalf("Failed to create container: %v", err)
}
// Check if the container exists
testing.WaitForTerraform(ctx, terraformOptions, nil)
// Cleanup resources
defer azure.DeleteResourceGroup(t, resourceGroupName)
}Unit testing involves testing individual modules or resources in isolation to ensure that they work as expected. This can again be done using tools like Terratest or using Terraform’s built-in testing functionality.
Unit testing with CDKTF
AWS CDK for Terraform (CDKTF) is a flexible open-source framework that provides a high-level object-oriented abstraction on top of Terraform to enable infrastructure-as-code development using popular programming languages such as TypeScript, Python, and Java. To unit test your CDKTF code, you can use a testing framework such as unittest, Jest or Mocha.
In the example below, CDKTF is used with Python to validate the creation of an AWS S3 bucket region, lifecycle rules, and access control policies, using the unittest framework. The stack is deployed using the app.synth() method, which generates the Terraform configuration and applies it to create the S3 bucket.
import unittest
from aws_cdk import App, Stack
from my_constructs import S3Bucket
class TestS3Bucket(unittest.TestCase):
def test_creates_bucket_with_correct_configuration(self):
app = App()
stack = Stack(app, 'test-stack', env={'region': 'eu-west-2'})
# Create the construct
bucket_name = 'my-test-bucket'
lifecycle_rules = [
{'expiration': {'days': 90}, 'id': 'log-files', 'prefix': 'logs/'},
{'expiration': {'days': 120}, 'id': 'backup-files', 'prefix': 'backup/'},
]
access_control_policy = {
'actions': ['s3:GetObject'],
'effect': 'Allow',
'principals': [{'service': 'lambda.amazonaws.com'}],
'resources': [f'arn:aws:s3:::{bucket_name}/backup/*'],
}
S3Bucket(stack, 's3-bucket', bucket_name=bucket_name, lifecycle_rules=lifecycle_rules, access_control_policy=access_control_policy)
# Deploy the stack
synthed = app.synth()
template = synthed.get_stack(stack.stack_name).template
# Verify that the bucket was created with the correct configuration
self.assertEqual(template['Resources'][bucket_name]['Type'], 'AWS::S3::Bucket')
self.assertEqual(template['Resources'][bucket_name]['Properties']['BucketName'], bucket_name)
self.assertEqual(template['Resources'][bucket_name]['Properties']['LifecycleConfiguration']['Rules'], lifecycle_rules)
self.assertEqual(template['Resources'][bucket_name]['Properties']['PublicAccessBlockConfiguration'], {'BlockPublicAcls': True, 'BlockPublicPolicy': True, 'IgnorePublicAcls': True, 'RestrictPublicBuckets': True})
self.assertEqual(template['Resources'][bucket_name]['Properties']['CorsConfiguration']['CorsRules'], [])
self.assertEqual(template['Resources'][bucket_name]['Properties']['BucketEncryption']['ServerSideEncryptionConfiguration'], [])
self.assertEqual(template['Resources'][bucket_name]['Properties']['AccessControl'], 'Private')
self.assertEqual(template['Resources'][bucket_name]['Properties']['AnalyticsConfigurations'], [])
self.assertEqual(template['Resources'][bucket_name]['Properties']['InventoryConfigurations'], [])
self.assertEqual(template['Resources'][bucket_name]['Properties']['LoggingConfiguration'], {'DestinationBucketName': bucket_name, 'LogFilePrefix': 's3-logs/'})
self.assertEqual(template['Resources'][bucket_name]['Properties']['ObjectLockConfiguration'], {'ObjectLockEnabled': 'Enabled'})
self.assertEqual(template['Resources'][bucket_name]['Properties']['VersioningConfiguration'], {'Status': 'Enabled'})
self.assertEqual(template['Resources'][bucket_name]['Properties']['WebsiteConfiguration'], {})
if __name__ == '__main__':
unittest.main()
The next example shows a similar test using Typescript in the Jest framework. The code uses the Jest expect function to compare the actual and expected values of the resource properties.
import { App } from 'cdktf';
import { S3Bucket } from './s3-bucket';
import { TerraformStack } from 'cdktf/provider/terraform';
describe('S3Bucket', () => {
it('creates bucket with correct configuration', () => {
const app = new App();
const stack = new TerraformStack(app, 'test-stack');
const bucketName = 'my-test-bucket';
const lifecycleRules = [
{ expiration: { days: 90 }, id: 'log-files', prefix: 'logs/' },
{ expiration: { days: 120 }, id: 'backup-files', prefix: 'backup/' },
];
const accessControlPolicy = {
actions: ['s3:GetObject'],
effect: 'Allow',
principals: [{ service: 'lambda.amazonaws.com' }],
resources: [`arn:aws:s3:::${bucketName}/backup/*`],
};
new S3Bucket(stack, 's3-bucket', {
bucketName,
lifecycleRules,
accessControlPolicy,
region: 'eu-west-2',
});
const config = app.synth().getStackArtifact(stack.artifactId).template;
expect(config.Resources[bucketName].Type).toEqual('AWS::S3::Bucket');
expect(config.Resources[bucketName].Properties.BucketName).toEqual(bucketName);
expect(config.Resources[bucketName].Properties.LifecycleConfiguration.Rules).toEqual(lifecycleRules);
expect(config.Resources[bucketName].Properties.PublicAccessBlockConfiguration).toEqual({
BlockPublicAcls: true,
BlockPublicPolicy: true,
IgnorePublicAcls: true,
RestrictPublicBuckets: true,
});
expect(config.Resources[bucketName].Properties.CorsConfiguration.CorsRules).toEqual([]);
expect(config.Resources[bucketName].Properties.BucketEncryption.ServerSideEncryptionConfiguration).toEqual([]);
expect(config.Resources[bucketName].Properties.AccessControl).toEqual('Private');
expect(config.Resources[bucketName].Properties.AnalyticsConfigurations).toEqual([]);
expect(config.Resources[bucketName].Properties.InventoryConfigurations).toEqual([]);
expect(config.Resources[bucketName].Properties.LoggingConfiguration).toEqual({
DestinationBucketName: bucketName,
LogFilePrefix: 's3-logs/',
});
expect(config.Resources[bucketName].Properties.ObjectLockConfiguration).toEqual({
ObjectLockEnabled: 'Enabled',
});
expect(config.Resources[bucketName].Properties.VersioningConfiguration).toEqual({ Status: 'Enabled' });
expect(config.Resources[bucketName].Properties.WebsiteConfiguration).toEqual({});
});
});
End-to-end testing involves testing your infrastructure configuration in a production-like environment to ensure that it works as expected in a real-world scenario. This can be done using tools like Terratest or manual testing in a staging environment.
Continuing with the Azure storage account example, let’s look at how we can use Terratest to implement end-to-end testing.
- Create a Terraform configuration file that defines an Azure Storage account and container:
main.tf
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "test" {
name = "test-resource-group"
location = "uksouth"
}
resource "azurerm_storage_account" "test" {
name = "teststorageaccount"
resource_group_name = azurerm_resource_group.test.name
location = azurerm_resource_group.test.location
account_tier = "Standard"
account_replication_type = "ZRS"
tags = {
environment = "test"
}
}
resource "azurerm_storage_container" "test" {
name = "testcontainer"
storage_account_name = azurerm_storage_account.test.name
container_access_type = "private"
}- Create a Terratest test file to deploy the Terraform configuration and validate the created Azure Storage account exists, along with testing that the container has the correct name and access type.
storage_test.go
package test
import (
"context"
"fmt"
"os"
"testing"
"github.com/Azure/azure-sdk-for-go/profiles/latest/storage/mgmt/storage"
"github.com/gruntwork-io/terratest/modules/azure"
"github.com/gruntwork-io/terratest/modules/terraform"
"github.com/stretchr/testify/assert"
)
func TestAzureStorageAccountAndContainer(t *testing.T) {
t.Parallel()
// Define the Terraform options with the desired variables
terraformOptions := &terraform.Options{
// The path to the Terraform code to be tested.
TerraformDir: "../",
// Variables to pass to our Terraform code using -var options
Vars: map[string]interface{}{
"environment": "test",
},
}
// Deploy the Terraform code
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
// Retrieve the Azure Storage account and container information using the Azure SDK
storageAccountName := terraform.Output(t, terraformOptions, "storage_account_name")
storageAccountGroupName := terraform.Output(t, terraformOptions, "resource_group_name")
containerName := terraform.Output(t, terraformOptions, "container_name")
// Authenticate using the environment variables or Azure CLI credentials
azClient, err := azure.NewClientFromEnvironmentWithResource("https://storage.azure.com/")
if err != nil {
t.Fatal(err)
}
// Get the container properties
container, err := azClient.GetBlobContainer(context.Background(), storageAccountGroupName, storageAccountName, containerName)
if err != nil {
t.Fatal(err)
}
// Check that the container was created with the expected configuration
assert.Equal(t, "testcontainer", *container.Name)
assert.Equal(t, "private", string(container.Properties.PublicAccess))
fmt.Printf("Container %s was created successfully\n", containerName)
}- Run the test by navigating to the directory where the go file is saved and run the following command.
go test -vLinting is the process of checking source code for syntax and style errors.
- Terraform fmt: This is a built-in Terraform command that should be the first port of call. The command formats your Terraform code based on a set of standard formatting rules.
- tflint: This is a popular open-source tool that checks for syntax errors, best practices, and code style consistency. Once installed, simply run it using the command:
tflint /path/to/terraform/code- Checkov: This is an open-source static analysis tool for Terraform that checks for security and compliance issues in your Terraform code. Install it using the python package manager pip and run it using the command below:
pip install checkov
checkov -d /path/to/terraform/code
Checkov will identify security issues and provides recommendations for how to fix the issue, along with the location of the relevant code, such as publically accessible storage accounts.
- Terrascan: This open-source tool performs static code analysis to identify security vulnerabilities and compliance violations in Terraform code. Example output is shown below for a publically accessible storage account:
=== [azure_storage_account] ===
Rules:
Risky network access configuration for storage account
Rule ID: AWS_3_2
Description: A storage account with unrestricted network access configuration may result in unauthorized access to the account and its data.
Severity: CRITICAL
Tags: [network,storage]
Status: FAILED
Resource:
Check out the list of other popular tools used in Terraform-managed deployments.
terraform-compliance enables you to write a set of conditions in YAML files that must be met and test your code against them.
It can easily be installed using pip and run using the command shown below:
pip install terraform-compliance
terraform-compliance -p /path/to/policy/file.yaml -f /path/to/terraform/code
For example, the YAML file below specifies that Azure Storage Account should not be publicly accessible:
controls:
- name: azure-storage-not-publicly-accessible
description: Azure Storage Account should not be publicly accessible
rules:
- azure_storage_account:
public_access_allowed: FalseTerraform will natively test for drift between your code and the real infrastructure when terraform plan is run. Terraform will compare the current state of your infrastructure to the state saved in the state file.
If there are any differences, Terraform will display an execution plan that shows how to update your infrastructure to match your configuration.
You can also make use of driftctl which is a free open-source tool that can report on infrastructure drift. Example output from the tool is shown below:
Found 11 resource(s) – 73% coverage
– 8 managed by terraform
– 2 not managed by terraform
– 1 found in terraform state but missing on the cloud providerPeriodic monitoring of the IaC-managed infrastructure to proactively check for drifts is a challenge. Drift detection provided by Spacelift helps with identifying and highlighting infrastructure drifts on time. This makes it easy to know what has changed and provides us with the direction to investigate. Configuring a drift monitor is as simple as configuring a cron job. Read more in the documentation.
By combining the testing strategies mentioned in this article and taking a holistic approach, you can ensure that your Terraform code is well-tested and reliable and that changes are deployed quickly and safely.
We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this.
Manage Terraform Better with Spacelift
Build more complex workflows based on Terraform using policy as code, programmatic configuration, context sharing, drift detection, resource visualization and many more.
