How to Manage Terraform State with GitLab [Tutorial]

Managing Terraform state is crucial for ensuring the consistency and integrity of your infrastructure. GitLab can be used as a version control system to store and manage your Terraform state files.

In this article, we will look at what GitLab CI/CD is, what features it brings to the table, and reasons why you might want to use it, before taking a look at how to use GitLab to manage your Terraform state files with some example configuration files.

What we will cover:

1. What is GitLab CI/CD?
2. What is a Terraform state file?
3. Why use GitLab-managed Terraform state? 
4. How to use GitLab to manage Terraform state
5. Managing Terraform state with Spacelift

CI/CD stands for Continuous Integration/Continuous Delivery and is a continuous method of software development.

GitLab CI/CD provides a .gitlab-ci.yml file at the root of your project where you can define your CI/CD pipeline configuration (it can be named as anything you wish but this is the most common name). This file follows the YAML format, has its own syntax, and specifies the stages, jobs, and scripts needed to build, test, and deploy your application. The CI/CD pipeline is triggered automatically whenever changes are pushed to the repository.

GitLab Runners are used to execute the CI/CD jobs. These runners can be distributed across different machines. A container image you want to use when running the job can be specified in the .gitlab-ci.yml file, the runner then loads the image, clones your project, and runs the job either locally or in the container.

Jobs define what you want to do with your pipeline. For example, to run a plan stage in your Terraform or deploy to a production environment. Jobs are grouped into stages. Each stage contains at least one job. Typical stages using a Terraform workflow might be init, plan, and deploy. Jobs within the pipeline can be executed in parallel, optimizing the build and test times.

GitLab CI/CD (and other CI/CD systems) allow you to store and manage artifacts generated during the build process, which can be reused in subsequent stages.

Variables used in your pipelines can be defined directly in the .gitlab-ci.yml file, set in your GitLab project settings, or dynamically generated. Using GitLab CI/CD environment variables or HashiCorp Vault for sensitive information like API tokens and credentials is recommended. Also, ensure that appropriate access controls are in place within your GitLab repository.

A simple example .gitlab-ci.yml file for use with Terraform could look like this:

Or you can use a GitLab provided template for your base configuration.

Terraform stores information about the infrastructure it manages in a state file. This file tracks the current state of your infrastructure, including the resources Terraform has provisioned and their current configurations. The state file is essential for Terraform to plan and apply changes to your infrastructure.

• The state file is persistently stored on disk and typically has a .tfstate extension.
• Terraform uses a state file lock to prevent concurrent operations that might modify the state. This ensures that only one Terraform operation can be applied at a time, preventing conflicts.
• The state file may contain sensitive information such as passwords or private keys (which can be mitigated by using the sensitive variables parameter when defining your variable). It’s important to limit access to the state file and avoid exposing it to unauthorized users.
• Terraform “backend configuration” refers to where the state file is stored. It can be set to use local storage, or remote storage such as AWS S3 or Azure Storage account. It can also be stored on hosted platforms such as a Terraform Enterprise instance, Spacelift or GitLab.

Read more about managing Terraform state.

GitLab provides a Terraform HTTP backend to securely store your state files in a remote and shared store with minimal configuration.

Benefits of managing Terraform state with GitLab include:

• Ability to version your Terraform state files since they are held in a repository. You can track changes over time, roll back to previous states, and collaborate with team members using Git’s branching and merging capabilities. Storing Terraform state in a version-controlled system aligns with best practices for infrastructure as code.
• GitLab provides a centralized repository for storing and version-controlling your Terraform state.
• Encrypt the state file both in transit and at rest.
• You can control who has access to the state files and ensure that sensitive information is handled securely.
• GitLab’s collaboration features enable multiple team members to work on the same Terraform codebase concurrently. With merge requests and code reviews, you can ensure that changes are reviewed and approved before being applied to the infrastructure.
• Lock and unlock states.
• Remotely execute terraform plan and terraform apply commands.

Before we jump into the example:

• You must enable the Infrastructure menu for your project. Go to Settings > General, expand Visibility, project features, permissions, and under Infrastructure, turn on the toggle.
• An administrator must set up Terraform state storage.
• To lock, unlock, and write to the state by using terraform apply, you must have at least the GitLab Maintainer role.
• To read the state by using terraform plan -lock=false, you must have at least the GitLab Developer role.

An example backend configuration might look like this:

backend.tf

As you can see, GitLab automatically provides an HTTP backend. The Terraform official docs provide more information about the options you can configure with this type of backend.

To view and manage individual Terraform state versions or remove state files in GitLab:

1. On the left sidebar, select Search or go to and find your project.
2. Select Operate > Terraform states.

Note that GitLab provides two helpers to ease your integration with the GitLab-managed Terraform State, which you can use if you want.

• The gitlab-terraform script, which is a thin wrapper around the terraform command.
• The terraform-images container images, which include the gitlab-terraform script and terraform itself.

Read more: How to Implement GitLab CI/CD Pipeline with Terraform

If you need remote state management and all the caveats you require to manage your infrastructure automation, Spacelift is the platform for you.

Spacelift can optionally manage the Terraform state for you, offering a backend synchronized with the rest of the platform to maximize convenience and security. You can also import your state during stack creation, which is very useful for engineers who are migrating their old configurations and states to Spacelift.

Behind the scenes, Spacelift uses Amazon S3 and stores all the data in Ireland. It’s super simple to have Spacelift manage the state for you, as this behavior is achieved by default without you needing to do anything. You can read more about how it actually works here.

At the same time, it’s protected against accidental or malicious access as Spacelift is able to map state access and changes to legitimate Spacelift runs, automatically blocking all unauthorized traffic.

Apart from being able to manage your state, you can easily integrate with OPA policies, build self-service infrastructure, easily detect and remediate drift, and also build end-to-end workflows that share outputs.

If you are interested in learning more about Spacelift, create a free account today or book a demo with one of our engineers.

GitLab CI/CD is a powerful tool that automates the software development lifecycle, ensures code quality, and streamlines the process of delivering software changes to production. Using it to manage your Terraform state file brings a key component of your infrastructure-as-code deployments under the same roof as your code repositories and pipelines and the versioning, security, and management benefits of this along with it.

We encourage you also to explore how Spacelift makes it easy to work with Terraform.

Note: New versions of Terraform will be placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that will expand on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6. OpenTofu retained all the features and functionalities that had made Terraform popular among developers while also introducing improvements and enhancements. OpenTofu works with your existing Terraform state file, so you won’t have any issues when you are migrating to it.