What is Terraform Lock File? .terraform.lock.hcl Explained

Once you get to the stage of a Terraform project where you are issuing terraform plan and terraform apply commands, your working directory will be populated by additional files and directories created by Terraform. One of these is the .terraform.lock.hcl file in the root of your working directory next to your Terraform configuration. This is the lock file, or dependency lock file.

In this post, we will explore the details of the Terraform lock file and why it is important.

What we will cover:

1. What is .terraform.lock.hcl file?
2. How does the Terraform lock file work?
3. How to version the Terraform lock file?
4. Can you delete the Terraform lock file?
5. Terraform lock file use cases 
6. Should .terraform.lock.hcl be included in the .gitignore file?

What does your Terraform configuration depend on? Unless you are using Terraform locals, variables, and outputs only, you are probably using one or more Terraform providers. A Terraform provider is a bridge between an external system and Terraform. Common external systems are cloud providers such as AWS, Azure, and GCP. You can also have providers things like your local firewall equipment or your source code repository provider.

Terraform also depends on modules. Terraform modules can be external, or they can be available in a local directory close to the rest of your Terraform configuration. An external module is a module that is published to the public Terraform registry, or to any other system implementing the Terraform registry API. In a later section, we will learn that the lock file does not currently concern itself with modules. For now, we will concentrate on providers.

If you have used any programming language before, you are probably familiar with the concept of a lock file:

• TypeScript and JavaScript developers working in the Node.js ecosystem are used to the package-lock.json file.
• Go developers are used to the go.sum file.

Similarly, Terraform has a lock file — the .terraform.lock.hcl file. In its current form, this file only keeps records of the provider versions you use.

Terraform providers follow a lifecycle independent of Terraform’s evolution. This means that each provider follows its own software development lifecycle (SDLC) with releases of new versions with improvements and bug fixes. The cadence of releases for a given provider is usually much higher than Terraform.

Provider versioning follows the semantic versioning scheme with a major version, a minor version, and a patch number. A given version is specified as <major>.<minor>.<patch>, e.g. 5.58.0 or 1.2.3. When you add providers to your Terraform configuration in the required_providers block, you may specify a version constraint. A version constraint specifies which provider version you want to use. A few examples of version constraints are:

• Specify an exact version to use, e.g. version = “5.58.0”.
• Specify a specific major version, but use the latest available minor and patch versions: version = “~> 5.58”. This example constraint allows for versions such as 5.58.1, 5.59.0, 5.61.1, etc. The major version is fixed, i.e. the version will follow the pattern 5.x.y.
• Specify a specific major and minor versions, but use the latest available patch version: version = “~> 5.58.0”. This example constraint allows for  versions such as 5.58.1, 5.58.2, 5.58.3, etc. The major and minor versions are fixed, i.e. the version will follow the pattern 5.58.x.
• Specify that you want any version greater than or equal to a specific version: version = “>= 5.58.0”. This is commonly used if a specific feature was introduced in a given version, and your Terraform configuration requires this feature. This example constraint allows for any version update, even major versions.

Best practice is to either use an exact version of a provider, or to allow patch version updates to take place. You should always be wary of introducing major and minor version updates and make sure you have proper testing in place to determine if you can update the provider.

The lock file is created by Terraform when you issue the terraform init command. During the initialization process Terraform performs a few operations, among them downloading the providers your configuration depends on.

Each published version of a provider comes as a binary file built for the specific system you run terraform init on. Each binary has a footprint in the form of a hash value. Details about the hash values for the binaries your Terraform configuration uses are recorded in the lock file. Together with the hashes, you will also see the exact version number of each provider, and the version constraint used to determine which version to download.

Here is a simplified example of what this looks like:

What is the difference between a Terraform state file and a lock file?

If you are new to Terraform, you might be confused initially about the various files that Terraform creates for you during the lifecycle of your Terraform configuration.

Arguably, the most important file Terraform creates appears when you issue the terraform apply command for the first time. This is the state file. The state file contains a record of all the resources your Terraform configuration has created. It is a snapshot of the world that your Terraform configuration knows about. The state file contains detailed descriptions of all resources and their properties, as well as a reference to what provider has been used to create each resource.

A simplified extract from the state file looks like this:

This is a state file with a single resource of type local_file. We can tell this resource type comes from the hashicorp/local provider. However, there are no additional details about the provider, such as what version was used to create the resource.

The lock file only concerns dependencies in the form of Terraform providers. It records the providers you are using and what versions of these providers are in use, and it records hash values calculated from the binary files of these providers. The lock file does not know anything about the infrastructure you create using Terraform.

Both the state file and the lock file should only be updated by Terraform, and should not be edited manually by hand. The state file is updated through CLI commands such as terraform apply and terraform state <mv|pull|rm> or the declarative equivalents available in HCL. The lock file is updated with the terraform init command when there are changes to the providers or provider versions that you use.

What about external modules?

Providers are one of two dependencies that your Terraform configuration might be using. The other is modules.

Modules come in two flavors: local and external. A local module is just another directory located somewhere relative to your current working directory. An external module is published to a Terraform-compliant registry. For the sake of the discussion of dependencies, we will only care about external modules.

An example of using an external module looks like this:

This particular example references the popular AWS VPC module for creating Virtual Private Clouds in AWS. As with providers, you specify a source and a version for external modules. You can also use version constraints instead of using an exact version.

In theory, external modules could be treated the same way as providers and be recorded by the lock file (in a module block instead of a provider block). However, this is not the case currently. An interesting open-source tool called TerraHash treats modules the same way that Terraform treats providers.

To illustrate how the Terraform lock file works, we will start from the following simple Terraform configuration in main.tf:

This configuration consists of the terraform block and one required provider and is currently using version 5.58.0 of the AWS provider (the current latest version as of this writing is 5.61.0). Later, we will see how to update the provider version and what this means for the lock file.

As with all new Terraform configurations, we start by issuing the terraform init command to initialize Terraform:

The output informs us that Terraform initializes our backend (the state storage), downloads the required providers, and creates the Terraform lock file .terraform.lock.hcl. If our configuration included external modules, they would have been downloaded in this step as well.

The contents of our working directory now include a new directory named .terraform, and the lock file .terraform.lock.hcl. Below is a snapshot of the current directory structure:

The name of the lock file indicates that .terraform.lock.hcl is a lock file for the contents of the .terraform directory. We see that the .terraform directory contains the downloaded provider binaries. The exact content of this directory will depend on your current system architecture. In this example, it is from a Mac system (darwin_arm64).

We can see the contents of the lock file in its current form:

The comment on the top of the file indicates that this file should not be edited manually. Terraform should handle any updates through the terraform init command.

Each provider your configuration uses has its own provider block in the lock file. The provider block has one label that is the registry URL of the provider (registry.terraform.io/hashicorp/aws). The arguments in the provider block specify what version of the provider is in use (version), the version constraint that was used to determine which version to download (constraints), and an array of hashes (hashes). To understand the lock file’s true purpose we will spend some time understanding this list of hashes.

Think of a hash as the output of a function given a certain input. The hash functions have two important properties

• Given the same input, they always produce the same output.
• Two different inputs will never produce the same outputs.

In our lock file, we see that each hash has a prefix of either h1: or zh:. These represent two different hashing schemes:

• zh is short for zip-hash, it is a legacy scheme. The input to the hash function is a zip file containing the provider binary for a given system architecture (e.g. darwin_arm64).
• h1 is short for hash-scheme 1, this is the currently recommended scheme. The input to the hash function is the contents of the provider zip file, but not the zip file itself.

In our lock file we have 16 different hash values, most of them with the zh prefix. Each zh hash value is calculated from each of the architectures for which this provider is built. It is common to build the provider for many different architectures to support as many different systems as possible.

Even if the zip-hash scheme (zh) is a legacy hashing scheme, it is easier to reason about, and it is still the most common one you will see. To understand how the hash value is computed for a given provider, we need to download the published zip file for the provider for our system architecture. Re-running the terraform init command from above with verbose logging shows us where the zip file can be obtained, note that most logs have been hidden for brevity:

We can manually download the provider binary and then compute the hash:

The hash function is the SHA-256 checksum of the zip file. Going back to our lock file we see that the same hash value can be found in the list of hashes for the AWS provider:

At this point, we have a lock file, and if we re-run terraform init we can be assured that the same provider version will be used, and there will be no surprising behavior. However, we will want to upgrade (or downgrade) the provider version at some point. The steps to achieve this are to first update the provider version in the list of required providers:

Next, we try to run terraform init, but we will quickly encounter an error:

The lock file specifies that this Terraform configuration should use version 5.58.0 of the AWS provider, but we are now trying to use version 5.61.0. The error message states that, we should add the -upgrade flag to the terraform init command:

The output informs us that changes have been made to the lock file and that we should review them and add them to our version control system. You can run a git diff command to review the changes that have been made, or create a pull request in your source code repository system and review the changes there. You will notice that all arguments in the provider block in the lock file have been updated: version, constraint, and hashes.

Where do the hashes come from? The provider publishes and signs them, and they are downloaded together with the provider binary. The steps of how a provider is published are beyond the scope of this post, but it is a good exercise to go through if you are interested in learning more!

You should keep your Terraform lock file together with the rest of your Terraform configuration in your version control repository. Your Terraform configuration uses specific versions of your providers, so the providers you use and your Terraform configuration should always match. 

As we saw in the previous section, when you update the version of a provider, the Terraform lock file is updated as well. Most provider updates will be minor or patch version updates. These updates might introduce backward-compatible changes, for instance introducing new features or resource properties that you might want to use. Major version updates of providers will most likely also come with backward-incompatible changes requiring you to update parts of your Terraform configuration. 

All changes in your Terraform configuration and lock file should be treated as a unit and bundled together in a peer-review process, such as pull request, pair programming, or whichever process you use.

You can delete the lock file without any immediate issues. However, there might be unintended consequences.

Issuing a new terraform init command will create a new lock file for you, so at first, it might not seem like a loss if this file is deleted. You can keep creating new lock files and move on to run terraform plan and terraform apply commands.

If you are using loose constraints for your providers (e.g. version = “> 5.58.0”), you could get a new minor version between one terraform apply command, and the next, or even a new major version of the provider. It is difficult to determine the consequences for your infrastructure, and these will vary depending on what providers you are using. Hopefully, you will notice if there are any differences from the output from the terraform plan command.

In summary, you can delete the lock file, but there are no good reasons to do so.

What is the lock file’s main purpose? At a high level, the Terraform lock file ensures that the same provider version is used across different systems and different Terraform runs.

When you run terraform init followed by a terraform plan and terraform apply with the same Terraform configuration on two different occasions, either separated in time or space, you should end up with the same end result.

The first time you run terraform init Terraform will create the lock file, specifying the exact versions of the providers that were downloaded. If someone or something else runs terraform init for the same Terraform configuration again, you should use the same provider versions to avoid surprise behaviors.

If an organization must host provider binaries in a private registry due to compliance reasons or because the automation system running Terraform lacks direct internet access. The automation system will not be able to download provider binaries from the public registry. This is an example of an air-gapped system, which is common in domains with strict regulations.

A developer who writes the initial Terraform configuration works from his or her own laptop with internet access and uses the public registry. The Terraform configuration works flawlessly on the developer’s system, and the configuration is pushed to production to be deployed by the air-gapped automation system. Using the lock file ensures the provider binary hosted in the private registry is the same as the provider binary from the public registry.

We could simulate this behavior with our sample Terraform configuration from above. We will outline the steps to do this will be briefly outlined as they are more for illustrative purposes. Start from the main.tf file we had in an earlier section:

Remove the existing .terraform directory but keep the lock file. This situation simulates that we have just checked out a working Terraform configuration and its corresponding lock file on a different system. Now add the following contents to your Terraform configuration file .terraformrc, usually located in your home directory:

This configuration says that any provider installations from registry.terraform.io/hashicorp/aws should be fetched from the local directory /usr/share/terraform/providers instead. 

Next, create the required directories:

Download the provider zip file, unzip the contents, and move it to the correct location:

Now, if we run terraform init the provider is downloaded from our local system instead of the public registry.

We can simulate what happens if the provider hash does not correspond to a value in the lock file by switching the binary to something else (e.g. download version 5.58.0 of the AWS provider, rename the binary file, and replace the corresponding binary in the 5.61.0 directory):

Terraform discovers that there is a mismatch in the hash values and stops the terraform init command from continuing. If you see this error message in real life it is prudent to thoroughly investigate what is going on.

The security-minded might have realized that the lock file could also catch attempts to introduce a compromised provider binary into your Terraform workflow. However, if the provider itself is compromised at the source, the attacker would also be in control of publishing the hashes for the providers.

Read more: 12 Terraform Security Best Practices (& 7 Common Risks)

For new users of Terraform, it is not clear whether the lock file should be included in the .gitignore file or not. However, let’s take another look at the output from the terraform init command:

Here we see that the recommendation is to include this file in your version control repository. Hence, you should not include it in the .gitignore file. If you include it, each terraform init command (either run manually or in a CI/CD pipeline) could potentially download different versions of your providers, leading to unexpected behaviors. At best, you would not notice any difference; at worst, there could be severe consequences for your infrastructure and even your organization.

Terraform is really powerful, but to achieve an end-to-end secure Gitops approach, you need to use a product that can run your Terraform workflows. Spacelift takes managing Terraform to the next level by giving you access to a powerful CI/CD workflow and unlocking features such as:

• Policies (based on Open Policy Agent) – You can control how many approvals you need for runs, what kind of resources you can create, and what kind of parameters these resources can have, and you can also control the behavior when a pull request is open or merged.
• Multi-IaC workflows – Combine Terraform with Kubernetes, Ansible, and other infrastructure-as-code (IaC) tools such as OpenTofu, Pulumi, and CloudFormation,  create dependencies among them, and share outputs
• Build self-service infrastructure – You can use Blueprints to build self-service infrastructure; simply complete a form to provision infrastructure based on Terraform and other supported tools.
• Integrations with any third-party tools – You can integrate with your favorite third-party tools and even build policies for them. For example, see how to Integrate security tools in your workflows using Custom Inputs.

Spacelift enables you to create private workers inside your infrastructure, which helps you execute Spacelift-related workflows on your end. For more information on configuring private workers, refer to the documentation.

Spacelift can also optionally manage the Terraform state for you, offering a backend synchronized with the rest of the platform to maximize convenience and security. You can also import your state during stack creation, which is very useful for engineers who are migrating their old configurations and states to Spacelift.

Behind the scenes, Spacelift uses Amazon S3 and stores all the data in Ireland. It’s super simple to have Spacelift manage the state for you, as this behavior is achieved by default without you needing to do anything. You can read more about how it actually works here.

It’s protected against accidental or malicious access because Spacelift can map state access and changes to legitimate Spacelift runs, automatically blocking all unauthorized traffic.

For more information, refer to this blog post, which shows in detail Spacelift’s remote state capabilities.

In this post, we have learned most of what we need to know about the Terraform lock file .terraform.lock.hcl. The key points to remember are:

• Terraform has two types of dependencies: providers and external modules.
• Providers are the bridge between Terraform and external systems exposed through an API (e.g. public cloud providers such as AWS, Azure, and GCP).
• Providers are published as binaries, and they are downloaded to your system when you issue the terraform init command.
• The lock file keeps a record of what provider versions have been downloaded and used with the current Terraform configuration. Subsequent Terraform runs should use the same provider versions to avoid surprising or damaging behaviors.
• Modules are not part of the .terraform.lock.hcl file.
• You specify what version of a provider you want to use through a version constraint. Terraform can update the provider version to newer versions allowed by the constraint if you run terraform init -upgrade. An upgrade of a provider will introduce corresponding changes in the lock file.
• Keep the lock file together with your Terraform configuration in your version control repository and be aware of any changes that are made to this file.
• Any changes to the lock file should only be performed by Terraform through the terraform init command. This is similar to how no manual changes should be made to the state file.

Note: New versions of Terraform are placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that expands on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6.