How to Share Terraform State (Tutorial)

Keeping all your cloud infrastructure in a single Terraform configuration could be considered the happy path of Terraform development. This is often how you learn Terraform, following tutorials and performing your own private experiments.

Inside your mono-configuration, you can freely reference attributes of any resource when needed. Everything is simple.
Real-life Terraform often involves multiple teams working on multiple Terraform configurations within the same larger context. Along with this way of working comes the need to share Terraform state between different configurations.

In this blog post, you will learn about Terraform state and how to share Terraform state between configurations. You will also learn about the best practices of Terraform state sharing.

What we’ll cover:

1. What is Terraform state?
2. Why share Terraform state?
3. How to share Terraform state
4. Best practices when sharing Terraform state

Terraform configurations contain the code that describes the infrastructure you want to provision. This code is declarative and written in the HashiCorp Configuration Language (HCL).

Terraform is a provider-agnostic tool and works in a unified way across a vast ecosystem of providers, including the big cloud providers Amazon Web Services, Microsoft Azure, and Google Cloud Platform.

To do its job, Terraform must store metadata for the resources it manages across these vastly different provider ecosystems in a unified way. This is the Terraform state.

The state is a record of the resources and their dependencies to each other that Terraform manages. It allows Terraform to perform updates to resources in the correct order, even across provider ecosystems.

The state is stored as plain JSON in the Terraform state file. In this JSON data, you can find all your resources and their attributes, dependencies, and other metadata, along with output values and check block results.

Before we discuss how to share Terraform state, we should understand why we would want to share it in the first place.

A monolithic Terraform configuration

The easiest way to work with Terraform is to include all your infrastructure in a single monolithic configuration. The huge benefit with this approach is that you can easily reference any attribute from any resource from anywhere within the Terraform configuration. There is no need to bring in external dependencies that complicate everything.

Using a large mono-configuration also has its drawbacks. These include the negative performance implication and the vast potential blast radius when something goes wrong.

Splitting infrastructure into smaller pieces

To combat these drawbacks, you can split your Terraform configuration into multiple smaller pieces. The infrastructure for a given application could be split into dedicated parts for networking, identity, compute, and data storage. Each dedicated part follows its own update cadence and lifecycle.

When you split your infrastructure into multiple pieces, you must often find a way to share details of your infrastructure between the different Terraform configurations. A typical example is when you have defined all your networking resources in a separate configuration, and you want to use these resources when creating compute or database instances in your other Terraform configurations.

A common organizational example

Another common situation is when your organization has a central platform team responsible for managing a large shared network infrastructure (or other infrastructure components).

Each development team needs to connect its infrastructure to this larger network infrastructure and requires attributes and metadata only available in the platform teams’ state files to do so.

These kinds of situations lead teams to share state files between their own Terraform configurations and the Terraform configurations other teams manage.

You can share state in a few different ways. Some of these depend on the specific automation platform that you are using (e.g. Spacelift or HCP Terraform), and some are more generic.

In the following discussion, we focus on the generic options for sharing state using Terraform, without focusing on how third-party tools or platforms can help.

From the previous section, we know that sharing state involves sharing data about the resources we manage in a specific Terraform configuration with other configurations. This creates a dependency between two or more Terraform configurations. This dependency can be implicit or explicit.

• Implicit dependencies involve a loose coupling between two or more Terraform configurations. The consuming configuration knows a few details about the producing configuration but does not necessarily know where, how, or by whom the producing configuration is managed.
• Explicit dependencies involve a stronger coupling between two or more Terraform configurations. The consuming configuration often must know explicit details about where the producing configuration’s state file is stored and must specify these details in code.

In the following sections, we will see examples of both implicit and explicit options for how to share state between two Terraform configurations.

Using data sources

An implicit way to share Terraform state without directly involving the state file is to use data sources.

Imagine we manage the following AWS VPC resource in a Terraform configuration:

resource "aws_vpc" "shared_platform" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name = "vpc-shared-platform"
  }
}

We can use an AWS VPC data source in a different Terraform configuration to read the same VPC:

data "aws_vpc" "shared_platform" {
  filter {
    name   = "tag:Name"
    values = ["vpc-shared-platform"]
  }
}

Now we can reference the data source to obtain any attribute value of the VPC resource.

Using data sources creates an implicit dependency between the two Terraform configurations. We can manage both Terraform configurations independently without worrying about giving the consuming configuration read-access to the state file.

Using output values or local files

A different way to share state between two Terraform configurations is to use implicit dependencies: Pass values between them via outputs and variables.

In the producer configuration, we add an output value for the attribute we want to share:

output "vpc_id" {
  value = aws_vpc.platform.id
}

In the consuming configuration, we add a corresponding variable:

variable "vpc_id" {
  type = string
}

Now we use the underlying automation platform to pass values from the producer configuration to the consumer configuration. A pseudo-example of how to do this using terminal commands is shown below (The exact details depend largely on which platform you use.):

$ cd producer
$ vpc_id=$(terraform output -raw vpc_id)
$ cd ../consumer
$ terraform plan -var="vpc_id=$vpc_id"
$ ...

An alternative way of doing this is to use the local provider to generate a Terraform variables file that the consuming configuration can use:

resource "local_file" "terraform_tfvars" {
  filename = "../consumer/terraform.tfvars"
  content = <<-EOF
    vpc_id = "${aws_vpc.platform.id}"
  EOF
}

Using either of the two options discussed above is best when the consuming and producing configurations are managed by the same team, ideally even within the same automation workflow.

If two different teams are involved, neither of these options will be ideal.

Sharing the actual state file with terraform_remote_state

If neither of the above methods are feasible, you can also share access to your state file and allow other Terraform configurations to read it.

This method is based on the terraform_remote_state data source. This data source has the following general configuration:

data "terraform_remote_state" "team_b" {
  backend = "<backend type>"

  config = {
    # backend configuration
  }
}

In the following example, we assume that an organization has two teams named Team A (the producers) and Team B (the consumers), and both use an AWS S3 remote backend for state storage.

Team A (the producers) use the following backend configuration for state storage:

terraform {
  backend "s3" {
    bucket = "spacelift"
    region = "eu-west-1"
    key    = "teams/team-a/terraform.tfstate"

    assume_role = {
      role_arn = "arn:aws:iam::123456789012:role/team-a"
    }
  }
}

Specifically, they use their own IAM role that has permissions to manage their own state files (read, write, and delete).

In their Terraform configuration, they have an AWS VPC resource and an output for its ID:

resource "aws_vpc" "default" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name = "vpc-team-a"
  }
}

output "vpc_id" {
  value = aws_vpc.default.id
}

Team B (the consumers) has configured their backend similar to the other team:

terraform {
  backend "s3" {
    bucket = "spacelift"
    region = "eu-west-1"
    key    = "teams/team-b/terraform.tfstate"

    assume_role = {
      role_arn = "arn:aws:iam::123456789012:role/team-b"
    }
  }
}

The difference is that they use their own IAM role that has permissions to read, write, and delete their own state files. In addition, the role has permission to read Team A’s state file.

With this additional permission, they can use the terraform_remote_state data source to read Team A’s state file and reference outputs from their configuration:

data "terraform_remote_state" "team_a" {
  backend = "s3"

  config = {
    bucket = "spacelift"
    region = "eu-west-1"
    key    = "teams/team-a/terraform.tfstate"

    assume_role = {
      role_arn = "arn:aws:iam::123456789012:role/team-b"
    }
  }
}

resource "aws_subnet" "team_a" {
  vpc_id     = data.terraform_remote_state.team_a.outputs.vpc_id
  cidr_block = "10.0.0.0/24"
}

Note the following details:

• The terraform_remote_state data source is configured similarly to how you configure the backend block for state storage.
• Team B uses their own IAM role in the configuration of the terraform_remote_state data source.
• Outputs from the producing configuration can be referenced from the terraform_remote_state data source through data.terraform_remote_state.team_a.outputs.<output name>.

Now, when Team B runs terraform apply the VPC ID is read from the producing configuration.

The terraform_remote_state data source can only reference outputs from the configuration that shares its state file. However, it is important to realize that Team B has been given AWS IAM permissions to read the state file blob in the S3 bucket.

In practice, this means that Team B can download the full state file and access all of the information within the file, including any sensitive values.

Keep the following best practices in mind when sharing your state between Terraform configurations.

1. Minimize sharing

It might sound counterintuitive to list “minimize sharing” as a best practice for sharing Terraform state, but the less state you share between Terraform configurations the easier your life will be.

Sharing state between Terraform configurations you manage yourself gives you the most control. You can design your configurations to minimize sharing as much as possible, potentially even use a mono-configuration with all of your infrastructure to remove state sharing completely.

It is difficult to completely remove state-sharing between different teams, but it is not impossible. Try to identify opportunities for removing state sharing wherever possible, even in a multi-team context.

2. Treat shared state as a contact

Sharing state with another Terraform configuration creates a dependency between the two configurations. If you change the names or formats of the outputs you exposed to a consuming configuration, you will likely break their infrastructure.

Once you have shared state with one or more consumers you should treat any changes to your outputs similar to how you would treat changes to an exposed API or a shared Terraform module.

Introducing new outputs is safe, but you should deprecate old outputs and give consumers details for how to upgrade to a different output value and give them sufficient time to perform this change.

3. Prefer to share stable data

Ideally, the state you share between Terraform configuration should be stable and long-lived. One example of a long-lived value is an AWS VPC ID. As long as the VPC is in use, the ID stays the same.

If you share data that changes often (e.g., an allow/deny list for a firewall or similar) you force the consuming configuration to re-run as often as the shared state is updated. If this is the case, it may be necessary to have an automatic trigger that runs the consuming Terraform configuration if the upstream state changes.

4. Prefer data sources or platform features for sharing state

If you use GitHub Actions or an equivalent automation platform that lacks specific Terraform features, you should use data sources to share state between Terraform configurations. With data sources, you avoid giving away complete access to your state file and all of its contents.

However, if you use an automation platform that is built for Terraform (e.g., Spacelift or HCP Terraform), you can use platform-specific features for sharing state between different Terraform configurations. This is often the best approach and allows you to be more explicit with what you share.

5. Do not give away write access to your state file

When you share your state file with others, restrict them to read access only. Giving away write access to the file is a recipe for future disaster.

In an earlier example, Team B’s IAM policy allowed them to manage their own state file(s) while simultaneously reading Team A’s state file.

The relevant portion of the policy for the IAM role that Team B used is shown below:

data "aws_iam_policy_document" "team_b" {
  # read team-a state file
  statement {
    effect = "Allow"
    actions = ["s3:GetObject"]
    resources = [     "${data.aws_s3_bucket.spacelift.arn}/teams/team-a/terraform.tfstate"
    ]
  }

  # manage team-b state files
  statement {
    effect = "Allow"
    actions = [
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject",
    ]
    resources = [
      "${data.aws_s3_bucket.spacelift.arn}/teams/team-b/*"
    ]
  }
}

Terraform is really powerful, but to achieve an end-to-end secure GitOps approach, you need to use a product that can run your Terraform workflows. Spacelift takes managing Terraform to the next level by giving you access to a powerful CI/CD workflow and unlocking features such as:

• Policies (based on Open Policy Agent) – You can control how many approvals you need for runs, what kind of resources you can create, and what kind of parameters these resources can have, and you can also control the behavior when a pull request is open or merged.
• Multi-IaC workflows – Combine Terraform with Kubernetes, Ansible, and other infrastructure-as-code (IaC) tools such as OpenTofu, Pulumi, and CloudFormation, create dependencies among them, and share outputs
• Build self-service infrastructure – You can use Blueprints to build self-service infrastructure; simply complete a form to provision infrastructure based on Terraform and other supported tools.
• Integrations with any third-party tools – You can integrate with your favorite third-party tools and even build policies for them. For example, see how to integrate security tools in your workflows using Custom Inputs.
• Secure state management and locking – Because Terraform and OpenTofu state is shared, preventing concurrent writes is essential. With a Spacelift-managed state, Spacelift injects a backend configuration for each run using one-time credentials, restricts state access to active runs and tasks, and stores state encrypted in Amazon S3. You also get state history and a break-glass rollback for rare cases of state corruption, such as after provider upgrades.

Spacelift enables you to create private workers inside your infrastructure, which helps you execute Spacelift-related workflows on your end. Read the documentation for more information on configuring private workers.

If you are interested in learning more about Spacelift, create a free account today or book a demo with one of our engineers.

Sharing Terraform state is often a necessary evil.

If you keep all your Terraform footprint in a single Terraform configuration, you can use a single Terraform state file with no need to share state between configurations. However, this mono-configuration approach comes with drawbacks of slow performance and a big blast radius when things go wrong.

Instead, you can split your infrastructure into multiple Terraform configurations. This is especially common when multiple teams in your organization are working on multiple different infrastructure components within the same larger environment.

You can share state implicitly through the use of data sources, output values, variables, or local files. You can also explicitly share your state file using the terraform_remote_state data source.

Sharing state means you introduce dependencies between configurations, and you can no longer evolve the producing configuration (the one that shares state) freely. Any updates to output values shared with consuming configurations must be handled carefully.

In the end, sharing state should be minimized. If it is required, use implicit methods of sharing state (data sources, output values and variables, etc) or platform-specific features (e.g. in Spacelift or HCP Terraform).