What are Terraform Workspaces? Overview with Examples

I recently wrote an article providing an introduction to Terraform, which goes through its benefits and features and offers a step-by-step explanation with an example. The general IaC development for AWS using Terraform follows the steps below:

1. Configure AWS CLI and credentials on Terraform host machine.
2. Configure and initialize AWS provider in Terraform.
3. Write Terraform configuration for the AWS resources to be created.
4. Configure a backend for state management that enables team collaboration.
5. Run terraform plan command to validate the configuration and generate a summary of changes that will be applied.
6. Run terraform apply command to actually apply the changes in configuration.
7. Run terraform destroy command to destroy all the AWS resources managed through Terraform configuration.

In this post, we will cover another interesting feature of Terraform – Workspaces. Workspace is a way to maintain multiple copies of deployments that can be created and destroyed on the go.

Terraform workspaces enable us to manage multiple deployments of the same configuration. When we create cloud resources using the Terraform configuration language, the resources are created in the default workspace. It is a very handy tool that lets us test configurations by giving us flexibility in resource allocation, regional deployments, multi-account deployments, and so on.

The information about all the resources managed by Terraform is stored in a state file. It is important to store this state file in a secure location. Every Terraform run is associated with a state file for validation and reference purposes. Any modifications to the Terraform configuration, planned or applied, are always validated first with references in the state files, and the execution result is updated back to it.

If you are not consciously using any workspace, all of this already happens in a default workspace. Workspaces help you isolate independent deployments of the same Terraform config while using the same state file.

Check out our guide on How to Manage Multiple Terraform Environments Efficiently.

Terraform workspace vs. Terraform module

Terraform workspaces and Terraform modules are two different concepts that serve different purposes in the Terraform ecosystem. Workspaces allow users to manage different sets of infrastructure using the same configuration by isolating state files. Modules, on the other hand, are a logical container for multiple resources that are used together, facilitating reusability and better organization of your code.

To begin, let’s look at the options available to us in the help:

The options are quite straightforward here. We can use the workspace command to list all the available workspaces and show the currently selected one. We can also create new workspaces and delete old ones. Finally, to navigate through workspaces, we use the select command.

1. Create an EC2 instance

For the sake of this blog post, let us consider a simple Terraform config that creates an EC2 instance with the configuration below. We are currently using three variables for AMI value, the type of instance to be created and the name tag.

If we run terraform plan command at this point, it will show that it needs to create one resource, i.e. an EC2 instance. When the resource is created, the state file is updated with its information and other attributes.

Go ahead and create this EC2 instance. For reference, I am creating a t2.micro instance with Ubuntu 20.04 image.

As we can see from the output, the EC2 instance was successfully created. Run the plan command again, and see if Terraform wants to perform any additional actions at this point. It probably won’t.

2. Run terraform workspace show

To check the current workspace we are in, run the command below.

The output here shows that we are currently in the workspace named default.

3. Run terraform workspace list

To be sure that no other workspaces currently exist, run the list command as shown below.

The list command lists all the currently created workspaces, including the default workspace. The start mark beside the default workspace indicates the currently selected workspace we are in.

4. Create a new workspace

Let us create another workspace and select the same. We can do this by running the new command with the desired name of the new workspace as below.

Here, I have selected the name of the new Terraform workspace as “test_workspace”. Note that running this command has created the new workspace and switched to it.

5. Verify the setup

We can verify this selection is made by running the show command as below.

Of course, another way to verify it would be to run the list command and see where the asterisk (*) is pointing to.

When we create a new workspace, Terraform creates a corresponding new state file in the same remote backend that is configured initially. It is important to note that the backend being used should also be able to support the workspaces.

In this example, I have used the AWS S3 bucket as the remote backend.

When we look at the contents of the Terraform state S3 bucket, apart from our default terraform.tfstate file, we can see that a new directory named “env:/” is created, within which another directory with the name of our workspace (test_workspace) is created. A new terraform.tfstate file is maintained at this location.

Ignore the other details in the screenshot below. The Key column is relevant here.

Looking closely, the size of the default state file is considerably larger than that of the custom workspace-specific state file. This shows that the new state file is created, but it does not hold any information from the default state file. This is how Terraform creates an isolated environment and maintains its state file differently.

The contents of the test_workspace state file before running terraform apply are shown below:

As a result of this, if we run the plan command in the same directory now, Terraform will consider the state file as per the selected workspace. No resources are captured or maintained in this state file, so it will propose creating a new EC2 instance.

Note: The plan output does not specify the workspace information it is using while planning, so be sure to be very cautious while applying these changes, as using the wrong workspace may break the existing working environment.

Despite creating an EC2 instance using the same configuration in the default workspace, Terraform disregards its existence in a new workspace. This creates many possibilities for how infrastructure management may happen in various environments. The isolated nature of the Terraform workspace is used to test out modifications to the existing configuration before applying them to the critical environment, but this is just one of the use cases.

To delete the workspace, first select a different workspace. In our case, we go back to the default workspace and run the delete command. Terraform does not let us delete the currently selected workspace.

The corresponding directory structure in our S3 backend is deleted along with the state file.

Also, if you attempt to delete a workspace where certain resources are being managed by Terraform, it will not let you delete that workspace, suggesting using the -force option instead.

Using the -force option may not be a good idea as we will lose track of all the resources being managed by Terraform. A better option would be to select that workspace, run the destroy command, and then attempt to delete the workspace again.

Note: Default workspace cannot be deleted.

As an additional point, if you don’t want to manage Terraform state, Spacelift can help overcome common state management issues and adds several must-have features for infrastructure management. It offers an optional sophisticated state backend synchronized with the rest of the application to maximize security and convenience. Read more about the state management.

Managing variables with Terraform workspaces is essential when you need different configurations for different environments, like dev, test, stage, and prod. 

First, you need to declare the variables as you would normally do for any Terraform configuration. Providing values to these variables can be done easily by using tfvars files.

For each environment, you can declare a tfvars file:

Based on the workspace you are on (let’s suppose you are on the dev workspace), you will run an apply like:

You can also conditionally assign values to different parameters based on the workspace. Let’s take a look at an example:

The above code will assign a t2.large value to the instance_type local variable if the workspace is prod or a t2.micro, if there is any other workspace selected.

There is also an option to set up environment variables conditionally based on the workspace you are on, but this implementation uses a differentwill take advantage of another scripting language (bash/powershell/python), and you will need to create the logic of the script yourself. That’s why using multiple tfvars files makes the most sense in this approach.

As a best practice, wherever possible, you should assign default values to your variables, especially when you are working with workspaces, to avoid repeating code in the tfvars files. This will make your configuration less error-prone.

With the basics of Terraform workspaces in the background, it makes sense to use this knowledge within the Terraform configuration objects to identify the resources belonging to the respective workspaces.

For example, the EC2 instances created using the same configuration in the previous example are created with the same name, i.e., whatever value is specified in the name_tag variable. When we look at these instances in the AWS console, it becomes difficult to quickly identify which EC2 instance belongs to which workspace.

Terraform provides an interpolation sequence to reference the value of the currently selected workspace, as shown below:

Let’s use this to set our name tags according to the respective workspace being selected. In the configuration below, we have set our name_tag variable with a default value of EC2. The aws_instance resource block uses this variable in combination with the workspace interpolation sequence to set different and respective names.

Note: the format() function is used to concatenate multiple strings to for a valid name value.

We deleted the workspace from the previous section, so let’s also create a new workspace named “test” and create EC2 instances in both the workspaces – default, and test. See the console output below:

Here, two EC2 instances were created using the same configuration but in different workspaces. To validate if the interpolation sequences worked, log in to the AWS console and verify the names of the newly created EC2 instances.

As we can see, the names are set as expected, and now we can easily identify which instance belongs to which Terraform workspace.

Using the Terraform workspace enables us to isolate the infrastructure management of production and sub-production environments, we can also leverage the workspace interpolation sequence to allocate appropriate resources to them. 

This helps us avoid the unnecessary costs of creating transient sub-production environments, as these can be the scaled-down version of the original configuration. With workspace interpolation sequence and conditions, configurations are improved, as shown below.

In the example shown below, we have used the workspace interpolation sequence to define the number of EC2 instances to be created based on the workspace selected. If the default workspace is selected, the given configuration would create three instances, and for all other workspaces, it would just create a single instance.

Furthermore, corresponding changes are made to the Name tag to include the count index to distinguish between multiple instances. When we apply this configuration in the default and test workspace (which we created in the last section), we should then be able to see the following instances with names:

1. EC2_default_0
2. EC2_default_1
3. EC2_default_2
4. EC2_test_0

Let’s repeat the steps to apply this configuration in both workspaces as described in the console output of the previous section. The screenshot below shows the corresponding EC2 instances created in default and test workspaces.

Thus we have been able to limit the resource utilization of transient environments using interpolation sequence for Terraform workspaces. Similarly, we can leverage the concept of workspaces with more specific use cases.

You shouldn’t confuse branches in the version control systems with Terraform workspaces. Both have different purposes.

Git branches maintain various versioned copies of the same configuration used to develop new features or Terraform modules, whereas workspaces completely depend upon the state file maintained in the remote backend by Terraform.

In general, it is not recommended to use feature branches for deployments in the default workspace. The table below summarizes the impact of various combinations. It assumes that:

1. The Terraform configuration is maintained in a Git repository
2. Workspaces are used to create replica sets for debugging or developmental purposes
3. The remote backend is configured for Terraform workflow

When working with Terraform, if workspaces are used, they take precedence over the version control strategy.

As discussed in the previous section, introducing workspaces in the Terraform workflow, along with existing Git practices, also increases the risk of human error. If the team is not well-versed in using workspaces and branches in conjunction, the chances of wrong infrastructure deployments are high.

As we have seen before, the workspaces create a separate working directory structure to store state files. This also means that the plugins and modules are cached separately for each workspace. In a team where developers may create their own workspaces to test their changes, this can cause bandwidth and space issues on the remote backend host.

Workspaces are best used to test the changes in an isolated replica of infrastructure just before the production deployment. They are meant to be temporary and may not be the best solution to manage multiple staging environments since organizations usually want these environments to be strictly separate.

To improve your Terraform workflow, check out these 20 Terraform Best Practices.

Note: New versions of Terraform will be placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that will expand on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6. OpenTofu retained all the features and functionalities that had made Terraform popular among developers while also introducing improvements and enhancements. OpenTofu works with your existing Terraform state file, so you won’t have any issues when you are migrating to it.

We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features right out of the box. You can check it for free by creating a trial account or booking a demo with one of our engineers.