Author title

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

Spacelift Worker Pools on Kubernetes with Spacelift

Learn more about new features, updates, and Spacelift product announcements.

Product Features and Updates | Spacelift Blog

Product

You are probably a bit confused by the description of the subtitle; stay with me! If you have not seen the latest news, Spacelift supports Kubernetes! Check out the announcement blog and documentation.
This blog aims to demonstrate the power of Spacelift by providing a view of how Spacelift uses Spacelift to provision Kubernetes clusters, a private worker pool, and Kubernetes deployments all within the same organization and structure.
To summarize, we are looking at the following objectives for Spacelift:
<ul>
<li style="font-weight: 400;">Provision Kubernetes clusters
<ul>
<li style="font-weight: 400;">Application</li>
<li style="font-weight: 400;">Worker (pool)</li>
</ul>
</li>
<li style="font-weight: 400;">Setup Private Worker Pool</li>
<li style="font-weight: 400;">Provision Spacelift Worker Pool</li>
<li style="font-weight: 400;">Deploy Test Kubernetes Deployment</li>
</ul>
All of this via the most influential management platform for IaC!

image5-13

Kubernetes clusters

Spacelift supports externally deployed and provision Kubernetes clusters, but what is the fun in that! If you bring your own Kubernetes clusters, you can skip this step.
As mentioned previously, we need two Kubernetes clusters for this scenario. One will be utilized as the Worker pool cluster and the other for a test deployment of Nginx.
Due to the length of this blog, I opted to separate this specific step to keep things brief and clean. You can follow the steps in the blog titled <a href="https://spacelift.io/blog/provision-kubernetes-cluster-terraform-spacelift" target="_blank" rel="noopener">Provision a Kubernetes Cluster with Spacelift and Terraform</a>.

<h3>Certificate Signing Request</h3>
To ensure that we have no access to your private key, you will need to generate it on your end and use it to create a certificate signing request to give to Spacelift. We will create a certificate for you so that workers can use it to authenticate with the Spacelift backend. The following command will generate the key and CSR:

You should now store the spacelift.key file in a secure place. You’ll need it later when launching workers in your worker pool.
<h3>Private Key Encode</h3>
You need to encode the entire private key using base-64, making it a single line of text. The most straightforward approach is to just run something like <code>cat spacelift.key | base64 -w 0</code> in your command line. For Mac users, the command will be something like <code>cat spacelift.key | base64 -b 0</code>.
Save the response in a safe place. You will need it shortly.
<h3>Add Worker Pool</h3>
From the Spacelift UI, navigate to the Worker Pool page by clicking the icon on the navigation bar. Choose the add worker pool button.

image3-17

Add worker pool

<h3>Create new worker pool</h3>
<ul>
<li style="font-weight: 400;">Name: Name of your worker pool</li>
<li style="font-weight: 400;">Your certificate: Upload the generated CSR that was created a few steps prior</li>
<li style="font-weight: 400;">Labels: Optionally add labels</li>
<li style="font-weight: 400;">Description: Optionally add a description</li>
</ul>
Name and Certificate are required.

image9-5

Create new worker pool

Now you can submit the spacelift.csr file in the worker pool creation form. In response, you will receive a Spacelift token and be taken to the newly created Worker Pool page.

image14

Spacelift token

Now that we have the Spacelift token, we can proceed with deploying the Spacelift Worker pool.

Support for worker pools on Kubernetes is nothing new, and you can check out the <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">repository</a> for additional details.
<h3>Repository Creation</h3>
Start by creating a new repository for your Helm stack. This repository only needs to contain a single item &#8211; a kustomization.yaml.

The kustomization file is used to tell kubectl where to find the file containing the output of the helm template command and prevents kubectl from attempting to apply every YAML file in your repository.
<h3>Create Worker Cluster Stack</h3>
In Spacelift, click the Add Stack button to create a Stack.
<h4>Integrate VCS</h4>
<ul>
<li style="font-weight: 400;">Select the repository you created in the initial step, as seen in the picture.</li>
</ul>

image8-8

Integrate VCS

<h4>Configure Backend</h4>
For this example, select the Kubernetes backend.

image10-5

Configure Backend

Type in spacelift-worker into the Namespace field. This namespace will be utilized for all deployments associated with this Stack.
<h4>Define Behavior</h4>
Select the arrow next to Show Advanced Options to expose the advanced configuration options. 
The following options should be reviewed and validated.
<ul>
<li style="font-weight: 400;">Worker Pool: Depending on the maturity of your environment. By default, the none &#8211; use shared option will work.</li>
<li style="font-weight: 400;">Runner Image: Spacelift does not provide an image that includes helm, which is required in this step. You must provide your own image.</li>
<li style="font-weight: 400;">Customize workflow: During the initialization phase, you must specify the following commands:
<ul>
<li style="font-weight: 400;"><code>aws eks update-kubeconfig --region $region-name --name $cluster_name</code></li>
<li style="font-weight: 400;"><code>helm repo add spacelift https://downloads.spacelift.io/helm</code></li>
<li style="font-weight: 400;"><code>helm repo update</code></li>
<li style="font-weight: 400;"><code>helm template spacelift-worker-pool spacelift/spacelift-worker --set "replicaCount=$SPACELIFT_WORKER_REPLICAS" --set "credentials.token=$SPACELIFT_TOKEN" --set "credentials.privateKey=$SPACELIFT_WORKER_POOL_PRIVATE_KEY" &gt; spacelift-worker-pool.yaml</code></li>
</ul>
</li>
</ul>
We will provide the necessary environment variables at a later stage.

image12-5

Define Behavior

<h4>Name Stack</h4>
Provide the name of your Stack. Labels and Description are not required but recommended.

image7-8

Name Stack

<h3>Environment Variables</h3>
To simplify the deployment and improve security, I have chosen to set up variables within the Stack. Each variable is used at different times throughout the deployment process.
<ul>
<li style="font-weight: 400;">SPACELIFT_WORKER_REPLICAS
<ul>
<li style="font-weight: 400;">Defines the number of replicas deployed for use in the worker pool</li>
</ul>
</li>
<li style="font-weight: 400;">SPACELIFT_TOKEN
<ul>
<li style="font-weight: 400;">This specific variable was provided earlier in the CSR phase and should be associated accordingly.</li>
</ul>
</li>
<li style="font-weight: 400;">SPACELIFT_WORKER_POOL_PRIVATE_KEY
<ul>
<li style="font-weight: 400;">This private key was created earlier during the CSR phase and should be added appropriately.</li>
</ul>
</li>
</ul>
<h3>Cloud Integration</h3>
Navigate to the Settings, Integrations page and select the dropdown arrow to access the following selection screen:

image13-4

image13

The following links will help you set up the necessary integration with your Cloud Provider of choice.
<ul>
<li style="font-weight: 400;"><a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="nofollow noopener">AWS</a></li>
<li style="font-weight: 400;"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="nofollow noopener">Azure</a></li>
<li style="font-weight: 400;"><a href="https://docs.spacelift.io/integrations/cloud-providers/gcp" target="_blank" rel="nofollow noopener">GCP</a></li>
</ul>
To simplify things, I decided to utilize an AWS role that already exists with the necessary permissions to provide the resources needed to be associated with this Terraform example.

image11-6

<h3>Trigger a Run</h3>
To Trigger a Run, select Trigger on the right side of the Stacks view.

image6-8

Trigger a Run

<h3>Trigger Run Status</h3>
<h4>Planning</h4>
After you manually trigger the Run in the Stack view, Spacelift will deploy a runner image, initialize the Cloud Provider, and run a Terraform Plan based upon the contents of the terraform code.
After a successful planning phase, you can check the log to see the planned changes.

image4-16

Trigger Run Status

To confirm the Triggered run, click the CONFIRM button at the top of the screen.
<h4>Finished Deployment</h4>
The following screen highlights the Finished Run and output from a successful deployment to your Kubernetes cluster.

image2-19

Finished Deployment

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bspacelift_worker_pools_on_kubernetes%7D" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin around the world! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities s for infrastructure management.

Running Spacelift CI/CD workers in Kubernetes using DinD

Achieving the Atlantis Workflow with Spacelift

How a Specialized Management Platform Can Improve Your IaC

See how Spacelift uses Spacelift to provision Kubernetes clusters, a private worker pool, and Kubernetes deployments within the same organization.

72-spacelift_worker_pools_on_kubernetes_with_spacelift

Provision a Kubernetes Cluster with Spacelift and Terraform

Want to level up your Kubernetes knowledge? Explore our tutorials and step-by-step guides written by Kubernetes experts. Learn Kubernetes best practices and solutions to the most difficult problems.

Kubernetes Guides and Tutorials | Spacelift Blog

Kubernetes

By now, you have probably heard about the initial availability of Kubernetes integration in Spacelift. You can read more about the announcement <a href="https://spacelift.io/blog/spacelift-kubernetes-integration-beta" target="_blank" rel="noopener">here</a> and review the documentation.
Spacelift, like many of our customers, utilizes Kubernetes in a variety of ways. Before the integration, we operated Kubernetes or Public worker pools, testing and hosting the Spacelift platform. Now that the Kubernetes integration is initially available, we thought it would be good to provide a simple way for you to provision a Kubernetes cluster on AWS utilizing Terraform with Spacelift.
This blog will guide you through the process of deploying a Terraform based deployment of an EKS cluster with Spacelift. Below is an example of the cloned Terraform repository.

image2-18

devrel-k8s-worker-cluster

Start by cloning the repository found on <a href="https://github.com/hashicorp/learn-terraform-provision-eks-cluster" target="_blank" rel="nofollow noopener">GitHub</a> and commit to your respective VCS provider. We made a few changes, but that was only to help with the naming and possible confusion in our dev environment. Otherwise, utilizing the repository as-is will work.

In Spacelift, click the Add Stack button to create a Stack.
<h3>Integrate VCS</h3>
<ul>
<li style="font-weight: 400;" aria-level="1">Select the repository you created in the initial step, as seen in the picture.</li>
</ul>

image11-5

<h3>Configure Backend</h3>
For this example, I will keep the default selections.

image7-7

<h3>Define Behavior</h3>
If you need to modify the configuration, select the arrow next to Show Advanced Options. You can find out more about these options by reading the <a href="https://docs.spacelift.io/concepts/stack/creating-a-stack#define-behavior" target="_blank" rel="nofollow noopener">documentation</a>.

image4-15

<h3>Name Stack</h3>
Name the stack and provide any labels or descriptions.

image8-7

To provision the EKS cluster, this Stack requires integration with AWS. 
Navigate to the Settings, Integrations page and select the dropdown arrow to access the following selection screen:

image1-18

image1

The following links will help you set up the necessary integration with your Cloud Provider of choice.
<ul>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="nofollow noopener">AWS</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="nofollow noopener">Azure</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/gcp" target="_blank" rel="nofollow noopener">GCP</a></li>
</ul>
To simplify things, I decided to utilize an AWS role that already exists with the necessary permissions to provide the essential resources associated with this Terraform example.

image9-4

To Trigger a Run, select Trigger on the right side of the Stacks view.

image3-16

<h3>Trigger Run Status</h3>
<h4>Planning</h4>
After you manually trigger the Run in the Stack view, Spacelift will deploy a runner image, initialize the Cloud Provider, and run a Terraform Plan based upon the contents of the Terraform code.
After a successful planning phase, you can check the log to see the planned changes.

image5-12

Trigger Run Status - Planning

image6-7

Trigger Run Status - Finished Deployment

To review the outputs associated with this EKS cluster deployment, navigate to the outputs page. You can find all pertinent information about this cluster.

image12-4

Outputs

You have completed the deployment of an EKS cluster via Terraform with Spacelift!

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bprovision_kubernetes_cluster_terraform_spacelift%7D" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin around the world! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and includes many more features.

How to Deploy your Infrastructure in CI/CD using Terraform

Managing Azure Resources using Spacelift

Now that the Kubernetes integration is initially available learn how to complete the deployment of an EKS cluster via Terraform with Spacelift!

70-kubernetes_cluster_with_terraform_and_spacelift

Initial Availability of Kubernetes Integration in Spacelift

At its most basic form, <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code (IaC)</a> is the provisioning and lifecycle management of infrastructure through code instead of manual processes. Over time, the code part has evolved, and each language is either imperative or declarative. The IaC tooling landscape has matured over the years. We have plenty of options to choose from; open-source and commercial. Over the past several years, a majority of the infrastructure related developments is summed up in a single word or ecosystem… Kubernetes!
Kubernetes is nothing new to Spacelift. The platform utilizes Kubernetes for infrastructure, our Public workers run on Kubernetes, and Spacelift support’s our customer&#8217;s deployment of <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">Private workers</a> on Kubernetes via <a href="https://helm.sh/" target="_blank" rel="nofollow noopener">Helm</a>.

image1-17

Kubernetes (Beta)

Kustomize drives Kubernetes support in Spacelift with native support in kubectl. The screenshot below highlights the ability to run kubectl from a Task within a given Stack.

image4-14

Tasks

The team is very excited to see what our customers and community decide to do with the newest integration. The following screenshot is a sample of the output from Spacelift during a Kubernetes Planning phase.

image2-17

Kubernetes Planning phase

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bspacelift_kubernetes_integration_beta%7D" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin worldwide! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Kubernetes is nothing new to Spacelift, but the team is very excited to see what our customers and community decide to do with the newest integration.

71-initial_kubernetes_integration

Achieving an Atlantis Workflow on Spacelift

Running a successful <code>terraform plan</code> command pre-merge does not always mean that the <code>terraform apply</code> command will also be successful post-merge. This statement is key to understanding the value of applying changes before merging. For example, Amazon S3 Bucket uniqueness is validated on the AWS server-side. Due to this, users are unaware that a bucket name is not actually unique until it is created during deployment.
Users can ensure that their main branch is always reflective of error-free deployed infrastructure by applying changes before merging. If errors occur during deployment, users can correct those errors before the code changes land in the main branch.
Atlantis, in particular, has been a popular choice in the Terraform community for addressing this problem and has brought about a unique approach to applying changes from a pull request before they are merged. In this blog post, we will be discussing Atlantis and how you can implement your own Atlantis style workflow using <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Batlantis_workflow_with_spacelift%7D" target="_blank" rel="noopener">Spacelift</a>.

Once upon a time, there was Atlantis. No, we aren’t talking about the ancient mythologic civilization of Atlantis described by Plato around 360 B.C. Instead, we are talking about Atlantis, the self-hosted, open-source tool used for automating Terraform deployments.
Atlantis has become quite popular in the Terraform community, primarily due to its ease of use and purpose-built nature. Historically speaking, there haven’t been many alternatives to Atlantis in the Terraform world. In the early years of Terraform, users needed to manually build out their automation through scripts, using their own <a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener">CI/CD</a> tool of choice &#8211; typically something like Jenkins. Unsurprisingly, some users still follow this approach and manage their CI/CD tooling automation from scratch.
Atlantis has been a game-changer in helping users accelerate their ability to obtain automation around their Terraform code. By spinning up Atlantis and configuring it to their version control system, Atlantis users can quickly get automated deployments around their code. Atlantis has some quirks, such as maintaining the Atlantis project file, but overall, Atlantis is a great tool that solves a specific problem. For some users, the features of Atlantis may satisfy their infrastructure automation needs, but for users with more comprehensive requirements, Atlantis may fall short.
<h3>Workflow Constraints</h3>
Out of the box, Atlantis is a tool that drives everything through pull requests (or merge requests, depending on which VCS you are using). Atlantis supports being able to customize what commands and arguments are to be run for a given Atlantis project and optionally pass in extra arguments to these commands. However, the overarching deployment workflow used by Atlantis cannot be changed. Users of Atlantis must open a pull request for their code if they want to invoke Atlantis in any fashion, whether their goal is to simply run a <code>terraform plan</code> or deploy their code with <code>terraform apply</code>. This reliance on pull requests brings about issues for users who want additional flexibility around their deployment workflows. For example, what if users wanted to trigger deployments when a new git tag is published? (rhetorical question, but this is just a single example of where this lack of flexibility begins).
<h3>Dependency Constraints</h3>
Dependency management when working with infrastructure as code is important, especially with cloud resources. Atlantis doesn’t have a native scalable means of managing your dependencies across your Terraform deployments. You can achieve this through native Terraform or Terragrunt by using things like <code>depends_on</code> and <code>dependencies</code> blocks. Additionally, you could structure your Atlantis project file in such a way to maintain your dependencies as well, but doing this is hacky. There have been third-party tools to help with this issue (Atlantis project file generators), but users are forced to run these tools anytime they add new files or change their dependencies. Later in this article, we will be describing an approach that users can utilize on Spacelift to manage their deployment dependencies.

With Spacelift, users can build flexible workflows through Spacelift Policies. The default behavior of Spacelift will deploy changes when they land into a Spacelift Stack’s tracked branch, but for those who prefer pre-merge apply &#8211; this behavior is customizable. You can also configure Spacelift to deploy changes based on other conditions, perhaps a specific branch, or always allow changes to be applied pre-merge from any branch. Spacelift Policies allow you to make your deployment flow as flexible as possible. You can even access date/time information and require approvals during certain days of the week or times of the day, etc. The possibilities are limitless!
In addition to workflow customization, you can also use Spacelift Policies to analyze your Terraform plan results. Plan Policies allow for the ability to prevent specific resources from being deployed or enforce resources to be configured in a certain way. For example, perhaps you want to enforce tags are present on all resources, or specific tags always are specified on your resources. 
<h3>How to Pre-Merge Apply</h3>
When using Spacelift, if you would like your pull request changes to be applied pre-merge, you simply need to visit your pull request in the Spacelift UI and click the promote button.
Select your pull request:

image2-16

Select your pull request

Click the promote button:

image4-13

Click the promote button

By clicking the promote button, you are “promoting” your proposed run, which did not run any apply commands, into a tracked run, which does run <code>terraform apply</code> commands.

image6-6

run `terraform apply` command

It’s that easy!

With Spacelift, dependency management is easy as well. One common pattern we see used is using a Spacelift Trigger policy to trigger dependencies. You simply define a policy that, in essence, says trigger all stacks that have a label “depends-on:$this-stack” after deployment completes. Then, you simply attach this policy to all of your stacks, and from then on, whenever you need something to depend on something else, you can simply label it as such.
Here’s an example of a Trigger policy capable of doing this:

Let’s walk through a real-world example of using this policy.
First, we’ve created this policy and attached it to our stacks:

image3-14

Trigger dependencies

In this case, we have already attached the above policy to all of our stacks, so now we can simply define our dependencies using labels. We have built a simple dependency flow from development &gt; staging &gt; production in the example below.
You’ll notice that the staging stack contains a label that depends on the development stack, and the production stack has a label that depends on the staging stack. Having the policy attached to these stacks described previously will trigger our stacks in the appropriate environment flow order. 
Note: Depending on your use case, you may also want to apply additional policies to the staging and production stacks to prevent them from being triggered directly and only allow them to be triggered from this flow.

image1-16

Stacks

Now that we have this setup, if we were to trigger our development stack, upon completion, the staging stack is triggered, and lastly, the production stack. Users can then visualize this deployment flow in the Spacelift Runs view. Pretty neat!

image5-11

Account Runs

As you might have picked up by now, Spacelift is the most flexible management platform for infrastructure as code. <a href="https://spacelift.io/blog/alternative-to-atlantis" target="_blank" rel="noopener">Similar to Atlantis</a>, Spacelift is purpose-built for infrastructure as code automation. Besides simply supporting Terraform deployments, Spacelift also supports other tools and languages such as AWS CloudFormation, Pulumi, and Kubernetes &#8211; with support for even more tools coming soon. But there’s more to Spacelift than simply supporting these additional tools.
<a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">Spacelift Policies</a> allow users to customize their Spacelift account using policy-as-code. User-defined policies can decide things such as:
<ul>
<li style="font-weight: 400;" aria-level="1">Who gets to log in to your Spacelift account and with what level of access</li>
<li style="font-weight: 400;" aria-level="1">Who gets to access individual Stacks and with what level of access</li>
<li style="font-weight: 400;" aria-level="1">Who can approve or reject a run, and how a run can be approved</li>
<li style="font-weight: 400;" aria-level="1">Which runs and tasks can be started</li>
<li style="font-weight: 400;" aria-level="1">What infrastructure changes can be applied</li>
<li style="font-weight: 400;" aria-level="1">How Git push events are interpreted</li>
<li style="font-weight: 400;" aria-level="1">Which task commands can be executed</li>
<li style="font-weight: 400;" aria-level="1">What happens when blocking runs terminate</li>
</ul>
In addition to policies, Spacelift also offers other unique features such as Resource and Run Observability, a Module Registry, Drift Detection, and more (check <a href="https://docs.spacelift.io/" target="_blank" rel="noopener">Documentation</a>).

In this article, we described Atlantis and its pull-request-driven workflow. We then discussed how users could use the promote functionality to achieve this same pre-merge apply workflow using Spacelift. Furthermore, we talked about using Spacelift to manage your dependencies by combining a trigger policy with labels. 
Atlantis is a great tool for Terraform automation, especially for those looking to run their own completely self-hosted CI/CD solution. Currently, Spacelift does not provide a self-hosted option but instead follows a hybrid approach allowing users to run their workers on their own infrastructure while using the Spacelift UI as a service. If you’d like to see Spacelift implement a completely self-hosted option, let us know in the comments of this article!
Thank you for reading. Feel free to comment and share on your social media platform if it is of any value to you. We wish you the best of luck in your infrastructure automation journey!

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Alternative to Atlantis – Spacelift

5 Ways to Manage Terraform at Scale - Best Practices

See Atlantis and its pull-request-driven workflow and learn how to achieve this same pre-merge apply workflow using Spacelift.

65-achieving_an_atlantis_workflow_on_spacelift

Want to level up your Terraform knowledge? Explore our tutorials and step-by-step guides written by Terraform experts. Learn Terraform best practices and solutions to the most difficult problems.

Terraform Guides and Tutorials | Spacelift Blog

Terraform

This blog post will deep dive into how Terraform handles output and how we can leverage and use output values efficiently across our Terraform projects. Output values allow us to share data between modules and workspaces while also providing us the flexibility to pass values to external systems for automation purposes.
You have come to the right place if you are new to Terraform! Spacelift has curated a ton of valuable material, tutorials, and <a href="https://spacelift.io/blog/terraform" target="_blank" rel="noopener">blog posts around Terraform</a> and how industry experts use it on its Spacelift blog.

Input variables permit us to customize Terraform configurations without hardcoding any values. This way, we can reuse <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> while assigning custom values based on our needs. Usually, we refer to them as just variables in the context of Terraform.
To define input variables, we must declare them using a variable block:

The variable&#8217;s name is the label we set following the variable keyword. For every variable, we have the option to set some arguments such as default, type, description, validation, sensitive, and nullable. Check the official documentation about these arguments and how to set them in detail <a href="https://www.terraform.io/language/values/variables#arguments" target="_blank" rel="nofollow noopener">here</a>. 
After declaring our input variables, we can utilize them in modules by referencing them like this var.&lt;variable_name&gt; where &lt;variable_name&gt; matches the label following the variable keyword. For example, to reference the variable ec2_instance_type that we defined above:

On the other hand, output values empower us to export helpful information from our Terraform projects that we have defined and provisioned with Terraform. In the context of Terraform, we refer to output values as just outputs for simplicity. 
Combining input and output variables, we get the flexibility to customize, automate, reuse and share our Terraform code easily. Input variables are similar to function arguments in traditional programming, while output variables work similarly to the return values of a function. Both are equally important to make our Terraform projects functional and facilitate data&#8217;s incoming and outgoing flow.

More specifically, output values are quite helpful in certain use cases:
<ul>
<li style="font-weight: 400;" aria-level="1">We can expose information from child modules to a parent module using outputs.</li>
<li style="font-weight: 400;" aria-level="1">Even more, from a root module, we can print outputs in the command line or pass these output values to external systems for automation purposes. </li>
</ul>
When we use a remote state, we can access the root module outputs by other configurations using the <a href="https://www.terraform.io/language/state/remote-state-data" target="_blank" rel="nofollow noopener">terraform_remote_state</a> data source. Output values from child modules aren&#8217;t accessible.

In order to define an output value, we have to use the output block:

In the above example, we define an output value with the name instance_public_ip. This way, we can pass the value to the parent module or display it to the end-user if it’s an output of the root module.
The value argument, which is the returned output value, takes an expression referencing other resources or module attributes. Terraform only renders and displays outputs when executing terraform apply and not when executing terraform plan. 
To use outputs of nested modules from parent modules, we have to reference them as: 
module.&lt;module_name&gt;.&lt;output_value_name&gt;
For example, to reference the output value instance_public_ip that we have declared above in a module named aws_web_server_instance from its parent module, we have to use:

Let’s examine how we can use all this in a real-world example. In <a href="https://github.com/spacelift-io-blog-posts/Blog-Technical-Content/tree/master/terraform-output" target="_blank" rel="nofollow noopener">this GitHub repository</a>, we define the Terraform configuration for this example’s infrastructure. To follow along, you will need to <a href="https://spacelift.io/blog/how-to-install-terraform" target="_blank" rel="noopener">install Terraform</a>, have an <a href="https://aws.amazon.com/console/" target="_blank" rel="nofollow noopener">AWS account</a> ready, and <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs" target="_blank" rel="nofollow noopener">authenticate with your AWS keys</a> via the command line. Note that you might be charged a few dollars in your AWS account if you follow along.
In this example, we create the necessary infrastructure for a webserver. For the needs of this demo, we split our Terraform configuration into three modules, the root one and two child modules responsible for handling VPC-related resources and EC2 instance-related resources. 

The project structure looks like this:

image6-5

The project structure

For each module, we define a main.tf file that handles the main functionality of the module. 
Variable’s declarations and default values are populated in variables.tf files, while for the root module, we also use a terraform.tfvars file to set some variable values.
A good practice is to define our outputs in separate outputs.tf files, as you can see in the above example project structure. By declaring output values in an outputs.tf file per module, we improve the clarity of our modules as it’s easier for users to understand what outputs to expect from them quickly. 
The root module utilizes and configures the aws provider and then just simply calls two child modules aws_web_server_vpc and aws_web_server_instance in main.tf of the top directory. 
root module main.tf

We notice that when calling the module aws_web_server_instance, we are passing two expressions using output values from the aws_web_server_vpc module with the notation module.&lt;module_name&gt;.&lt;output_value_name&gt; we have seen earlier. 
root module outputs.tf

We define three output values for our root module, and we expect to see them at the command line after our infrastructure is provisioned. Checking the value parameter of each block, we notice that all of them are coming from output values of the two child modules, and by declaring them as output values of the root module, we are able to pass them through to the command line.
root module variables.tf

root module terraform.tfvars

Let’s examine next our two child modules and how we use output values to pass parameters between them.
aws-web-server-vpc module main.tf

In the above module, we define some resources necessary for the networking layer of our infrastructure.
aws-web-server-vpc module variables.tf

aws-web-server-vpc module outputs.tf

The two outputs we export here from this module are passed to the aws-web-server-instance module as parameters in order to create the EC2 instance inside the vpc and subnet that we have just created. We saw how this was handled in the main.tf file of the root module. The output value vpc_id is passed along as an output of the root module and should be printed in the command line after we apply the plan.
Finally, the Terraform configuration for the aws-web-server-instance module uses the passed info from the aws-web-server-vpc module. It creates and configures the web server instance accordingly.  
aws-web-server-instance module main.tf

aws-web-server-instance module variables.tf

The two output values that we pass through the root module are also defined in this module’s outputs.tf file.

Time to wrap up everything and execute the plan to provision our demo infrastructure.

image8-6

plan to provision the demo infrastructure

Our terraform plan shows 7 new resources to be added and displays the changes to our three output values declared in the root module. Let’s go ahead and apply the plan.

image3-13

Apply the plan

As expected, the three outputs declared in the root module are displayed at the command line, sweet! 
We could use these values to automate other parts of our systems and process, but for now, we can get the value from instance_public_ip and head to http://&lt;instance_public_ip&gt;, and we should see our demo web server up and running.

image7-6

demo web server

Great! Everything works as expected.

Output values are stored in the state Terraform file. Since we have successfully applied our plan, we can now access these output values at will. We can leverage the terraform output command for this purpose.

image2-15

Terraform output command

Additionally, we can query individual output values by name like this
terraform output &lt;output_value_name&gt;

image5-10

individual output values

To get the raw value without quotes, use the -raw flag.
terraform output -raw &lt;output_value_name&gt; 
To get the JSON-formatted output, we can use the -json flag.

image1-15

the -json flag

This is quite useful when we want to pass the outputs to other tools for automation since JSON is way easier to handle programmatically. Note that Terraform does not protect sensitive output values when using the -json flag.
When we are done, let’s go ahead and delete all these resources to avoid paying for them. 
From the top of our repository, execute:
<code><a href="https://spacelift.io/blog/how-to-destroy-terraform-resources" target="_blank" rel="noopener">terraform destroy</a></code>

When defining output values, we have a couple of options that might help us better define and organize them. 
The argument description is optional, but it is always considered good practice to include it in our output declarations to document their purpose. This argument should briefly explain each output&#8217;s intent and should be used as a helper description for the users of the module. We have already seen examples like this since we defined the description argument in all our output block declarations in our previous demo.
In cases where we want to handle sensitive values and suppress them in command line output, we can declare an output value as sensitive. Terraform will redact the values of sensitive outputs when planning, applying, destroying, or querying outputs to avoid printing them to the console. In practice, this is a good use case when we would like to pass values to other Terraform modules or automation tools without exposing them to the intermediate users.

Note that Terraform won’t redact sensitive output values when you query a specific output by name. After we apply a plan with an output declared as sensitive, the console displays a message with the value redacted.

image4-12

These values are still recorded in the state files, so anyone who can access them can also access any sensitive values of our Terraform configuration. 
The depends_on argument on output declarations is used to define dependencies explicitly when this is necessary. Most of the time, Terraform handles this automatically, but there are some rare uses cases where you might find this option handy when it’s not the case. Consider including a comment when you use this option to explain why this is necessary.

Occasionally, we might need to share data between different Terraform configurations with separate states. This is where the terraform_remote_state data sources come into play. We can retrieve the root module outputs from another Terraform configuration using this data source. This built-in data source is available without any extra configuration needed. 
Following up on our previous example, let’s say that we would like to create a new subnet in the vpc of our aws-web-server-vpc module. This time, the new subnet needs to be defined in a completely separate Terraform configuration that has its own state. We can leverage the terraform_remote_state to get the value of the vpc_id defined as an output of our previous example’s root module. In this case, we use the local backend to reach the state of another configuration in the local machine. The backend could be any remote backend that points to a Terraform state in a real-world scenario.

Note that only the output values of the root module are accessible from the remote state. If we want to pass values from nested modules, we have to configure a passthrough output value declaration as we defined earlier in the root module of our previous example. 
Although this option is handy for some use cases, it also has some caveats. To use this data source, the user must have access to the entire state snapshot, which could potentially expose sensitive data. Check out the official docs to find <a href="https://www.terraform.io/language/state/remote-state-data#alternative-ways-to-share-data-between-configurations" target="_blank" rel="nofollow noopener">alternative ways to share data between configurations</a>.

We have seen how Terraform handles and exports output values between modules and the different options for outputs configuration. Even more, we compared input and output variables and examined multiple use cases where the use of outputs is helpful. Finally, we went through a complete example of using output values in our Terraform configuration between different modules and printing them to the console. 
Thank you for reading, and I hope you enjoyed this “Terraform Outputs” blog post as much as I did.

Terraform Count vs. For Each Meta-Argument - When to Use It

How to Use Terraform Variables (Locals, Input, Output, Environment)

What are Terraform outputs? See how Terraform handles and exports output values between modules and the different options for outputs configuration.

Spacelift Worker Pools on Kubernetes with Spacelift

Get the newsletter

Recent posts

Spacelift Worker Pools on Kubernetes with Spacelift

Provision a Kubernetes Cluster with Spacelift and Terraform

Initial Availability of Kubernetes Integration in Spacelift

Achieving the Atlantis Workflow with Spacelift

Terraform Output Values : Complete Guide & Examples

How to Use Different Types of Ansible Variables (Examples)

Working with Ansible Playbooks – Tips & Tricks with Examples

What is Pulumi? Key Concepts and Features Overview

Terraform Init – Command Overview with Quick Usage Examples

Terraform vs. AWS CloudFormation : The Ultimate Comparison

What are Terraform Templates? Basics, Use Cases, Examples

SRE vs. DevOps: What’s the Difference Between Them?