Learn more about new features, updates, and Spacelift product announcements.

Product Features and Updates | Spacelift Blog

Author title

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

General | Spacelift Blog

General

Product

So your team learned about <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code (IaC)</a>? They got all excited about this way of managing infrastructure. Soon enough, you had multiple stacks described in definition files. This source code was versioned in your favorite Version Control System such as GitHub, and integrated with your favorite CI/CD platform.
Finally, the infrastructure was built from those definition files, and it was time to celebrate the end of this journey, right? Or was it the end of the journey?
What the definition files describe is the desired state of the infrastructure. The way you want it to be set up. The responsibility of your IaC tool is to turn that desired state into reality. We call the current reality of your infrastructure the actual state. Right after you ran your IaC tool, both are identical but unfortunately, they might not stay in sync for long.
Any difference between the desired state and the actual state is called drift.

It might be surprising at first that infrastructure drifts. After all, you are managing it via code, right? Unfortunately, there are different ways your infrastructure can be modified outside of your IaC process.
<h3>Manual changes</h3>
Most of the drift is usually caused by manual changes performed by individuals.
Some of those reasons are understandable. For example, during an incident, an engineer might need to increase the number of resources to handle an elevated load or make up for resources being down. During those times of high stress when a lot is at stake, manual mitigation changes are perfectly acceptable. The goal is to get to a better place as soon as possible, and the regular IaC process might take too long. Especially when you need to respond to fast-changing conditions.
This becomes a problem if, after the fire has been put out, the changes are not reverted or backported to the IaC definition files.
There are also bad reasons for manual changes. Those cannot be justified, even temporarily, and often stem from poor education on best IaC practices, loose access permissions, and a lack of proper communication regarding the infrastructure management process.
<h3>Overlapping/conflicting IaC code</h3>
In some cases, humans are not directly at fault. Resources may end up being managed by multiple sets of IaC definition files. Applying some definition files might revert changes made by other definition files.
This can happen when the IaC practices evolve over time. For example, it is not uncommon to switch to a different IaC tool after a few years because the team realized that, in hindsight, they had not picked the best tool for their use cases.
Another reason for this is overlapping boundaries between stack definitions which can happen easily when the infrastructure is extensive and managed by many different teams over the course of a long period of time.

IaC is all about improving the governance of your infrastructure by defining it as code which allows you to leverage a wealth of practices and tools that have been available to developers for a long time such as code versioning, code reviews, static analysis, automated tests, etc.
Letting drift occur undermines that effort and provides a false sense of governance.

As we have seen earlier, the primary source of drift is manual changes. Those are often linked to loose access control practices. The Principle of least privilege recommends granting only the necessary permissions. The fewer people can manually modify the infrastructure, the better. Admin-level access is typically limited to senior infrastructure engineers and SREs.
Because you will always need to have some people with the permissions to perform manual changes to your infrastructure, you need to make sure that they are aware of the process to either revert or backport those changes in due time.
That being said, even with the most trained engineers and the best intentions, drift will happen. It is inevitable, so you need to make sure that you can easily and quickly detect it and possibly revert it.

As we said earlier, drift is the difference between the desired state of the infrastructure as defined in the IaC source code and the actual state of the infrastructure.
Said differently, drift is getting a non-empty list of proposed changes when running the plan command for your IaC tool.
Here is how to display the proposed changes with the main IaC tools.
<h3>Terraform</h3>
Run the <code>terraform plan</code> command to get the list of proposed changes. 

In the screenshot below, we can see that the maximum number of servers in the Auto Scaling Group was set to 5 outside of Terraform which is drift.

image4-19

Auto Scaling Group outside Terraform

<h3>CloudFormation</h3>
CloudFormation has a built-in drift detection feature that can be used either via the AWS Console or via the AWS CLI command.
CloudFormation’s drift detection must be triggered manually. There is no built-in automation to make it run on a schedule. Also, <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-supported-resources.html" target="_blank" rel="nofollow noopener">not all resource types can detect drift at this time</a>.
Drift detection checks can be run via the AWS Console:

image14-2

Drift detection checks via the AWS Console

image6-11

AWS Console Overview

image1-25

AWS Console Drifts

image10-7

AWS Console Drift details

Or with on the command line with AWS CLI:

image8-10

Drift detection on the command line with AWS CLI

image9-7

describe-steck-drift-detection-status

image2-24

describe-stack-resource-drifts

<h3>Pulumi</h3>
Run the <code>pulumi preview --refresh --stack &lt;STACK NAME&gt;</code> command to get the list of proposed changes.
The screenshot below shows that the tags and user data of the AWS EC2 instance have been modified manually.

image11-8

pulumi preview --refresh --stack <STACK NAME>

<h3>Spacelift</h3>
Drift can occur at any time. As a result, drift detection must be run on a regular schedule to catch it as quickly as possible which is not practical when running those commands on one’s laptop.
A better approach would be to use a tool such as Spacelift that can check for drift automatically on a schedule that you set.

image12-7

Spacelift drift detection

The view that shows all the resources for a stack uses an eye-catching icon for resources that have drifted so that they can be easily spotted.

image3-21

resources that have drifted

Another benefit of using Spacelift is that the drift detection management experience is consistent across the supported IaC tools. Under the hood, different commands will be run but for the most part, the workflow and the screens will be identical.

You probably want to get rid of most drift which we will explain in a bit, but there might be manual changes that should make their way to the definition files.
For example, you had to increase the number of resources during an elevated load episode, but realistically, this is not a one-off but the new normal. Then, you should not eliminate the drift but update the IaC definition files to reflect your new expectations.

Since drift is having a non-empty list of proposed changes when the definition files have not changed, fixing that drift is applying the proposed changes. That will restore the infrastructure to its desired state.
Here is how to remove the drift and get back to the desired infrastructure state with the main IaC tools.
<h3>Terraform</h3>
Run the <code>terraform apply</code> command to revert the external changes and remove the drift.

image15-2

terraform apply

<h3>AWS CloudFormation</h3>
CloudFormation can revert drift in some cases only.
For example, if a resource is missing it will be recreated but if a property of a resource was modified it might not be detected by CloudFormation and as a result, it won&#8217;t be fixed automatically.
If CloudFormation cannot automatically fix the detected drift, you can use the information provided to manually revert the unexpected changes.
<h3>Pulumi</h3>
Run the <code>pulumi up --stack &lt;STACK NAME&gt;</code> command to revert the external changes and remove the drift.

image5-17

pulumi up --stack <STACK NAME>

image13-6

Spacelift Edit Drift Detection

When drift is detected, Spacelift can optionally revert the changes found by following the same workflow that is used for regular IaC code changes, enforcing all the configured guardrails such as <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="nofollow noopener">automated validation of the plan</a> and <a href="https://docs.spacelift.io/concepts/policy/approval-policy" target="_blank" rel="nofollow noopener">approval workflow</a>.

image7-10

Spacelift Drift Detected

Like incidents, drift is inevitable and part of the life of any infrastructure. It must be taken into account when defining the processes and selecting your tools so that you do not get caught off guard and stay on top of things regarding your infrastructure governance.

Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures.

How a Specialized Management Platform Can Improve Your IaC

Infrastructure Problems that Spacelift Solves

5 Ways to Manage Terraform at Scale - Best Practices

What is drift and why does it happen? See how IaC tools, including Spacelift, can help you with drift detection and remediation.

36-drift_detection_1

Drift detection with IaC tools

Drift Detection and How to Fix It With Different IaC Tools

As Terraform users utilize their code at scale, they typically find redundant variable declarations, complexity in deploying to multiple cloud providers, and a need to manage their growing number of dependencies across deployments. Terragrunt was built to address these use cases and others and has become quite popular amongst the Terraform community.
<h3>What is Terragrunt?</h3>
<a href="https://github.com/gruntwork-io/terragrunt" target="_blank" rel="nofollow noopener">Terragrunt</a> is an open-source wrapper for Terraform, created by Gruntwork, that provides a layer of extra features to help users manage their Terraform code at scale. The tool has become widely adopted, with over 5,800 stars and 700 forks on GitHub. Terragrunt has a laundry list of use cases for users to use. The Gruntwork team documents an official list of Terragrunt use cases <a href="https://terragrunt.gruntwork.io/docs/" target="_blank" rel="nofollow noopener">here</a>. 
In this article, we will be walking through some of the most notable use cases of Terragrunt and how they relate to the problems Terraform users face daily.

In this section, we will be discussing a select group of notable use cases (in our own words) and the benefits they provide Terraform users.
<h3>Keep Your Terraform Configuration DRY</h3>
Arguably, the most popular feature of Terragrunt is its ability to help keep your Terraform configuration D.R.Y. &#8211; Don’t Repeat Yourself &#8211; this relates to the redundancy problem mentioned earlier. Terragrunt is able to achieve this through the use of built-in functions such as find_in_parent_folders() and read_terragrunt_config(). These functions allow you to dynamically load files upstream within “parent” folders, defining them only once in a hierarchical folder structure. 
As a somewhat unintentional conflict of this DRY benefit, one could argue that fully leveraging the DRY features of Terragrunt forces users down a path of building large monorepositories. This typically occurs when users start a single repository in which they are organizing/managing all of their Terragrunt code. This isn’t always the case, of course, as it truly depends on how users choose to organize their folder structure, but it can quickly become an issue if users are not mindful of this. Although monorepositories are not necessarily always a bad thing.
The typical mono-repository Terragrunt structure follows a top-down pattern such as &lt;account&gt;/&lt;region&gt;/&lt;service&gt;/&lt;resourceName&gt;. This allows users to define variables within any top-level folder and have those variables nest down within their respective child folders. For example, one might define top-level account variables such as <code>accountId</code> or <code>accountAlias</code> within a top-level <code>account.hcl</code> file. You can define these variables only once and have them “loaded” into your lower-level folders at runtime. To do this, you will need to make use of the Terragrunt functions mentioned above. Users will find that using this feature at scale allows for cleaner code, especially when deploying many resources to consume redundant “top-level” values.
As an example, below you’ll find a code snippet of how easy it is to load in top-level variables from a file with a specific name:

<h3>Working With Multiple Providers</h3>
Using Terragrunt’s <a href="https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#generate" target="_blank" rel="nofollow noopener">generate</a> block, users are able to dynamically generate Terraform code. This feature is most commonly used for dynamically generating Terraform providers to allow for a dynamic multi-account / multi-region Terraform implementation. 
<h4>Provider Generation</h4>
Within the root-level <code>terragrunt.hcl</code> file, you will typically find the code looking something like this:

The code above will dynamically generate a <code>provider.tf</code> file containing dynamic provider contents depending upon where Terragrunt is being run. For example, if you were running <code>terragrunt apply</code> at the following path:

The <code>provider.tf</code> file would be generated within the <code>example</code> folder.
The <code>aws_region</code> variable would be loaded from the <code>region.hcl</code> file which should live within the <code>us-east-1</code> directory (assume that it exists here with that variable defined)
As a result, the contents of the <code>provider.tf</code> file would be:

Pretty cool, right? As you can see, the provider was dynamically generated at runtime simply because we defined the region within a top-level folder. Users can take this approach a step further by using account/role information within their provider blocks and can ultimately achieve the ability to assume roles in multiple accounts dynamically using this method. Keep in mind that technically the generate feature isn’t necessarily restricted to generating provider blocks, so you can always follow this same method to generate Terraform code for other use cases as well.
<h3>Dependency Management</h3>
With Terragrunt, two dependency-related features can easily be confused. The “dependency” block in Terragrunt allows users to define a dependency to load information from it, primarily used for loading Terraform outputs as inputs. The other dependency feature used by Terragrunt is dependencies. In the dependencies block, users specify a list of paths to folders/files, which Terragrunt should execute before executing on the current file.
Let’s walk through an example of each of these types!
<h4>The “Dependency” Block</h4>
In the code below, we specify VPC as a dependency and specify the config_path, a relative path to the dependency from which we need to load configuration. In this example, the VPC path lives back a directory in the VPC folder and produces the <code>vpc_id</code> output. Using this dependency block, we can load the vpc_id output and then use it as an input in our current configuration file (full example not shown, as we are focused on simply showing the usage of the dependency block here).
There are, of course, other ways you can pass configuration between your Terraform code, such as using AWS Parameter Store or accessing the configuration using the <code>terraform_remote_state</code> data source. Still, as you can see here, it’s rather convenient to load dependency values using this method. One limitation of this approach you’ll quickly find is that you’re understandably unable to reference dependency values deployed from other repositories. Using AWS Parameter Store or another configuration management solution is a great solution in these situations.

<h4>The “Dependencies” Block</h4>
The dependencies block is designed for enforcing the order of operations by Terragrunt. This block is meant for users who intend to eventually make use of the `run-all` functionality of Terragrunt, which allows you to execute Terraform against a group of deployments all at once or in a specified order. 
Let’s walk through a basic example of what this looks like:

Not much to see here, but notice how you can specify a list of strings, allowing you to specify n number of dependencies that you want to ensure Terragrunt executes on before executing the current file. Similar to the dependency limitation, you are constrained by relative paths, meaning you won’t be able to build dependencies for code that lives in other repositories.
Once you have your dependencies defined, you can then make the <code>run-all</code> functionality, allowing you to run plan/apply commands across your entire stack; how neat!

Terragrunt is natively supported on Spacelift and can be easily enabled by simply adding the label terragrunt on any stack(s) you would like to enable. Once this label is present, Spacelift will then utilize terragrunt plan/apply commands instead of the typical terraform plan/apply commands.

image2-22

Terragrunt: S3 Bucket

Check out this short video on how easy it is to enable Terragrunt on Spacelift:
<div style="position: relative; padding-bottom: 50.919377652050926%; height: 0;"><iframe style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;" src="https://www.loom.com/embed/4e41f88cf32c4c6b96305e94b72d68e3" frameborder="0" allowfullscreen="allowfullscreen"></iframe></div>
<h3>Why Spacelift?</h3>
Spacelift is the most flexible management platform for infrastructure-as-code. It supports not only Terraform, but AWS CloudFormation, Pulumi, and Kubernetes. Out of the box, you’ll have support for deploying to multiple cloud providers and multiple cloud accounts.
Furthermore, by running your Terragrunt commands on Spacelift, you can extend the powers of Terragrunt with the powers of Spacelift. Providing you access to other powerful features such as Spacelift <a href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="nofollow noopener">Contexts</a> and <a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="nofollow noopener">Policies</a>. For example, using Spacelift Contexts, you can define a collection of variables and files and then attach these to Spacelift stack(s). By doing this, you then have access to those variables/files during runtime. Remember that <code>account.hcl</code> file scenario we mentioned earlier, in which you would define all of your account-specific variables? This is a great example scenario. You could create a Spacelift Context for (an account-specific context) and then attach this context to all of your Spacelift Stacks that are deploying code for that specific account. The same concept can be applied to any other grouping mechanism you’d like, whether creating a Context for a region, environment, application, etc. Contexts are a more scalable means of keeping your configuration DRY, without the side effect of creating a large monorepository.

image1-22

example account

With Spacelift Policies, we could easily have an entirely separate blog post on this topic. In summary, they allow you to build rules using the <a href="https://spacelift.io/blog/what-is-open-policy-agent-and-how-it-works" target="_blank" rel="noopener">Open Policy Agent (OPA)</a> to govern your infrastructure as code and its automation. For example, you may want to disallow a specific resource type or even a certain way of configuring that resource type &#8211; you can easily achieve such a thing using a <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="nofollow noopener">Plan Policy</a>.

In this article, we discussed some of the common issues that Terraform users face and how Terragrunt solves them. We discussed Spacelift’s support for Terragrunt and a few of the additional features. If you found any value in this article, please share it on your social media platform of choice. Wishing you the best of luck in your infrastructure as code journey!

Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation, and controls to simplify and accelerate the provisioning of cloud based infrastructures.

How to Deploy your Infrastructure in CI/CD using Terraform

By running your Terragrunt commands on Spacelift, you can extend the powers of Terragrunt with other features such as Spacelift Contexts and Policies.

67-terragrunt_with_spacelift

Stay DRY with Terragrunt on Spacelift

Keep Your Configuration DRY with Terragrunt on Spacelift

Spacelift Worker Pools on Kubernetes with Spacelift

You are probably a bit confused by the description of the subtitle; stay with me! If you have not seen the latest news, Spacelift supports Kubernetes! Check out the announcement blog and documentation.
This blog aims to demonstrate the power of Spacelift by providing a view of how Spacelift uses Spacelift to provision Kubernetes clusters, a private worker pool, and Kubernetes deployments all within the same organization and structure.
To summarize, we are looking at the following objectives for Spacelift:
<ul>
<li style="font-weight: 400;">Provision Kubernetes clusters
<ul>
<li style="font-weight: 400;">Application</li>
<li style="font-weight: 400;">Worker (pool)</li>
</ul>
</li>
<li style="font-weight: 400;">Setup Private Worker Pool</li>
<li style="font-weight: 400;">Provision Spacelift Worker Pool</li>
<li style="font-weight: 400;">Deploy Test Kubernetes Deployment</li>
</ul>
All of this via the most influential management platform for IaC!

image5-13

Kubernetes clusters

Spacelift supports externally deployed and provision Kubernetes clusters, but what is the fun in that! If you bring your own Kubernetes clusters, you can skip this step.
As mentioned previously, we need two Kubernetes clusters for this scenario. One will be utilized as the Worker pool cluster and the other for a test deployment of Nginx.
Due to the length of this blog, I opted to separate this specific step to keep things brief and clean. You can follow the steps in the blog titled <a href="https://spacelift.io/blog/provision-kubernetes-cluster-terraform-spacelift" target="_blank" rel="noopener">Provision a Kubernetes Cluster with Spacelift and Terraform</a>.

<h3>Certificate Signing Request</h3>
To ensure that we have no access to your private key, you will need to generate it on your end and use it to create a certificate signing request to give to Spacelift. We will create a certificate for you so that workers can use it to authenticate with the Spacelift backend. The following command will generate the key and CSR:

You should now store the spacelift.key file in a secure place. You’ll need it later when launching workers in your worker pool.
<h3>Private Key Encode</h3>
You need to encode the entire private key using base-64, making it a single line of text. The most straightforward approach is to just run something like <code>cat spacelift.key | base64 -w 0</code> in your command line. For Mac users, the command will be something like <code>cat spacelift.key | base64 -b 0</code>.
Save the response in a safe place. You will need it shortly.
<h3>Add Worker Pool</h3>
From the Spacelift UI, navigate to the Worker Pool page by clicking the icon on the navigation bar. Choose the add worker pool button.

image3-17

Add worker pool

<h3>Create new worker pool</h3>
<ul>
<li style="font-weight: 400;">Name: Name of your worker pool</li>
<li style="font-weight: 400;">Your certificate: Upload the generated CSR that was created a few steps prior</li>
<li style="font-weight: 400;">Labels: Optionally add labels</li>
<li style="font-weight: 400;">Description: Optionally add a description</li>
</ul>
Name and Certificate are required.

image9-5

Create new worker pool

Now you can submit the spacelift.csr file in the worker pool creation form. In response, you will receive a Spacelift token and be taken to the newly created Worker Pool page.

image14

Spacelift token

Now that we have the Spacelift token, we can proceed with deploying the Spacelift Worker pool.

Support for worker pools on Kubernetes is nothing new, and you can check out the <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">repository</a> for additional details.
<h3>Repository Creation</h3>
Start by creating a new repository for your Helm stack. This repository only needs to contain a single item &#8211; a kustomization.yaml.

The kustomization file is used to tell kubectl where to find the file containing the output of the helm template command and prevents kubectl from attempting to apply every YAML file in your repository.
<h3>Create Worker Cluster Stack</h3>
In Spacelift, click the Add Stack button to create a Stack.
<h4>Integrate VCS</h4>
<ul>
<li style="font-weight: 400;">Select the repository you created in the initial step, as seen in the picture.</li>
</ul>

image8-8

Integrate VCS

<h4>Configure Backend</h4>
For this example, select the Kubernetes backend.

image10-5

Configure Backend

Type in spacelift-worker into the Namespace field. This namespace will be utilized for all deployments associated with this Stack.
<h4>Define Behavior</h4>
Select the arrow next to Show Advanced Options to expose the advanced configuration options. 
The following options should be reviewed and validated.
<ul>
<li style="font-weight: 400;">Worker Pool: Depending on the maturity of your environment. By default, the none &#8211; use shared option will work.</li>
<li style="font-weight: 400;">Runner Image: Spacelift does not provide an image that includes helm, which is required in this step. You must provide your own image.</li>
<li style="font-weight: 400;">Customize workflow: During the initialization phase, you must specify the following commands:
<ul>
<li style="font-weight: 400;"><code>aws eks update-kubeconfig --region $region-name --name $cluster_name</code></li>
<li style="font-weight: 400;"><code>helm repo add spacelift https://downloads.spacelift.io/helm</code></li>
<li style="font-weight: 400;"><code>helm repo update</code></li>
<li style="font-weight: 400;"><code>helm template spacelift-worker-pool spacelift/spacelift-worker --set "replicaCount=$SPACELIFT_WORKER_REPLICAS" --set "credentials.token=$SPACELIFT_TOKEN" --set "credentials.privateKey=$SPACELIFT_WORKER_POOL_PRIVATE_KEY" &gt; spacelift-worker-pool.yaml</code></li>
</ul>
</li>
</ul>
We will provide the necessary environment variables at a later stage.

image12-5

Define Behavior

<h4>Name Stack</h4>
Provide the name of your Stack. Labels and Description are not required but recommended.

image7-8

Name Stack

<h3>Environment Variables</h3>
To simplify the deployment and improve security, I have chosen to set up variables within the Stack. Each variable is used at different times throughout the deployment process.
<ul>
<li style="font-weight: 400;">SPACELIFT_WORKER_REPLICAS
<ul>
<li style="font-weight: 400;">Defines the number of replicas deployed for use in the worker pool</li>
</ul>
</li>
<li style="font-weight: 400;">SPACELIFT_TOKEN
<ul>
<li style="font-weight: 400;">This specific variable was provided earlier in the CSR phase and should be associated accordingly.</li>
</ul>
</li>
<li style="font-weight: 400;">SPACELIFT_WORKER_POOL_PRIVATE_KEY
<ul>
<li style="font-weight: 400;">This private key was created earlier during the CSR phase and should be added appropriately.</li>
</ul>
</li>
</ul>
<h3>Cloud Integration</h3>
Navigate to the Settings, Integrations page and select the dropdown arrow to access the following selection screen:

image13-4

image13

The following links will help you set up the necessary integration with your Cloud Provider of choice.
<ul>
<li style="font-weight: 400;"><a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="nofollow noopener">AWS</a></li>
<li style="font-weight: 400;"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="nofollow noopener">Azure</a></li>
<li style="font-weight: 400;"><a href="https://docs.spacelift.io/integrations/cloud-providers/gcp" target="_blank" rel="nofollow noopener">GCP</a></li>
</ul>
To simplify things, I decided to utilize an AWS role that already exists with the necessary permissions to provide the resources needed to be associated with this Terraform example.

image11-6

<h3>Trigger a Run</h3>
To Trigger a Run, select Trigger on the right side of the Stacks view.

image6-8

Trigger a Run

<h3>Trigger Run Status</h3>
<h4>Planning</h4>
After you manually trigger the Run in the Stack view, Spacelift will deploy a runner image, initialize the Cloud Provider, and run a Terraform Plan based upon the contents of the terraform code.
After a successful planning phase, you can check the log to see the planned changes.

image4-16

Trigger Run Status

To confirm the Triggered run, click the CONFIRM button at the top of the screen.
<h4>Finished Deployment</h4>
The following screen highlights the Finished Run and output from a successful deployment to your Kubernetes cluster.

image2-19

Finished Deployment

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bspacelift_worker_pools_on_kubernetes%7D" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin around the world! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities s for infrastructure management.

Running Spacelift CI/CD workers in Kubernetes using DinD

Achieving the Atlantis Workflow with Spacelift

See how Spacelift uses Spacelift to provision Kubernetes clusters, a private worker pool, and Kubernetes deployments within the same organization.

72-spacelift_worker_pools_on_kubernetes_with_spacelift

Provision a Kubernetes Cluster with Spacelift and Terraform

Want to level up your Kubernetes knowledge? Explore our tutorials and step-by-step guides written by Kubernetes experts. Learn Kubernetes best practices and solutions to the most difficult problems.

Kubernetes Guides and Tutorials | Spacelift Blog

Kubernetes

By now, you have probably heard about the initial availability of Kubernetes integration in Spacelift. You can read more about the announcement <a href="https://spacelift.io/blog/spacelift-kubernetes-integration-beta" target="_blank" rel="noopener">here</a> and review the documentation.
Spacelift, like many of our customers, utilizes Kubernetes in a variety of ways. Before the integration, we operated Kubernetes or Public worker pools, testing and hosting the Spacelift platform. Now that the Kubernetes integration is initially available, we thought it would be good to provide a simple way for you to provision a Kubernetes cluster on AWS utilizing Terraform with Spacelift.
This blog will guide you through the process of deploying a Terraform based deployment of an EKS cluster with Spacelift. Below is an example of the cloned Terraform repository.

image2-18

devrel-k8s-worker-cluster

Start by cloning the repository found on <a href="https://github.com/hashicorp/learn-terraform-provision-eks-cluster" target="_blank" rel="nofollow noopener">GitHub</a> and commit to your respective VCS provider. We made a few changes, but that was only to help with the naming and possible confusion in our dev environment. Otherwise, utilizing the repository as-is will work.

In Spacelift, click the Add Stack button to create a Stack.
<h3>Integrate VCS</h3>
<ul>
<li style="font-weight: 400;" aria-level="1">Select the repository you created in the initial step, as seen in the picture.</li>
</ul>

image11-5

<h3>Configure Backend</h3>
For this example, I will keep the default selections.

image7-7

<h3>Define Behavior</h3>
If you need to modify the configuration, select the arrow next to Show Advanced Options. You can find out more about these options by reading the <a href="https://docs.spacelift.io/concepts/stack/creating-a-stack#define-behavior" target="_blank" rel="nofollow noopener">documentation</a>.

image4-15

<h3>Name Stack</h3>
Name the stack and provide any labels or descriptions.

image8-7

To provision the EKS cluster, this Stack requires integration with AWS. 
Navigate to the Settings, Integrations page and select the dropdown arrow to access the following selection screen:

image1-18

image1

The following links will help you set up the necessary integration with your Cloud Provider of choice.
<ul>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="nofollow noopener">AWS</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="nofollow noopener">Azure</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/gcp" target="_blank" rel="nofollow noopener">GCP</a></li>
</ul>
To simplify things, I decided to utilize an AWS role that already exists with the necessary permissions to provide the essential resources associated with this Terraform example.

image9-4

To Trigger a Run, select Trigger on the right side of the Stacks view.

image3-16

<h3>Trigger Run Status</h3>
<h4>Planning</h4>
After you manually trigger the Run in the Stack view, Spacelift will deploy a runner image, initialize the Cloud Provider, and run a Terraform Plan based upon the contents of the Terraform code.
After a successful planning phase, you can check the log to see the planned changes.

image5-12

Trigger Run Status - Planning

image6-7

Trigger Run Status - Finished Deployment

To review the outputs associated with this EKS cluster deployment, navigate to the outputs page. You can find all pertinent information about this cluster.

image12-4

Outputs

You have completed the deployment of an EKS cluster via Terraform with Spacelift!

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bprovision_kubernetes_cluster_terraform_spacelift%7D" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin around the world! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and includes many more features.

Managing Azure Resources using Spacelift

Now that the Kubernetes integration is initially available learn how to complete the deployment of an EKS cluster via Terraform with Spacelift!

70-kubernetes_cluster_with_terraform_and_spacelift

Initial Availability of Kubernetes Integration in Spacelift

At its most basic form, <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code (IaC)</a> is the provisioning and lifecycle management of infrastructure through code instead of manual processes. Over time, the code part has evolved, and each language is either imperative or declarative. The IaC tooling landscape has matured over the years. We have plenty of options to choose from; open-source and commercial. Over the past several years, a majority of the infrastructure related developments is summed up in a single word or ecosystem… Kubernetes!
Kubernetes is nothing new to Spacelift. The platform utilizes Kubernetes for infrastructure, our Public workers run on Kubernetes, and Spacelift support’s our customer&#8217;s deployment of <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">Private workers</a> on Kubernetes via <a href="https://helm.sh/" target="_blank" rel="nofollow noopener">Helm</a>.

image1-17

Kubernetes (Beta)

Kustomize drives Kubernetes support in Spacelift with native support in kubectl. The screenshot below highlights the ability to run kubectl from a Task within a given Stack.

image4-14

Tasks

The team is very excited to see what our customers and community decide to do with the newest integration. The following screenshot is a sample of the output from Spacelift during a Kubernetes Planning phase.

image2-17

Kubernetes Planning phase

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bspacelift_kubernetes_integration_beta%7D" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin worldwide! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Kubernetes is nothing new to Spacelift, but the team is very excited to see what our customers and community decide to do with the newest integration.

71-initial_kubernetes_integration

Achieving an Atlantis Workflow on Spacelift

Running a successful <code>terraform plan</code> command pre-merge does not always mean that the <code>terraform apply</code> command will also be successful post-merge. This statement is key to understanding the value of applying changes before merging. For example, Amazon S3 Bucket uniqueness is validated on the AWS server-side. Due to this, users are unaware that a bucket name is not actually unique until it is created during deployment.
Users can ensure that their main branch is always reflective of error-free deployed infrastructure by applying changes before merging. If errors occur during deployment, users can correct those errors before the code changes land in the main branch.
Atlantis, in particular, has been a popular choice in the Terraform community for addressing this problem and has brought about a unique approach to applying changes from a pull request before they are merged. In this blog post, we will be discussing Atlantis and how you can implement your own Atlantis style workflow using <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Batlantis_workflow_with_spacelift%7D" target="_blank" rel="noopener">Spacelift</a>.

Once upon a time, there was Atlantis. No, we aren’t talking about the ancient mythologic civilization of Atlantis described by Plato around 360 B.C. Instead, we are talking about Atlantis, the self-hosted, open-source tool used for automating Terraform deployments.
Atlantis has become quite popular in the Terraform community, primarily due to its ease of use and purpose-built nature. Historically speaking, there haven’t been many alternatives to Atlantis in the Terraform world. In the early years of Terraform, users needed to manually build out their automation through scripts, using their own <a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener">CI/CD</a> tool of choice &#8211; typically something like Jenkins. Unsurprisingly, some users still follow this approach and manage their CI/CD tooling automation from scratch.
Atlantis has been a game-changer in helping users accelerate their ability to obtain automation around their Terraform code. By spinning up Atlantis and configuring it to their version control system, Atlantis users can quickly get automated deployments around their code. Atlantis has some quirks, such as maintaining the Atlantis project file, but overall, Atlantis is a great tool that solves a specific problem. For some users, the features of Atlantis may satisfy their infrastructure automation needs, but for users with more comprehensive requirements, Atlantis may fall short.
<h3>Workflow Constraints</h3>
Out of the box, Atlantis is a tool that drives everything through pull requests (or merge requests, depending on which VCS you are using). Atlantis supports being able to customize what commands and arguments are to be run for a given Atlantis project and optionally pass in extra arguments to these commands. However, the overarching deployment workflow used by Atlantis cannot be changed. Users of Atlantis must open a pull request for their code if they want to invoke Atlantis in any fashion, whether their goal is to simply run a <code>terraform plan</code> or deploy their code with <code>terraform apply</code>. This reliance on pull requests brings about issues for users who want additional flexibility around their deployment workflows. For example, what if users wanted to trigger deployments when a new git tag is published? (rhetorical question, but this is just a single example of where this lack of flexibility begins).
<h3>Dependency Constraints</h3>
Dependency management when working with infrastructure as code is important, especially with cloud resources. Atlantis doesn’t have a native scalable means of managing your dependencies across your Terraform deployments. You can achieve this through native Terraform or Terragrunt by using things like <code>depends_on</code> and <code>dependencies</code> blocks. Additionally, you could structure your Atlantis project file in such a way to maintain your dependencies as well, but doing this is hacky. There have been third-party tools to help with this issue (Atlantis project file generators), but users are forced to run these tools anytime they add new files or change their dependencies. Later in this article, we will be describing an approach that users can utilize on Spacelift to manage their deployment dependencies.

With Spacelift, users can build flexible workflows through Spacelift Policies. The default behavior of Spacelift will deploy changes when they land into a Spacelift Stack’s tracked branch, but for those who prefer pre-merge apply &#8211; this behavior is customizable. You can also configure Spacelift to deploy changes based on other conditions, perhaps a specific branch, or always allow changes to be applied pre-merge from any branch. Spacelift Policies allow you to make your deployment flow as flexible as possible. You can even access date/time information and require approvals during certain days of the week or times of the day, etc. The possibilities are limitless!
In addition to workflow customization, you can also use Spacelift Policies to analyze your Terraform plan results. Plan Policies allow for the ability to prevent specific resources from being deployed or enforce resources to be configured in a certain way. For example, perhaps you want to enforce tags are present on all resources, or specific tags always are specified on your resources. 
<h3>How to Pre-Merge Apply</h3>
When using Spacelift, if you would like your pull request changes to be applied pre-merge, you simply need to visit your pull request in the Spacelift UI and click the promote button.
Select your pull request:

image2-16

Select your pull request

Click the promote button:

image4-13

Click the promote button

By clicking the promote button, you are “promoting” your proposed run, which did not run any apply commands, into a tracked run, which does run <code>terraform apply</code> commands.

image6-6

run `terraform apply` command

See Atlantis and its pull-request-driven workflow and learn how to achieve this same pre-merge apply workflow using Spacelift.