Learn more about new features, updates, and Spacelift product announcements.

Product Features and Updates | Spacelift Blog

Author title

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

AWS: Spacelift Reduces Time Spent on Cloud Management by 90% 🚀

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

Initial Availability of Kubernetes Integration in Spacelift

Want to level up your Kubernetes knowledge? Explore our tutorials and step-by-step guides written by Kubernetes experts. Learn Kubernetes best practices and solutions to the most difficult problems.

Kubernetes Guides and Tutorials | Spacelift Blog

Kubernetes

Product

Sean O'Dell

At its most basic form, <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code (IaC)</a> is the provisioning and lifecycle management of infrastructure through code instead of manual processes. Over time, the code part has evolved, and each language is either imperative or declarative. The IaC tooling landscape has matured over the years. We have plenty of options to choose from; open-source and commercial. Over the past several years, a majority of the infrastructure related developments is summed up in a single word or ecosystem… Kubernetes!
Kubernetes is nothing new to Spacelift. The platform utilizes Kubernetes for infrastructure, our Public workers run on Kubernetes, and Spacelift support’s our customer&#8217;s deployment of <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">Private workers</a> on Kubernetes via <a href="https://helm.sh/" target="_blank" rel="nofollow noopener">Helm</a>.

image1-17

Kubernetes (Beta)

Kustomize drives Kubernetes support in Spacelift with native support in kubectl. The screenshot below highlights the ability to run kubectl from a Task within a given Stack.

image4-14

Tasks

The team is very excited to see what our customers and community decide to do with the newest integration. The following screenshot is a sample of the output from Spacelift during a Kubernetes Planning phase.

image2-17

Kubernetes Planning phase

<ul>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Infrastructure Problems that Spacelift Solves&quot;}" data-sheets-userformat="{&quot;2&quot;:4224,&quot;10&quot;:2,&quot;15&quot;:&quot;Arial&quot;}">Infrastructure Problems that Spacelift Solves</a></li>
<li><a href="https://spacelift.io/blog/why-devops-engineers-recommend-spacelift" target="_blank" rel="noopener">Why DevOps Engineers Recommend Spacelift</a></li>
<li><a href="https://spacelift.io/blog/spacelift-worker-pools-on-kubernetes" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Spacelift Worker Pools on Kubernetes with Spacelift&quot;}" data-sheets-userformat="{&quot;2&quot;:4224,&quot;10&quot;:2,&quot;15&quot;:&quot;Arial&quot;}">Spacelift Worker Pools on Kubernetes with Spacelift</a></li>
</ul>

Get started on your journey by signing up for a <a href="https://spacelift.io" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin worldwide! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Achieving the Atlantis Pull Request Automation with Spacelift

Managing Azure Resources using Spacelift

How Spacelift Can Improve Your Infrastructure as Code

Kubernetes is nothing new to Spacelift, but the team is very excited to see what our customers and community decide to do with the newest integration.

63-kubernetes-integration

Achieving an Atlantis Workflow on Spacelift

Chris Spitzenberger

Running a successful <code>terraform plan</code> command pre-merge does not always mean that the <code>terraform apply</code> command will also be successful post-merge. This statement is key to understanding the value of applying changes before merging. 
For example, Amazon S3 Bucket uniqueness is validated on the AWS server-side. Due to this, users are unaware that a bucket name is not actually unique until it is created during deployment.
Users can ensure that their main branch is always reflective of error-free deployed infrastructure by applying changes before merging. If errors occur during deployment, users can correct those errors before the code changes land in the main branch.
Atlantis, in particular, has been a popular choice in the Terraform community for addressing this problem and has brought about a unique approach to applying changes from a pull request before they are merged. In this blog post, we will be discussing Atlantis and how you can implement your own Atlantis style workflow using <a href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a>.

Once upon a time, there was Atlantis. No, we aren’t talking about the ancient mythologic civilization of Atlantis described by Plato around 360 B.C. Instead, we are talking about Atlantis, the self-hosted, open-source tool used for automating Terraform deployments.
Atlantis has become quite popular in the Terraform community, primarily due to its ease of use and purpose-built nature. Historically speaking, there haven’t been many alternatives to Atlantis in the Terraform world. In the early years of Terraform, users needed to manually build out their automation through scripts, using their own <a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener">CI/CD</a> tool of choice &#8211; typically something like Jenkins. Unsurprisingly, some users still follow this approach and manage their CI/CD tooling automation from scratch.
Atlantis has been a game-changer in helping users accelerate their ability to obtain automation around their Terraform code. By spinning up Atlantis and configuring it to their version control system, Atlantis users can quickly get automated deployments around their code. Atlantis has some quirks, such as maintaining the Atlantis project file, but overall, Atlantis is a great tool that solves a specific problem. 
For some users, the features of Atlantis may satisfy their infrastructure automation needs, but for users with more comprehensive requirements, Atlantis may fall short.
<h3>Workflow Constraints</h3>
Out of the box, Atlantis is a tool that drives everything through pull requests (or merge requests, depending on which VCS you are using). Atlantis supports being able to customize what commands and arguments are to be run for a given Atlantis project and optionally pass in extra arguments to these commands. 
However, the overarching deployment workflow used by Atlantis cannot be changed. Users of Atlantis must open a pull request for their code if they want to invoke Atlantis in any fashion, whether their goal is to simply run a <code>terraform plan</code> or deploy their code with <code>terraform apply</code>. 
This reliance on pull requests brings about issues for users who want additional flexibility around their deployment workflows. For example, what if users wanted to trigger deployments when a new git tag is published? (rhetorical question, but this is just a single example of where this lack of flexibility begins).
<h3>Dependency Constraints</h3>
Dependency management when working with infrastructure as code is important, especially with cloud resources. Atlantis doesn’t have a native scalable means of managing your dependencies across your Terraform deployments. You can achieve this through native Terraform or Terragrunt by using things like <code>depends_on</code> and <code>dependencies</code> blocks.
 Additionally, you could structure your Atlantis project file in such a way to maintain your dependencies as well, but doing this is hacky. There have been third-party tools to help with this issue (Atlantis project file generators), but users are forced to run these tools anytime they add new files or change their dependencies. 
Later in this article, we will be describing an approach that users can utilize on Spacelift to manage their deployment dependencies.

<ul>
<li><a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Deploy your Infrastructure in CI/CD using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Deploy your Infrastructure in CI/CD using Terraform</a></li>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Common Infrastructure Challenges and How to Solve Them&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Common Infrastructure Challenges and How to Solve Them</a></li>
<li><a href="https://spacelift.io/blog/terraform-automation" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Automate Terraform Deployments and Infrastructure Provisioning&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Automate Terraform Deployments and Infrastructure Provisioning</a></li>
</ul>

With Spacelift, users can build flexible workflows through Spacelift Policies. The default behavior of Spacelift will deploy changes when they land into a Spacelift Stack’s tracked branch, but for those who prefer pre-merge apply &#8211; this behavior is customizable. 
You can also configure Spacelift to deploy changes based on other conditions, perhaps a specific branch, or always allow changes to be applied pre-merge from any branch. Spacelift Policies allow you to make your deployment flow as flexible as possible. 
You can even access date/time information and require approvals during certain days of the week or times of the day, etc. The possibilities are limitless!
In addition to workflow customization, you can also use Spacelift Policies to analyze your Terraform plan results. Plan Policies allow for the ability to prevent specific resources from being deployed or enforce resources to be configured in a certain way. 
For example, perhaps you want to enforce tags are present on all resources, or specific tags always are specified on your resources. 
<h3>How to Pre-Merge Apply</h3>
When using Spacelift, if you would like your pull request changes to be applied pre-merge, you simply need to visit your pull request in the Spacelift UI and click the promote button.
Select your pull request:

image2-16

Select your pull request

Click the promote button:

image4-13

Click the promote button

By clicking the promote button, you are “promoting” your proposed run, which did not run any apply commands, into a tracked run, which does run <code>terraform apply</code> commands.

image6-6

run `terraform apply` command

It’s that easy!

With Spacelift, dependency management is easy as well. One common pattern we see used is using a Spacelift Trigger policy to trigger dependencies. You simply define a policy that, in essence, says trigger all stacks that have a label “depends-on:$this-stack” after deployment completes. 
Then, you simply attach this policy to all of your stacks, and from then on, whenever you need something to depend on something else, you can simply label it as such.
Here’s an example of a Trigger policy capable of doing this:

Let’s walk through a real-world example of using this policy.
First, we’ve created this policy and attached it to our stacks:

image3-14

Trigger dependencies

In this case, we have already attached the above policy to all of our stacks, so now we can simply define our dependencies using labels. We have built a simple dependency flow from development &gt; staging &gt; production in the example below.
You’ll notice that the staging stack contains a label that depends on the development stack, and the production stack has a label that depends on the staging stack. Having the policy attached to these stacks described previously will trigger our stacks in the appropriate environment flow order. 
Note: Depending on your use case, you may also want to apply additional policies to the staging and production stacks to prevent them from being triggered directly and only allow them to be triggered from this flow.

image1-16

Stacks

Now that we have this setup, if we were to trigger our development stack, upon completion, the staging stack is triggered, and lastly, the production stack. Users can then visualize this deployment flow in the Spacelift Runs view. Pretty neat!

image5-11

Account Runs

As you might have picked up by now, Spacelift is the most flexible management platform for infrastructure as code. <a href="https://spacelift.io/blog/alternative-to-atlantis" target="_blank" rel="noopener">Similar to Atlantis</a>, Spacelift is purpose-built for infrastructure as code automation. 
Besides simply supporting Terraform deployments, Spacelift also supports other tools and languages such as AWS CloudFormation, Pulumi, and Kubernetes &#8211; with support for even more tools coming soon. But there’s more to Spacelift than simply supporting these additional tools.
<a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">Spacelift Policies</a> allow users to customize their Spacelift account using policy-as-code. User-defined policies can decide things such as:
<ul>
<li style="font-weight: 400;" aria-level="1">Who gets to log in to your Spacelift account and with what level of access</li>
<li style="font-weight: 400;" aria-level="1">Who gets to access individual Stacks and with what level of access</li>
<li style="font-weight: 400;" aria-level="1">Who can approve or reject a run, and how a run can be approved</li>
<li style="font-weight: 400;" aria-level="1">Which runs and tasks can be started</li>
<li style="font-weight: 400;" aria-level="1">What infrastructure changes can be applied</li>
<li style="font-weight: 400;" aria-level="1">How Git push events are interpreted</li>
<li style="font-weight: 400;" aria-level="1">Which task commands can be executed</li>
<li style="font-weight: 400;" aria-level="1">What happens when blocking runs terminate</li>
</ul>
In addition to policies, Spacelift also offers other unique features such as Resource and Run Observability, a Module Registry, Drift Detection, and more (check <a href="https://docs.spacelift.io/" target="_blank" rel="noopener">Documentation</a>).

In this article, we described Atlantis and its pull-request-driven workflow. We then discussed how users could use the promote functionality to achieve this same pre-merge apply workflow using Spacelift. Furthermore, we talked about using Spacelift to manage your dependencies by combining a trigger policy with labels. 
Atlantis is a great tool for Terraform automation, especially for those looking to run their own completely self-hosted CI/CD solution. Currently, Spacelift does not provide a self-hosted option but instead follows a hybrid approach allowing users to run their workers on their own infrastructure while using the Spacelift UI as a service. If you’d like to see Spacelift implement a completely self-hosted option, let us know in the comments of this article!
Thank you for reading. Feel free to comment and share on your social media platform if it is of any value to you. We wish you the best of luck in your infrastructure automation journey!

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Alternative to Atlantis – Spacelift

5 Ways to Manage Terraform at Scale - Best Practices

See Atlantis and its pull-request-driven workflow and learn how to achieve this same pre-merge apply workflow using Spacelift.

65-achieving-an-atlantis-workflow-on-spacelift

Achieving the Atlantis Workflow with Spacelift

Infrastructure Problems that Spacelift Solves

Stanislaw Szymanski

For every issue, there is a solution.
Infrastructure is no different—due to its intricacies, complexity and the purpose it serves, troubles are guaranteed. Especially when it comes to Infrastructure as Code, as even though this concept was created to fix many extremely difficult problems it did create a few new ones.
For those, Spacelift comes to the rescue. Let’s talk about five infrastructure problems that Spacelift solves for you.

Although the modular approach is great, it presents a little quirk that you might find out about only when it’s too late. A painful one, at that. Modules are clean, efficient, granular… but not immortal. Things in the infrastructure field tend to change and some of those changes might end up breaking your modules—one unfortunate version bump, and things collapse.
Spacelift provides automated testing for the configuration modules it manages and I cannot emphasize enough how significant this is. So significant, in fact, that we would want to dedicate an entire article just to this outstanding functionality (stay tuned for an in-depth explanation of the registry testing). 
To test is to know what to expect. Knowing what to expect gives you peace of mind and confidence that things will work out correctly. Even better—you don’t have to lift a finger to run comprehensive tests for your modules and infrastructure.

image4-5

Broken Modules Tearing Down Your Configurations

Inanimate objects and things have an irritating tendency to not stay as you left them. If you have ever lost your keys, you will surely relate. 
Infrastructure is no different in this matter. Unexpected situations, manual tampering or the workings of force majeure could lead to differences between the configuration you have applied and the current state of the system.
Sometimes the reasons are understandable. For example, when in an emergency your engineer manually introduces crucial infrastructure changes to avoid downtime or other damages; sometimes it’s not just a result of a script or person doing what they shouldn’t be. 
Spacelift introduces <a href="https://docs.spacelift.io/concepts/stack/drift-detection.html" target="_blank" rel="noopener">automatic drift detection runs</a>. That’s right—periodically (how often? You decide or trust the defaults), Spacelift can check your infrastructure, detect drifts, provide you information about the resources they impact:
<ul>
<li style="font-weight: 400;" aria-level="1">What was impacted? </li>
<li style="font-weight: 400;" aria-level="1">When did this happen?</li>
</ul>
Then, you can properly introduce them into your state in a process called reconciliation if you decide that they are beneficial or required.

image5-4

Drifting Away From What You Had Defined

Security is an important factor of every infrastructure, Big or small—if insecure, it won’t stand long. Unfortunately, maintaining security and keeping compliance is cumbersome, to say the least. There are many things to take into consideration and even more that you might forget about&#8230; until they come back to haunt you. 
With some smart tricks, Spacelift makes security much easier—for example, it <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">provides support for policies</a>. And what are those? Internally, Spacelift leverages the well-known <a href="https://spacelift.io/blog/what-is-open-policy-agent-and-how-it-works" target="_blank" rel="noopener">Open Policy Agent</a> and the Rego language. This combination allows you to define policies, affecting various aspects of daily operation such as logging in, starting Tasks and Runs, or command execution.
You can enforce compliance by code. 
Another example is data handling. Data that you exchange with Spacelift is encrypted both at rest and in transit, and temporary credentials are used where needed—the secret data is protected by multiple layers of additional security. 
And when it comes to shared workers and their credentials, granting static, permanent access to your AWS, Azure Cloud or GCP wouldn’t exactly be a good practice. Thanks to Spacelift’s integration with identity management systems, dynamically-generated temporary credentials can be used to provide necessary access during runs.
Whether you don’t trust shared workers or just prefer to provide and use your own with Spacelift —this is also possible.

<ul>
<li><a href="https://spacelift.io/blog/why-devops-engineers-recommend-spacelift" target="_blank" rel="noopener">Why DevOps Engineers Recommend Spacelift</a></li>
<li><a href="https://spacelift.io/blog/spacelift-module-registry" target="_blank" rel="noopener">How to Use Spacelift Module Registry</a></li>
<li><a href="https://spacelift.io/blog/alternative-to-atlantis" target="_blank" rel="noopener">Alternative to Atlantis – Spacelift</a></li>
</ul>

<a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code</a> significantly improves collaboration. It is much easier to work with someone on a piece of code than on a server spinning up, but there are still a few peculiarities that you might find out about along the way. 
Take this example. Let’s assume you’ve got some configuration elements, like environment variables, that you wish to use as shared—between various groups of resources. Normally, you’d need to individually provide those values to each resource set that needs them by defining them multiple times, in different places. To significantly ease this process, Spacelift introduces the <a href="https://docs.spacelift.io/concepts/configuration/context.html" target="_blank" rel="noopener">concept of contexts</a>—grouped declarations of variable values and useful files that you can mount to your stacks. You declare them once and use them where you need, with no security disadvantages or time wasted on useless repetition.
Another situation that you might run into, would be when you’d want to use your <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> in configurations outside your organization. For example, you have five customers. Each one has a separate account, and ideally, you’d want to retain control over the configuration modules you made, to make sure no one breaks anything. But—you also want to use the configuration you already have, for repetitive tasks between the infrastructures of those customers. Spacelift makes this possible:

image3-6

Troublesome Collaboration

Just provide the names of accounts you wish to share your code with, and let those accounts build their infrastructure with your curated resources. Easy!
Still unconvinced? There’s much more. Seamless integration with popular VCS providers such as GitHub or BitBucket, push status notifications… and of course, preview runs, showing you what will change as a result of the pull request.

image2-6

Seamless integration with popular VCS providers

As you can see, there’s something here for everyone. Spacelift is developed with collaboration in mind. It blends in very well but provides features that stand out. Even if you don’t really like using external tools or technologies, you won’t be disappointed with the results.

Everyone at least once in their lifetime bought something and then watched in disbelief as it diminished before their eyes. The pain of a huge Amazon Web Services bill is definitely real. 
AWS bills are often mentioned in the countdowns of some of the most massive things in existence—somewhere between cargo ships, black holes, and the NodeJS node_modules directories. Some engineers even say that you don’t really need coffee in your life if you’re provisioning AWS infrastructure—the bill will raise your blood pressure just the same. 
Spacelift integrates with <a href="https://www.infracost.io/" target="_blank" rel="nofollow noopener">Infracost</a>, to provide you with <a href="https://docs.spacelift.io/vendors/terraform/infracost.html" target="_blank" rel="noopener">cost estimates of your runs</a>. This way your pull request can tell you how much it will cost to apply the configuration you have designed. It also allows you to set up proper cost barriers to make sure that you won’t spend more than expected just by misconfiguration or burn the entire monthly budget in a week by one unfortunate deployment. 
Read more about it in our <a href="https://spacelift.io/blog/terraform-cost-estimation-using-infracost" target="_blank" rel="noopener">Terraform Cost Estimation with Infracost</a> article.
The setup is fairly simple, but to make it even easier you can create a context mentioned in an earlier section and attach it to the stacks you wish to monitor in terms of budget.

image1-6

Budgets Out of Hand

As you’ve seen, while the Infrastructure as Code approach definitely solves many issues, there’s still a huge lot to go around. Every bit of assistance or automation is precious and can provide tremendous benefits.
If you wish to make the management of your infrastructure easier, less time-consuming, the infrastructure itself more robust and dependable… and also keep everything as secure and as under control as possible—<a href="https://spacelift.io/">Spacelift</a> is the way to go. I’m sure you will love it.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence.

Infrastructure as Code : Best Practices, Benefits & Examples

How to Deploy your Infrastructure in CI/CD using Terraform

See what infrastructure problems Spacelift solves. Learn about Spacelift’s automated testing, auto drift detection runs, and cost estimation.

3-infrastructure_problems_that_spacelift_solves

Running Spacelift CICD Workers in Kubernetes Using DinD

Check out the Spacelift Engineering blog to discover internal technical architecture posts written by Spacelift's very own engineers. 

Engineering | Spacelift Blog

Engineering

Adam Connelly

For a while now at Spacelift we’ve been hearing from users that they’d like to be able to run <a href="https://docs.spacelift.io/concepts/worker-pools" target="_blank" rel="noopener">Spacelift worker pools</a> in their Kubernetes clusters. As a first step towards that we decided to investigate whether we could run workers in Kubernetes using a Docker-in-Docker sidecar container. The rest of this post gives a rough overview of the architecture of Spacelift workers, and also explains how the sidecar container strategy works.
I’d just like to say thanks to <a href="https://medium.com/@v.m_hs" target="_blank" rel="nofollow noopener">Vadym Martsynovskyy</a> for his great article about running <a href="https://medium.com/hootsuite-engineering/building-docker-images-inside-kubernetes-42c6af855f25" target="_blank" rel="nofollow noopener">Jenkins agents in Kubernetes</a>. In that article he outlines the general approach taken here, and also describes some of the pros and cons of using Docker-in-Docker. It’s definitely well worth a read.

Before going into more detail, it’s worth explaining what a Spacelift worker is, and giving a quick overview of our architecture. At Spacelift, we provide a hybrid-SaaS model like many other CI/CD systems. What this means in practice is that the control plane that handles scheduling runs is managed by us, but the execution of runs happens in a separate process called a worker. We provide a public worker pool, but also allow you to self-host your own workers, providing complete control over your infrastructure.
Perhaps confusingly, if you follow the instructions to create a private worker pool, you’ll notice that it involves downloading something called the “launcher”. The launcher is responsible for communicating with the Spacelift mothership (control plane), and creating and destroying worker containers using Docker for each Spacelift run:

image2-5

Spacelift mothership (control plane)

As you can see, the launcher process needs access to a Docker daemon in order to create a container for each run.

<ul>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">Top Most Useful CI/CD Tools for DevOps</a></li>
<li><a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Deploy your Infrastructure in CI/CD using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Deploy your Infrastructure in CI/CD using Terraform</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices" target="_blank" rel="noopener">16 DevOps Best Practices to Follow</a></li>
</ul>

We found that the launcher architecture maps very nicely to Kubernetes using a sidecar container to run the Docker daemon. For each worker in the pool, we need to create a Kubernetes Pod with two containers: one for the launcher; and another for Docker. The following shows a stripped down Deployment definition for creating a worker pool with 4 workers:

The launcher communicates with the Docker-in-Docker sidecar container via TCP, which is configured via the <code>DOCKER_HOST</code> environment variable for the launcher container. This works because the containers in a Kubernetes Pod share the same <a href="https://kubernetes.io/docs/concepts/workloads/pods/#pod-networking" target="_blank" rel="nofollow noopener">IP address and port space, and can communicate with each other via localhost</a>.
In addition, a shared volume called <code>launcher-storage</code> is mounted into each container. This is used by the launcher to store the workspaces for runs, along with other things like cached tool binaries (for example Terraform). The Docker sidecar needs access to the run workspaces in order to mount them into the worker containers, and it also uses that volume to store its image cache.
The last thing worth pointing out is that the <code>dind</code> container sets <code>securityContext.privileged</code> to <code>true</code>. This is required for Docker-in-Docker to function correctly.

The Spacelift launcher is distributed as a statically linked binary, making it simple to build an image. As you can see, the Dockerfile simply copies the launcher into an Alpine container, and then sets the startup command:

This image is rebuilt and published to our public ECR repository any time the launcher binary is updated.

We provide <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> to make it really easy for users to deploy worker pools to AWS, Azure and GCP, and we wanted to provide a similar experience for Kubernetes. To achieve this, we created a Helm chart that makes it simple to deploy workers to Kubernetes clusters: <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">https://github.com/spacelift-io/spacelift-workerpool-k8s</a>.
Assuming you’ve already configured your worker pool in Spacelift and have access to the credentials for the pool, deploying this chart to your cluster simply requires two steps.
First, add the Spacelift Helm chart repository and update your local chart cache:

Next, install the chart:

Replace <code>&lt;worker-pool-token&gt;</code> and <code>&lt;worker-pool-private-key&gt;</code> with your own credentials, and make sure to base64-encode the private key.
If all goes well, you should be able to view the pods for your worker pool using <code>kubectl get pods</code>:

You should also be able to view the workers in your pool in Spacelift:

image1-5

Bitbucket DC Local Pool

When implementing in a production environment, you may want to use an alternative approach to providing credentials, and may want to configure a custom storage volume for the worker Pods. For more information about configuring the chart, check out the README in the <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">chart GitHub repo</a>.

If you’re interested in running <a href="https://spacelift.io/blog/spacelift-worker-pools-on-kubernetes" target="_blank" rel="noopener">Spacelift workers in Kubernetes</a>, we’d welcome any feedback about the approach, contributions to the Helm chart, and also any issues you encounter so that we can make improvements. Also, if you aren’t already using Spacelift but are interested in trying it, you can signup and <a href="https://spacelift.io/free-trial" target="_blank" rel="noopener">start your free evaluation of Spacelift</a>.

This article is an overview of the architecture of Spacelift workers with an explanation of how the sidecar container strategy works.

49-running_spacelift_cicd_workers_in_kubernetes_using_dind-3

Running Spacelift CI/CD workers in Kubernetes using DinD

At Spacelift, we&#8217;re excited to announce our new fully-managed integration with Azure. This integration makes it even easier for customers to get up and running fast without having to manually manage Azure Service Principal credentials.

In keeping with our tagline &#8220;the most flexible management platform for Infrastructure as Code&#8221;, and to give you maximum flexibility, you have multiple choices to integrate Spacelift with Azure:
<ul>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure#spacelift-managed-integration" target="_blank" rel="noopener">Using our Azure integration</a>.</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure#static-credentials" target="_blank" rel="noopener">Manually configuring Azure credentials via environment variables</a>.</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure#managed-identities" target="_blank" rel="noopener">Running a private worker, and using a Managed Identity</a>.</li>
</ul>
Spacelift’s Azure integration is aimed at customers who want a fully managed experience, new customers who want to quickly get up and running, and existing customers who are manually managing their Azure credentials via Spacelift environment variables or contexts.
We recommend that customers who desire more control for security or compliance reasons should continue using private runners.

Spacelift’s Azure integration provides the following features:
<ul>
<li style="font-weight: 400;" aria-level="1">Easily connect to your Azure subscriptions and attach them to your stacks.</li>
<li style="font-weight: 400;" aria-level="1">Automatic credential rotation.</li>
<li style="font-weight: 400;" aria-level="1">The ability to use granular permissions depending on the action being performed.</li>
</ul>

To connect to your Azure subscription, simply add a new Azure integration via your account settings, specifying your Tenant ID and optionally a default subscription ID:

Azure Integration

Install the Enterprise Application for the integration into your Azure AD tenant using the Provide Consent button:

Azure AD Provide Consent

Setup your permissions in Azure:

setting up permissions in Azure

Attach the integration to any Spacelift stacks that require it:

Attaching the integration to any Spacelift stacks

That’s it, you’re now ready for launch!

When you set up an Azure integration in Spacelift, a Service Principal is created in your directory that you can use to manage permissions. We automatically rotate the credentials for this Service Principal roughly every 24 hours while making sure that there is always a valid credential for your stacks to use.

<ul>
<li><a href="https://spacelift.io/blog/why-devops-engineers-recommend-spacelift" target="_blank" rel="noopener">Why DevOps Engineers Recommend Spacelift</a></li>
<li><a href="https://spacelift.io/blog/spacelift-worker-pools-on-kubernetes" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Spacelift Worker Pools on Kubernetes with Spacelift&quot;}" data-sheets-userformat="{&quot;2&quot;:4224,&quot;10&quot;:2,&quot;15&quot;:&quot;Arial&quot;}">Spacelift Worker Pools on Kubernetes with Spacelift</a></li>
<li><a href="https://spacelift.io/blog/spacelift-spaces-in-action" target="_blank" rel="noopener">Spacelift Spaces in Action</a></li>
</ul>

Spacelift’s Azure integration supports granular permissions by allowing you to setup more than one integration per tenant:

Spacelift’s Azure integration

Each integration will create a Service Principal in Azure with a unique display name:

Service Principal

You can then assign the relevant permissions to each role in Azure:

assigning the relevant permissions

And finally attach them to your stack as read or write:

read or write integration

Once you’ve attached both Azure integrations to your stack, Spacelift will automatically use the correct one depending on the phase of the run being executed. This allows you to provide read-only roles to planning runs (for example PRs), while providing write access for deployments.

To find out more about the Azure integration, take a look at our documentation available at <a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="noopener">Spacelift Documentation</a>. You might also be interested in the detailed overview of <a href="https://spacelift.io/blog/building-secure-cicd-integration-with-azure" target="_blank" rel="noopener">How we Built a Secure CI/CD Integration with Azure</a> and some of the issues we encountered and solved while designing and developing it.

Building a Secure CI/CD Integration with Azure

We're excited to announce our new fully-managed integration with Azure. This integration makes it even easier for customers to get up and running fast without having to manually manage Azure Service Principal credentials.

1-azure-integration

Introducing the Spacelift Azure Integration

Marcin Wyszynski

The introduction of Infrastructure as Code or IaC has transformed the way you can provision and deploy high-performance cloud-based IT infrastructures.
IaC tools, such as Terraform, have been integrated into DevOps toolchains, saving DevOps IaC teams from the excessive manual effort.
While these tools undoubtedly help accelerate building IT infrastructures, their limitations can impact DevOps’ ability to optimize and improve control of their IaC processes supporting future business needs.
In this article, you’ll learn about IT infrastructure limitations that IaC DevOps teams deal with on a daily basis and how Spacelift is able to get past them.

One of the most frequent challenges while using more generic IaC tools is the non-intuitive workflow driven by its reliance on pull requests. Some solutions offer multiple workspaces, but the result can be fragile and nondeterministic. 
Since there’s no concept of mapping projects to branches or tags, anyone commenting on an approved pull request can deploy arbitrary code to production, even if the approval was meant for a short-lived experimental environment.
Spacelift does not depend on pull requests. It is mostly driven by push and tag events, so building a sophisticated Git flow is much easier. Spacelift reports the outcome of its jobs as commit status checks, allowing you to block merging the code on a failing Spacelift check.
Triggering a run can be customized using <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">Git push policies</a>. Thanks to that, Spacelift can provide the same level of comfort and security to teams using one project per repository and those using mono repo with hundreds of interdependent projects. You can read more about our approach to VCS integration <a href="https://docs.spacelift.io/integrations/source-control/github#tracked-branches" target="_blank" rel="noopener">here</a>.

The majority of generic tools don’t offer access control models but rely on comments on pull requests to drive infrastructure deployments. While it is usually fine when a single repository drives a single Terraform project, it becomes a huge liability for more complex scenarios.
Spacelift ships with a sophisticated mechanism allowing administrators to declare <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">who can log in</a> (and under what circumstances) and what their <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">level of access</a> to each of the managed projects should be. Even our <a href="https://docs.spacelift.io/integrations/slack" target="_blank" rel="noopener">Slack integration</a> can be subject to policy controls, allowing an admin to grant access to a project <a href="https://docs.spacelift.io/integrations/slack#managing-access-to-stacks-with-policies" target="_blank" rel="noopener">based on Slack-specific data</a> (think team, channel, user, etc.).

One thing that’s not in scope for most IaC solutions is the way to ensure that your infrastructure is compliant with industry best practices and your company policies.
Spacelift puts policy-as-code in the center of its value proposition and builds a consistent, <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">robust policy framework</a> on top of <a href="https://www.openpolicyagent.org/" target="_blank" rel="nofollow noopener">Open Policy Agent</a>. Apart from providing a comprehensive automated change review and ensuring compliance of your <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">Terraform changes</a>, Spacelift uses the same approach to allow you to declare rules around the <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">account</a> and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">project access</a>, <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">handling push notifications</a>, <a href="https://docs.spacelift.io/concepts/policy/run-initialization-policy" target="_blank" rel="noopener">starting runs</a> and <a href="https://docs.spacelift.io/concepts/policy/task-run-policy" target="_blank" rel="noopener">triggering tasks</a>, and creating <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">relationships between projects</a>.

Handling interdependencies between projects has always been Terraform’s Achilles’ heel. The usual approach to this problem is adding another layer of abstraction in the form of a Terraform wrapper like <a href="https://terragrunt.gruntwork.io/" target="_blank" rel="nofollow noopener">Terragrunt</a>. But it’s only a partial solution as it breaks the problem into smaller chunks.
<a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">Spacelift’s trigger policies</a> on the other hand provide a smart, declarative automation layer on top of vanilla Terraform. They allow you to plug into state changes of individual projects and declare dependencies that should be resolved, following the changes that have just been applied. Read more <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">here</a> to discover other exciting possibilities.

<ul>
<li><a href="https://spacelift.io/blog/spacelift-module-registry" target="_blank" rel="noopener">How to Use Spacelift Module Registry</a></li>
<li><a href="https://spacelift.io/blog/why-devops-engineers-recommend-spacelift" target="_blank" rel="noopener">Why DevOps Engineers Recommend Spacelift</a></li>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Infrastructure Problems that Spacelift Solves&quot;}" data-sheets-userformat="{&quot;2&quot;:4224,&quot;10&quot;:2,&quot;15&quot;:&quot;Arial&quot;}">Infrastructure Problems that Spacelift Solves</a></li>
</ul>

Another problem to solve externally when using some of the generic tools is authoring and maintaining reusable <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> for your organization. Terraform is flexible in allowing modules to come from <a href="https://www.terraform.io/docs/modules/sources.html" target="_blank" rel="nofollow noopener">various sources</a>, but ensuring confidential access, as well as testing and versioning, are left to you, the user.
Until now, the golden standard in that regard has been the <a href="https://www.terraform.io/docs/cloud/registry/index.html" target="_blank" rel="nofollow noopener">private module registry from HashiCorp</a>. But Spacelift offers much more. Far from being just a glorified package manager, Spacelift adds a <a href="https://docs.spacelift.io/vendors/terraform/module-registry" target="_blank" rel="noopener">full CI solution for Terraform modules</a>, out of the box and free of charge. You can thus ensure that your <a href="https://spacelift.io/blog/spacelift-module-registry" target="_blank" rel="noopener">private modules</a> are healthy before you distribute them to the rest of your organization.

If you manage a single or a handful of rarely changing projects, it’s likely that you just set your IaC up once and forget about it. But in a more dynamic environment, where microservices come and go, new environments proliferate and new product teams require their own Terraform workspaces. The need to configure it each and every time become a major nuisance, putting a lot of pressure on your DevOps team.
Enter Spacelift. In Spacelift, much of the configuration can be handled by the project owners themselves—you can add Terraform and/or <a href="https://docs.spacelift.io/concepts/configuration/environment" target="_blank" rel="noopener">environment variables</a> and mount files (even inject Terraform code!) programmatically or through the GUI without the need for administrative privileges or changing the central server configuration. For administrators, adding new projects requires minimal hassle since there’s no need to set up webhooks or change any YAML. And it can all be done programmatically using Terraform.

What comes as a pleasant surprise to users of generic CI tools, Spacelift entities such as stacks, contexts, modules or policies can be managed in a declarative way using your favorite infra-as-code tool (this rule applies also to their configuration). Yes, that’s right—<a href="https://docs.spacelift.io/vendors/terraform/terraform-provider" target="_blank" rel="noopener">Spacelift offers a Terraform provider</a> that allows you to programmatically manage the lifecycle of its own resources.
Administrative stacks get credential-less access to the subset of our <a href="https://docs.spacelift.io/integrations/api" target="_blank" rel="noopener">GraphQL API</a> that does not involve managing the actual infrastructure. For more sophisticated use cases, Spacelift allows you to generate API keys that are subject to the same access controls as normal users are, allowing you to create single-purpose tokens for restricted use by your internal scripts.

Generic IaC platforms do not provide any mechanisms to detect if your infrastructure is undergoing drift. Drift is a condition that represents the difference between the desired and the actual state of the infrastructure managed by your tool of choice &#8211; <a href="https://www.terraform.io/" target="_blank" rel="nofollow noopener">Terraform</a>, <a href="https://www.pulumi.com/" target="_blank" rel="nofollow noopener">Pulumi</a>, <a href="https://aws.amazon.com/cloudformation/" target="_blank" rel="nofollow noopener">CloudFormation,</a><a href="https://docs.spacelift.io/vendors/kubernetes/" rel=""> Kubernetes</a>, or another. Drift can be caused by either or a combination of changes directly introduced by external actors &#8211; either humans or machines (scripts) or via the dependency of your resources on external data sources. In any case, drift is not good.
Spacelift has got you covered here. You can configure periodic <a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">drift detection </a>to notify you whenever drift happens, and take immediate action. You can even go a step further with optional automatic <a href="https://docs.spacelift.io/concepts/stack/drift-detection#to-reconcile-or-not-to-reconcile" target="_blank" rel="noopener">reconciliation</a>, ensuring your infrastructure always resembles your Terraform configuration.

General-purpose CI/CD platforms provide little to no insight into resource utilization from either a real-time or historical perspective. Which resources are over-or underutilized? 
Developers need to be able to intimately understand the material they’re working with. With regards to infra-as-code, the most important part of this story is understanding the managed resources in-depth. Both from the current perspective and through being able to put each resource in its historical context.
The resources view in Spacelift gives you greater visibility into each and every resource. With this deep insight into resources, DevOps are able to gain an understanding of the lifecycle of each resource managed by Spacelift and document it, regardless of the technology used — <a href="https://www.terraform.io/" target="_blank" rel="nofollow noopener">Terraform</a>, <a href="https://github.com/gruntwork-io/terragrunt" target="_blank" rel="nofollow noopener">Terragrunt</a>, <a href="https://spacelift.io/blog/what-is-pulumi" target="_blank" rel="noopener">Pulumi</a>, <a href="https://docs.spacelift.io/vendors/kubernetes/" target="_blank" rel="noopener">Kubernetes</a>, or <a href="https://aws.amazon.com/cloudformation/" target="_blank" rel="nofollow noopener">CloudFormation</a>.

Last but not least, Spacelift puts an emphasis on great user experience, offering a myriad of creature comforts. <a href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="noopener">Contexts</a> for example allow you to attach entire collections of configuration to individual stacks and modules. <a href="https://docs.spacelift.io/concepts/run/task" target="_blank" rel="noopener">Tasks</a> provide a powerful audited way of running one-off administrative commands on an initialized Terraform environment &#8211; subject to <a href="https://docs.spacelift.io/concepts/run/task" target="_blank" rel="noopener">their own policy constraints</a>. <a href="https://docs.spacelift.io/concepts/stack/#stack-locking" target="_blank" rel="noopener">Stack locking</a> allows a single individual to take exclusive control over a stack to ensure that nobody is able to modify its state while crucial changes are being made.

Spacelift is an innovative and sophisticated SaaS product for Infrastructure as Code which helps IaC DevOps develop and deploy new infrastructures or changes quickly and with confidence.
Spacelift offers a unique set of IaC management capabilities:
<ol>
<li style="font-weight: 400;" aria-level="1">an intuitive, <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">versatile</a> and <a href="https://docs.spacelift.io/concepts/run/#run-state-machine" target="_blank" rel="noopener">robust</a> workflow</li>
<li style="font-weight: 400;" aria-level="1">granular access controls on <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">account</a> and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">project</a> level that work well with an <a href="https://docs.spacelift.io/integrations/single-sign-on/" target="_blank" rel="noopener">identity provider of your choice</a> (SSO)</li>
<li style="font-weight: 400;" aria-level="1">an <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">automated code review</a> and <a href="https://docs.spacelift.io/concepts/policy/task-run-policy" target="_blank" rel="noopener">threat detection</a> </li>
<li style="font-weight: 400;" aria-level="1">the ability to <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">declare complex workflows</a> between projects across multiple repositories</li>
<li style="font-weight: 400;" aria-level="1">a <a href="https://docs.spacelift.io/vendors/terraform/module-registry" target="_blank" rel="noopener">built-in private module registry</a> with a full CI system for modules</li>
<li style="font-weight: 400;" aria-level="1">effortless setup and customization with <a href="https://docs.spacelift.io/concepts/configuration/environment" target="_blank" rel="noopener">per-project environment management</a> and <a href="https://docs.spacelift.io/integrations/docker" target="_blank" rel="noopener">Docker integration</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/vendors/terraform/terraform-provider" target="_blank" rel="noopener">programmatic configuration</a> using Terraform</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">drift detection </a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/concepts/resources" target="_blank" rel="noopener">resource visualization</a></li>
</ol>
… and a multitude of additional perks like <a href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="noopener">contexts</a>, <a href="https://docs.spacelift.io/concepts/run/task" target="_blank" rel="noopener">tasks</a> or <a href="https://docs.spacelift.io/concepts/stack/#stack-locking" target="_blank" rel="noopener">stack locking</a>.

There are many ways to manage your Infrastructure as Code. Each way is different in terms of complexity and offers a different set of features. It is important to keep in mind that choosing one way or another should be based on business and technical requirements. Most often, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may exceed its potential benefits. It is much easier and more efficient to leverage platforms such as Spacelift to provide these features for you instead.

Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures

Terraform vs. Ansible : Key Differences and Comparison of Tools

Infrastructure Drift Detection and How to Fix It With IaC Tools

Most often, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may exceed its potential benefits. It is much easier and more efficient to leverage ready-to-go platform to provide these features instead.

how_spacelift_can_improve_your_iac

How Spacelift Can Improve Your Infrastructure as Code 

In this post I want to show how easy it is to create a <a class="dt id" href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a> Stack that manages Azure resources. To keep things simple, I’m going to assume that you already know what Spacelift can do, and the basics of creating and running Stacks, but if not check out our <a class="dt id" href="https://github.com/spacelift-io/terraform-starter" target="_blank" rel="nofollow noopener">terraform-starter</a> repo that walks you through the process of setting up an account, and getting started.

In order to allow Spacelift to manage Azure resources, we need to do the following:
<ul class="">
<li id="f0f8" class="hj hk fo hl b hm jf hn ho hp jg hq hr hs jh ht hu hv ji hw hx hy jj hz ia ic jk jl jm gk" data-selectable-paragraph="">Create a Service Principal in Azure with the correct ARM permissions to manage our resources.</li>
<li id="8f2c" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">Create a Stack in Spacelift that uses the <a class="dt id" href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs" target="_blank" rel="nofollow noopener">Azure</a> provider.</li>
<li id="ba4d" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">Give our Stack the credentials for our Service Principal.</li>
</ul>
All of the information you need to configure the Azure provider can be found <a class="dt id" href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#authenticating-to-azure" target="_blank" rel="nofollow noopener">here</a>. This post just explains how to combine that information with Spacelift.

To create our Service Principal, we need to use the <code class="js jt ju jv jw b">az ad sp create-for-rbac</code> command (replacing <code class="js jt ju jv jw b">&lt;subscription-id&gt;</code> with your actual subscription ID):

This will output something like the following:

The command creates a new Service Principal called <code class="js jt ju jv jw b">spacelift-sp</code>, and grants it the Contributor role on your subscription. It also outputs the <code class="js jt ju jv jw b">appId</code>, <code class="js jt ju jv jw b">password</code> and <code class="js jt ju jv jw b">tenant</code> for the Service Principal. Make a note of these because you&#8217;ll need them later.
If you would rather assign permissions yourself later, you can run the following command to just create a Service Principal with no permissions:

You should now be able to view your Service Principal in the Azure Active Directory -&gt; App registrations section of the Azure Portal:

default directory

Now that we’ve got a service principal, we need to create a Stack to actually create some Azure resources. To keep things simple all we’re going to do is add some terraform definitions that specify the version of the Azure Provider we want to use, and create an Azure Resource Group:

If you commit that change and try to run your Stack just now, you should see something similar to the following failure:

adding Azure resources

Here’s the full error message:
Error building AzureRM Client: obtain subscription() from Azure CLI: Error parsing json result from the Azure CLI: Error launching Azure CLI: exec: “az”: executable file not found in $PATH
At first glance this error is slightly confusing, but what it means is that because we didn’t provide any credentials to the Azure provider it attempted to fallback to using the <code class="js jt ju jv jw b">az</code> CLI for authentication. This works fine locally, but won&#8217;t work on Spacelift.
To solve this, we need to provide our stack with our credentials, which is explained in the next section.

<ul>
<li><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Improve Your Infrastructure as Code using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Improve Your Infrastructure as Code using Terraform</a></li>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Common Infrastructure Challenges and How to Solve Them&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Common Infrastructure Challenges and How to Solve Them</a></li>
<li><a href="https://spacelift.io/blog/terraform-environments" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Managing Multiple Terraform Environments Efficiently&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Managing Multiple Terraform Environments Efficiently</a></li>
</ul>

If you take a look at the <a class="dt id" href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#authenticating-to-azure" target="_blank" rel="nofollow noopener">Terraform documentation</a>, you’ll find that there are a number of different ways that you can authenticate <a href="https://spacelift.io/blog/what-is-terraform" target="_blank" rel="noopener">Terraform</a> with Azure. What we’re going to do here is authenticate using a Service Principal with a Client Secret, and provide that configuration via <a href="https://spacelift.io/blog/how-to-use-terraform-variables" target="_blank" rel="noopener">environment variables</a>.
To do this we need to set the following environment variables, using the information returned by the Azure CLI when we created our Service Principal:
<ul class="">
<li id="1508" class="hj hk fo hl b hm jf hn ho hp jg hq hr hs jh ht hu hv ji hw hx hy jj hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_CLIENT_ID — the <code class="js jt ju jv jw b">appId</code> from the response.</li>
<li id="4e7d" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_CLIENT_SECRET — the <code class="js jt ju jv jw b">password</code> from the response.</li>
<li id="8a57" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_SUBSCRIPTION_ID — your subscription ID.</li>
<li id="7340" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_TENANT_ID — the <code class="js jt ju jv jw b">tenant</code> from the response.</li>
</ul>
You can set these environment variables directly on your Stack, but what we’re going to do here is create a <a class="dt id" href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="noopener">Spacelift Context</a> and attach it to our Stack. This allows us to share the same credentials with multiple Stacks if we want.
Add a new Context via the Contexts screen in Spacelift:

3-2-2

adding contexts

Once your context has been created, edit it and enter your environment variables, making sure to mark the <code class="js jt ju jv jw b">ARM_CLIENT_SECRET</code> as a secret rather than plaintext:

environment variables

Now that we’ve got a Context containing our credentials, you can attach it to your Stack. To do this, click on the Manage link next to your Stack, go to Contexts, choose your Context, and click Attach:

5-2-2

Azure stack

Now that your Context is attached, you should be able to see the it in the Environment section of your Stack:

Azure credentials

If you try to run your Azure Stack again you should get a successful plan:

running Azure Stack again

applying

If you take a look in the Azure Portal, you should now be able to see the Resource Group that was created:

Azure resource groups

Before finishing up, I wanted to just point out a few things about the Service Principal credential expiry. Although this isn’t really related to Spacelift, it’s something that you need to be aware of.
When you create your Service Principal via the Azure CLI, it automatically creates a Client Secret for you with an expiry of 1 year:

Spacelift Client Secret

What this means is that if you do absolutely nothing, your Spacelift Stacks will stop working after the credentials expire a year later. To solve this, all you need to do is generate a new Client Secret, and update the <code class="js jt ju jv jw b">ARM_CLIENT_SECRET</code> environment variable in your Context.

Hopefully, in this post, I’ve shown that although Spacelift doesn’t have an official Azure integration (yet!), it’s simple to manage Azure resources using Spacelift. Once you’ve performed a little bit of initial setup, you should be able to rapidly get new Stacks up and running, creating Azure resources.

It is easy to manage Azure resources using Spacelift. After a bit of initial setup, you will rapidly get new Stacks up and running, creating Azure resources.

Initial Availability of Kubernetes Integration in Spacelift

Get the newsletter

Recent posts

Achieving the Atlantis Pull Request Automation with Spacelift

Infrastructure Problems that Spacelift Solves

Running Spacelift CI/CD workers in Kubernetes using DinD

Introducing the Spacelift Azure Integration!

How Spacelift Can Improve Your Infrastructure as Code

Managing Azure Resources using Spacelift

Alternative to Atlantis – Spacelift