Learn more about new features, updates, and Spacelift product announcements.

Product Features and Updates | Spacelift Blog

Author title

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

Infrastructure Problems That Spacelift Solves

Product

For every issue, there is a solution.
Infrastructure is no different—due to its intricacies, complexity and the purpose it serves, troubles are guaranteed. Especially when it comes to Infrastructure as Code, as even though this concept was created to fix many extremely difficult problems it did create a few new ones.
For those, Spacelift comes to the rescue. Let’s talk about five infrastructure problems that Spacelift solves for you.

Although the modular approach is great, it presents a little quirk that you might find out about only when it’s too late. A painful one, at that. Modules are clean, efficient, granular… but not immortal. Things in the infrastructure field tend to change and some of those changes might end up breaking your modules—one unfortunate version bump, and things collapse.
Spacelift provides automated testing for the configuration modules it manages and I cannot emphasize enough how significant this is. So significant, in fact, that we would want to dedicate an entire article just to this outstanding functionality (stay tuned for an in-depth explanation of the registry testing). 
To test is to know what to expect. Knowing what to expect gives you peace of mind and confidence that things will work out correctly. Even better—you don’t have to lift a finger to run comprehensive tests for your modules and infrastructure.

image4-5

Broken Modules Tearing Down Your Configurations

Inanimate objects and things have an irritating tendency to not stay as you left them. If you have ever lost your keys, you will surely relate. 
Infrastructure is no different in this matter. Unexpected situations, manual tampering or the workings of force majeure could lead to differences between the configuration you have applied and the current state of the system.
Sometimes the reasons are understandable. For example, when in an emergency your engineer manually introduces crucial infrastructure changes to avoid downtime or other damages; sometimes it’s not just a result of a script or person doing what they shouldn’t be. 
Spacelift introduces automatic drift detection runs. That’s right—periodically (how often? You decide or trust the defaults), Spacelift can check your infrastructure, detect drifts, provide you information about the resources they impact:
<ul>
<li style="font-weight: 400;" aria-level="1">What was impacted? </li>
<li style="font-weight: 400;" aria-level="1">When did this happen?</li>
</ul>
Then, you can properly introduce them into your state in a process called reconciliation if you decide that they are beneficial or required.

image5-4

Drifting Away From What You Had Defined

Security is an important factor of every infrastructure, Big or small—if insecure, it won’t stand long. Unfortunately, maintaining security and keeping compliance is cumbersome, to say the least. There are many things to take into consideration and even more that you might forget about&#8230; until they come back to haunt you. 
With some smart tricks, Spacelift makes security much easier—for example, it provides support for policies. And what are those? Internally, Spacelift leverages the well-known Open Policy Agent and the Rego language. This combination allows you to define policies, affecting various aspects of daily operation such as logging in, starting Tasks and Runs, or command execution.
You can enforce compliance by code. 
Another example is data handling. Data that you exchange with Spacelift is encrypted both at rest and in transit, and temporary credentials are used where needed—the secret data is protected by multiple layers of additional security. 
And when it comes to shared workers and their credentials, granting static, permanent access to your AWS, Azure Cloud or GCP wouldn’t exactly be a good practice. Thanks to Spacelift’s integration with identity management systems, dynamically-generated temporary credentials can be used to provide necessary access during runs.
Whether you don’t trust shared workers or just prefer to provide and use your own with Spacelift —this is also possible.

<a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code</a> significantly improves collaboration. It is much easier to work with someone on a piece of code than on a server spinning up, but there are still a few peculiarities that you might find out about along the way. 
Take this example. Let’s assume you’ve got some configuration elements, like <a href="https://spacelift.io/blog/how-to-use-terraform-variables" target="_blank" rel="noopener">environment variables</a>, that you wish to use as shared—between various groups of resources. Normally, you’d need to individually provide those values to each resource set that needs them by defining them multiple times, in different places. To significantly ease this process, Spacelift introduces the concept of contexts—grouped declarations of variable values and useful files that you can mount to your stacks. You declare them once and use them where you need, with no security disadvantages or time wasted on useless repetition.
Another situation that you might run into, would be when you’d want to use your <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> in configurations outside your organization. For example, you have five customers. Each one has a separate account, and ideally, you’d want to retain control over the configuration modules you made, to make sure no one breaks anything. But—you also want to use the configuration you already have, for repetitive tasks between the infrastructures of those customers. Spacelift makes this possible:

image3-6

Troublesome Collaboration

Just provide the names of accounts you wish to share your code with, and let those accounts build their infrastructure with your curated resources. Easy!
Still unconvinced? There’s much more. Seamless integration with popular VCS providers such as GitHub or BitBucket, push status notifications… and of course, preview runs, showing you what will change as a result of the pull request.

image2-6

Seamless integration with popular VCS providers

As you can see, there’s something here for everyone. Spacelift is developed with collaboration in mind. It blends in very well but provides features that stand out. Even if you don’t really like using external tools or technologies, you won’t be disappointed with the results.

Everyone at least once in their lifetime bought something and then watched in disbelief as it diminished before their eyes. The pain of a huge Amazon Web Services bill is definitely real. 
AWS bills are often mentioned in the countdowns of some of the most massive things in existence—somewhere between cargo ships, black holes, and the NodeJS node_modules directories. Some engineers even say that you don’t really need coffee in your life if you’re provisioning AWS infrastructure—the bill will raise your blood pressure just the same. 
Spacelift integrates with <a href="https://www.infracost.io/" target="_blank" rel="noopener">Infracost</a>, to provide you with cost estimates of your runs. This way your pull request can tell you how much it will cost to apply the configuration you have designed. It also allows you to set up proper cost barriers to make sure that you won’t spend more than expected just by misconfiguration or burn the entire monthly budget in a week by one unfortunate deployment. 
The setup is fairly simple, but to make it even easier you can create a context mentioned in an earlier section and attach it to the stacks you wish to monitor in terms of budget.

image1-6

Budgets Out of Hand

As you’ve seen, while the Infrastructure as Code approach definitely solves many issues, there’s still a huge lot to go around. Every bit of assistance or automation is precious and can provide tremendous benefits.
If you wish to make the management of your infrastructure easier, less time-consuming, the infrastructure itself more robust and dependable… and also keep everything as secure and as under control as possible—Spacelift is the way to go. I’m sure you will love it.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence.

What Is Infrastructure as Code? Examples, Best Practices &amp; Tools

Managing Azure Resources using Spacelift

How to Deploy your Infrastructure in CI/CD using Terraform

Learn what infrastructure problems Spacelift solves. See how Spacelift’s automated testing, auto drift detection runs, and cost estimation can help you manage your infrastructure.

infrastructure-problems-spacelift-solves

Infrastructure Problems that Spacelift Solves

Running Spacelift CI/CD workers in Kubernetes using DinD

Check out the Spacelift Engineering blog to discover internal technical architecture posts written by Spacelift's very own engineers. 

Engineering | Spacelift Blog

Engineering

For a while now at Spacelift we’ve been hearing from users that they’d like to be able to run <a href="https://docs.spacelift.io/concepts/worker-pools" target="_blank" rel="noopener">Spacelift worker pools</a> in their Kubernetes clusters. As a first step towards that we decided to investigate whether we could run workers in Kubernetes using a Docker-in-Docker sidecar container. The rest of this post gives a rough overview of the architecture of Spacelift workers, and also explains how the sidecar container strategy works.
I’d just like to say thanks to <a href="https://medium.com/@v.m_hs" target="_blank" rel="nofollow noopener">Vadym Martsynovskyy</a> for his great article about running <a href="https://medium.com/hootsuite-engineering/building-docker-images-inside-kubernetes-42c6af855f25" target="_blank" rel="nofollow noopener">Jenkins agents in Kubernetes</a>. In that article he outlines the general approach taken here, and also describes some of the pros and cons of using Docker-in-Docker. It’s definitely well worth a read.

Before going into more detail, it’s worth explaining what a Spacelift worker is, and giving a quick overview of our architecture. At Spacelift, we provide a hybrid-SaaS model like many other CI/CD systems. What this means in practice is that the control plane that handles scheduling runs is managed by us, but the execution of runs happens in a separate process called a worker. We provide a public worker pool, but also allow you to self-host your own workers, providing complete control over your infrastructure.
Perhaps confusingly, if you follow the instructions to create a private worker pool, you’ll notice that it involves downloading something called the “launcher”. The launcher is responsible for communicating with the Spacelift mothership (control plane), and creating and destroying worker containers using Docker for each Spacelift run:

image2-5

Spacelift mothership (control plane)

As you can see, the launcher process needs access to a Docker daemon in order to create a container for each run.

We found that the launcher architecture maps very nicely to Kubernetes using a sidecar container to run the Docker daemon. For each worker in the pool, we need to create a Kubernetes Pod with two containers: one for the launcher; and another for Docker. The following shows a stripped down Deployment definition for creating a worker pool with 4 workers:

The launcher communicates with the Docker-in-Docker sidecar container via TCP, which is configured via the <code>DOCKER_HOST</code> environment variable for the launcher container. This works because the containers in a Kubernetes Pod share the same <a href="https://kubernetes.io/docs/concepts/workloads/pods/#pod-networking" target="_blank" rel="nofollow noopener">IP address and port space, and can communicate with each other via localhost</a>.
In addition, a shared volume called <code>launcher-storage</code> is mounted into each container. This is used by the launcher to store the workspaces for runs, along with other things like cached tool binaries (for example Terraform). The Docker sidecar needs access to the run workspaces in order to mount them into the worker containers, and it also uses that volume to store its image cache.
The last thing worth pointing out is that the <code>dind</code> container sets <code>securityContext.privileged</code> to <code>true</code>. This is required for Docker-in-Docker to function correctly.

The Spacelift launcher is distributed as a statically linked binary, making it simple to build an image. As you can see, the Dockerfile simply copies the launcher into an Alpine container, and then sets the startup command:

This image is rebuilt and published to our public ECR repository any time the launcher binary is updated.

We provide <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work">Terraform modules</a> to make it really easy for users to deploy worker pools to AWS, Azure and GCP, and we wanted to provide a similar experience for Kubernetes. To achieve this, we created a Helm chart that makes it simple to deploy workers to Kubernetes clusters: <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">https://github.com/spacelift-io/spacelift-workerpool-k8s</a>.
Assuming you’ve already configured your worker pool in Spacelift and have access to the credentials for the pool, deploying this chart to your cluster simply requires two steps.
First, add the Spacelift Helm chart repository and update your local chart cache:

Next, install the chart:

Replace <code>&lt;worker-pool-token&gt;</code> and <code>&lt;worker-pool-private-key&gt;</code> with your own credentials, and make sure to base64-encode the private key.
If all goes well, you should be able to view the pods for your worker pool using <code>kubectl get pods</code>:

You should also be able to view the workers in your pool in Spacelift:

image1-5

Bitbucket DC Local Pool

When implementing in a production environment, you may want to use an alternative approach to providing credentials, and may want to configure a custom storage volume for the worker Pods. For more information about configuring the chart, check out the README in the <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">chart GitHub repo</a>.

If you’re interested in running Spacelift workers in Kubernetes, we’d welcome any feedback about the approach, contributions to the Helm chart, and also any issues you encounter so that we can make improvements. Also, if you aren’t already using Spacelift but are interested in trying it, you can signup and start your free evaluation of Spacelift <a href="https://spacelift.io/free-trial" target="_blank" rel="noopener">here</a>.

In this post, we discuss how to manage the destruction of cloud infrastructure resources that are under Terraform management. By Terraform management we mean – resources that are created using Terraform IaC. The topics here cover what we mean by the destruction of such resources and how to overcome some common pitfalls.

Pro tip

5 Ways to Manage Terraform at Scale

Building a Secure CI/CD Integration with Azure

How a Specialized Management Platform Can Improve Your IaC

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

This article is an overview of the architecture of Spacelift workers with an explanation of how the sidecar container strategy works.

1-4-2

Manage Active Directory Objects

At Spacelift, we&#8217;re excited to announce our new fully-managed integration with Azure. This integration makes it even easier for customers to get up and running fast without having to manually manage Azure Service Principal credentials.

In keeping with our tagline &#8220;the most flexible management platform for Infrastructure as Code&#8221;, and to give you maximum flexibility, you have multiple choices to integrate Spacelift with Azure:
<ul>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure#spacelift-managed-integration" target="_blank" rel="noopener">Using our Azure integration</a>.</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure#static-credentials" target="_blank" rel="noopener">Manually configuring Azure credentials via environment variables</a>.</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure#managed-identities" target="_blank" rel="noopener">Running a private worker, and using a Managed Identity</a>.</li>
</ul>
Spacelift’s Azure integration is aimed at customers who want a fully managed experience, new customers who want to quickly get up and running, and existing customers who are manually managing their Azure credentials via Spacelift environment variables or contexts.
We recommend that customers who desire more control for security or compliance reasons should continue using private runners.

Spacelift’s Azure integration provides the following features:
<ul>
<li style="font-weight: 400;" aria-level="1">Easily connect to your Azure subscriptions and attach them to your stacks.</li>
<li style="font-weight: 400;" aria-level="1">Automatic credential rotation.</li>
<li style="font-weight: 400;" aria-level="1">The ability to use granular permissions depending on the action being performed.</li>
</ul>

To connect to your Azure subscription, simply add a new Azure integration via your account settings, specifying your Tenant ID and optionally a default subscription ID:

Azure Integration

Install the Enterprise Application for the integration into your Azure AD tenant using the Provide Consent button:

Azure AD Provide Consent

Setup your permissions in Azure:

setting up permissions in Azure

Attach the integration to any Spacelift stacks that require it:

Attaching the integration to any Spacelift stacks

That’s it, you’re now ready for launch!

When you set up an Azure integration in Spacelift, a Service Principal is created in your directory that you can use to manage permissions. We automatically rotate the credentials for this Service Principal roughly every 24 hours while making sure that there is always a valid credential for your stacks to use.

Spacelift’s Azure integration supports granular permissions by allowing you to setup more than one integration per tenant:

Spacelift’s Azure integration

Each integration will create a Service Principal in Azure with a unique display name:

Service Principal

You can then assign the relevant permissions to each role in Azure:

assigning the relevant permissions

And finally attach them to your stack as read or write:

read or write integration

Once you’ve attached both Azure integrations to your stack, Spacelift will automatically use the correct one depending on the phase of the run being executed. This allows you to provide read-only roles to planning runs (for example PRs), while providing write access for deployments.

To find out more about the Azure integration, take a look at our documentation available at <a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="noopener">Spacelift Documentation</a>. You might also be interested in the detailed overview of <a href="https://spacelift.io/blog/building-secure-cicd-integration-with-azure" target="_blank" rel="noopener">How we Built a Secure CI/CD Integration with Azure</a> and some of the issues we encountered and solved while designing and developing it.

We're excited to announce our new fully-managed integration with Azure. This integration makes it even easier for customers to get up and running fast without having to manually manage Azure Service Principal credentials.

spacelift_azure_integration

Spacelift_Azure_integration

Introducing the Spacelift Azure Integration

how specialized platform can help your iac

General | Spacelift Blog

General

The introduction of Infrastructure as Code or IaC has transformed the way you can provision and deploy high-performance cloud-based IT infrastructures.
IaC tools, such as Terraform, have been integrated into DevOps toolchains, saving DevOps IaC teams from the excessive manual effort.
While these tools undoubtedly help accelerate building IT infrastructures, their limitations can impact DevOps’ ability to optimize and improve control of their IaC processes supporting future business needs.
In this article, you’ll learn about IT infrastructure limitations that IaC DevOps teams deal with on a daily basis and how Spacelift is able to get past them.

One of the most frequent challenges while using more generic IaC tools is the non-intuitive workflow driven by its reliance on pull requests. Some solutions offer multiple workspaces, but the result can be fragile and nondeterministic. 
Since there’s no concept of mapping projects to branches or tags, anyone commenting on an approved pull request can deploy arbitrary code to production, even if the approval was meant for a short-lived experimental environment.
Spacelift does not depend on pull requests. It is mostly driven by push and tag events, so building a sophisticated Git flow is much easier. Spacelift reports the outcome of its jobs as commit status checks, allowing you to block merging the code on a failing Spacelift check.
Triggering a run can be customized using <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">Git push policies</a>. Thanks to that, Spacelift can provide the same level of comfort and security to teams using one project per repository and those using mono repo with hundreds of interdependent projects. You can read more about our approach to VCS integration <a href="https://docs.spacelift.io/integrations/github#tracked-branches" target="_blank" rel="noopener">here</a>.

The majority of generic tools don’t offer access control models but rely on comments on pull requests to drive infrastructure deployments. While it is usually fine when a single repository drives a single Terraform project, it becomes a huge liability for more complex scenarios.
Spacelift ships with a sophisticated mechanism allowing administrators to declare <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">who can log in</a> (and under what circumstances) and what their <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">level of access</a> to each of the managed projects should be. Even our <a href="https://docs.spacelift.io/integrations/slack" target="_blank" rel="noopener">Slack integration</a> can be subject to policy controls, allowing an admin to grant access to a project <a href="https://docs.spacelift.io/integrations/slack#managing-access-to-stacks-with-policies" target="_blank" rel="noopener">based on Slack-specific data</a> (think team, channel, user, etc.).

One thing that’s not in scope for most IaC solutions is the way to ensure that your infrastructure is compliant with industry best practices and your company policies.
Spacelift puts policy-as-code in the center of its value proposition and builds a consistent, <a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">robust policy framework</a> on top of <a href="https://www.openpolicyagent.org/" target="_blank" rel="noopener">Open Policy Agent</a>. Apart from providing a comprehensive automated change review and ensuring compliance of your <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">Terraform changes</a>, Spacelift uses the same approach to allow you to declare rules around the <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">account</a> and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">project access</a>, <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">handling push notifications</a>, <a href="https://docs.spacelift.io/concepts/policy/run-initialization-policy" target="_blank" rel="noopener">starting runs</a> and <a href="https://docs.spacelift.io/concepts/policy/task-run-policy" target="_blank" rel="noopener">triggering tasks</a>, and creating <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">relationships between projects</a>.

Handling interdependencies between projects has always been Terraform’s Achilles’ heel. The usual approach to this problem is adding another layer of abstraction in the form of a Terraform wrapper like <a href="https://terragrunt.gruntwork.io/" target="_blank" rel="noopener">Terragrunt</a>. But it’s only a partial solution as it breaks the problem into smaller chunks.
<a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">Spacelift’s trigger policies</a> on the other hand provide a smart, declarative automation layer on top of vanilla Terraform. They allow you to plug into state changes of individual projects and declare dependencies that should be resolved, following the changes that have just been applied. Read more <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">here</a> to discover other exciting possibilities.

Another problem to solve externally when using some of the generic tools is authoring and maintaining reusable <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> for your organization. Terraform is flexible in allowing modules to come from <a href="https://www.terraform.io/docs/modules/sources.html" target="_blank" rel="nofollow noopener">various sources</a>, but ensuring confidential access, as well as testing and versioning, are left to you, the user.
Until now, the golden standard in that regard has been the <a href="https://www.terraform.io/docs/cloud/registry/index.html" target="_blank" rel="nofollow noopener">private module registry from HashiCorp</a>. But Spacelift offers much more. Far from being just a glorified package manager, Spacelift adds a <a href="https://docs.spacelift.io/concepts/modules" target="_blank" rel="noopener">full CI solution for Terraform modules</a>, out of the box and free of charge. You can thus ensure that your private modules are healthy before you distribute them to the rest of your organization.

If you manage a single or a handful of rarely changing projects, it’s likely that you just set your IaC up once and forget about it. But in a more dynamic environment, where microservices come and go, new environments proliferate and new product teams require their own Terraform workspaces. The need to configure it each and every time become a major nuisance, putting a lot of pressure on your DevOps team.
Enter Spacelift. In Spacelift, much of the configuration can be handled by the project owners themselves—you can add Terraform and/or <a href="https://docs.spacelift.io/concepts/environment" target="_blank" rel="noopener">environment variables</a> and mount files (even inject Terraform code!) programmatically or through the GUI without the need for administrative privileges or changing the central server configuration. For administrators, adding new projects requires minimal hassle since there’s no need to set up webhooks or change any YAML. And it can all be done programmatically using Terraform.

What comes as a pleasant surprise to users of generic CI tools, Spacelift entities such as stacks, contexts, modules or policies can be managed in a declarative way using your favorite infra-as-code tool (this rule applies also to their configuration). Yes, that’s right—<a href="https://docs.spacelift.io/integrations/terraform-provider" target="_blank" rel="noopener">Spacelift offers a Terraform provider</a> that allows you to programmatically manage the lifecycle of its own resources.
Administrative stacks get credential-less access to the subset of our <a href="https://docs.spacelift.io/integrations/api" target="_blank" rel="noopener">GraphQL API</a> that does not involve managing the actual infrastructure. For more sophisticated use cases, Spacelift allows you to generate API keys that are subject to the same access controls as normal users are, allowing you to create single-purpose tokens for restricted use by your internal scripts.

Generic IaC platforms do not provide any mechanisms to detect if your infrastructure is undergoing drift. Drift is a condition that represents the difference between the desired and the actual state of the infrastructure managed by your tool of choice &#8211; <a href="https://www.terraform.io/" target="_blank" rel="nofollow noopener">Terraform</a>, <a href="https://www.pulumi.com/" target="_blank" rel="noopener">Pulumi</a>, <a href="https://aws.amazon.com/cloudformation/" target="_blank" rel="noopener">CloudFormation</a> or another. Drift can be caused by either or a combination of changes directly introduced by external actors &#8211; either humans or machines (scripts) or via the dependency of your resources on external data sources. In any case, drift is not good.
Spacelift has got you covered here. You can configure periodic <a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">drift detection </a>to notify you whenever drift happens, and take immediate action. You can even go a step further with optional automatic <a href="https://docs.spacelift.io/concepts/stack/drift-detection#to-reconcile-or-not-to-reconcile" target="_blank" rel="noopener">reconciliation</a>, ensuring your infrastructure always resembles your Terraform configuration.

General-purpose CI/CD platforms provide little to no insight into resource utilization from either a real-time or historical perspective. Which resources are over-or underutilized? 
Developers need to be able to intimately understand the material they’re working with. With regards to infra-as-code, the most important part of this story is understanding the managed resources in-depth. Both from the current perspective and through being able to put each resource in its historical context.
The resources view in Spacelift gives you greater visibility into each and every resource. With this deep insight into resources, DevOps are able to gain an understanding of the lifecycle of each resource managed by Spacelift and document it, regardless of the technology used — <a href="https://www.terraform.io/" target="_blank" rel="nofollow noopener">Terraform</a>, <a href="https://github.com/gruntwork-io/terragrunt" target="_blank" rel="nofollow noopener">Terragrunt</a>, <a href="https://www.pulumi.com/" target="_blank" rel="noopener">Pulumi</a>, or <a href="https://aws.amazon.com/cloudformation/" target="_blank" rel="noopener">CloudFormation</a>.

Last but not least, Spacelift puts an emphasis on great user experience, offering a myriad of creature comforts. <a href="https://docs.spacelift.io/concepts/context" target="_blank" rel="noopener">Contexts</a> for example allow you to attach entire collections of configuration to individual stacks and modules. <a href="https://docs.spacelift.io/concepts/task" target="_blank" rel="noopener">Tasks</a> provide a powerful audited way of running one-off administrative commands on an initialized Terraform environment &#8211; subject to <a href="https://docs.spacelift.io/concepts/task" target="_blank" rel="noopener">their own policy constraints</a>. <a href="https://docs.spacelift.io/concepts/stack#stack-locking" target="_blank" rel="noopener">Stack locking</a> allows a single individual to take exclusive control over a stack to ensure that nobody is able to modify its state while crucial changes are being made.

Spacelift is an innovative and sophisticated SaaS product for Infrastructure as Code which helps IaC DevOps develop and deploy new infrastructures or changes quickly and with confidence.
Spacelift offers a unique set of IaC management capabilities:
<ol>
<li style="font-weight: 400;" aria-level="1">an intuitive, <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">versatile</a> and <a href="https://docs.spacelift.io/concepts/run#run-state-machine" target="_blank" rel="noopener">robust</a> workflow</li>
<li style="font-weight: 400;" aria-level="1">granular access controls on <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">account</a> and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">project</a> level that work well with an <a href="https://docs.spacelift.io/integrations/single-sign-on" target="_blank" rel="noopener">identity provider of your choice</a> (SSO)</li>
<li style="font-weight: 400;" aria-level="1">an <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">automated code review</a> and <a href="https://docs.spacelift.io/concepts/policy/task-run-policy" target="_blank" rel="noopener">threat detection</a> </li>
<li style="font-weight: 400;" aria-level="1">the ability to <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">declare complex workflows</a> between projects across multiple repositories</li>
<li style="font-weight: 400;" aria-level="1">a <a href="https://docs.spacelift.io/concepts/modules" target="_blank" rel="noopener">built-in private module registry</a> with a full CI system for modules</li>
<li style="font-weight: 400;" aria-level="1">effortless setup and customization with <a href="https://docs.spacelift.io/concepts/environment" target="_blank" rel="noopener">per-project environment management</a> and <a href="https://docs.spacelift.io/integrations/docker" target="_blank" rel="noopener">Docker integration</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/terraform-provider" target="_blank" rel="noopener">programmatic configuration</a> using Terraform</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">drift detection </a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/concepts/resources" target="_blank" rel="noopener">resource visualization</a></li>
</ol>
… and a multitude of additional perks like <a href="https://docs.spacelift.io/concepts/context" target="_blank" rel="noopener">contexts</a>, <a href="https://docs.spacelift.io/concepts/task" target="_blank" rel="noopener">tasks</a> or <a href="https://docs.spacelift.io/concepts/stack#stack-locking" target="_blank" rel="noopener">stack locking</a>.

There are many ways to manage your Infrastructure as Code. Each way is different in terms of complexity and offers a different set of features. It is important to keep in mind that choosing one way or another should be based on business and technical requirements. Most often, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may exceed its potential benefits. It is much easier and more efficient to leverage platforms such as Spacelift to provide these features for you instead.

Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures

Terraform Functions, Expressions, Loops (Examples)

Terraform vs. Ansible : Key Differences

Most often, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may exceed its potential benefits. It is much easier and more efficient to leverage ready-to-go platform to provide these features instead.

How a Specialized Management Solution Can Improve Your IaC

In this post I want to show how easy it is to create a <a class="dt id" href="https://spacelift.io/" target="_blank" rel="noopener">Spacelift</a> Stack that manages Azure resources. To keep things simple, I’m going to assume that you already know what Spacelift can do, and the basics of creating and running Stacks, but if not check out our <a class="dt id" href="https://github.com/spacelift-io/terraform-starter" target="_blank" rel="nofollow noopener">terraform-starter</a> repo that walks you through the process of setting up an account, and getting started.

In order to allow Spacelift to manage Azure resources, we need to do the following:
<ul class="">
<li id="f0f8" class="hj hk fo hl b hm jf hn ho hp jg hq hr hs jh ht hu hv ji hw hx hy jj hz ia ic jk jl jm gk" data-selectable-paragraph="">Create a Service Principal in Azure with the correct ARM permissions to manage our resources.</li>
<li id="8f2c" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">Create a Stack in Spacelift that uses the <a class="dt id" href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs" target="_blank" rel="nofollow noopener">Azure</a> provider.</li>
<li id="ba4d" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">Give our Stack the credentials for our Service Principal.</li>
</ul>
All of the information you need to configure the Azure provider can be found <a class="dt id" href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#authenticating-to-azure" target="_blank" rel="nofollow noopener">here</a>. This post just explains how to combine that information with Spacelift.

To create our Service Principal, we need to use the <code class="js jt ju jv jw b">az ad sp create-for-rbac</code> command (replacing<code class="js jt ju jv jw b">&lt;subscription-id&gt;</code> with your actual subscription ID):

This will output something like the following:

The command creates a new Service Principal called <code class="js jt ju jv jw b">spacelift-sp</code>, and grants it the Contributor role on your subscription. It also outputs the <code class="js jt ju jv jw b">appId</code>, <code class="js jt ju jv jw b">password</code> and <code class="js jt ju jv jw b">tenant</code> for the Service Principal. Make a note of these because you&#8217;ll need them later.
If you would rather assign permissions yourself later, you can run the following command to just create a Service Principal with no permissions:

You should now be able to view your Service Principal in the Azure Active Directory -&gt; App registrations section of the Azure Portal:

default directory

Now that we’ve got a service principal, we need to create a Stack to actually create some Azure resources. To keep things simple all we’re going to do is add some terraform definitions that specify the version of the Azure Provider we want to use, and create an Azure Resource Group:

If you commit that change and try to run your Stack just now, you should see something similar to the following failure:

adding Azure resources

Here’s the full error message:
Error building AzureRM Client: obtain subscription() from Azure CLI: Error parsing json result from the Azure CLI: Error launching Azure CLI: exec: “az”: executable file not found in $PATH
At first glance this error is slightly confusing, but what it means is that because we didn’t provide any credentials to the Azure provider it attempted to fallback to using the <code class="js jt ju jv jw b">az</code> CLI for authentication. This works fine locally, but won&#8217;t work on Spacelift.
To solve this, we need to provide our stack with our credentials, which is explained in the next section.

If you take a look at the <a class="dt id" href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#authenticating-to-azure" target="_blank" rel="nofollow noopener">Terraform documentation</a>, you’ll find that there are a number of different ways that you can authenticate <a href="https://spacelift.io/blog/what-is-terraform">Terraform</a> with Azure. What we’re going to do here is authenticate using a Service Principal with a Client Secret, and provide that configuration via <a href="https://spacelift.io/blog/how-to-use-terraform-variables">environment variables</a>.
To do this we need to set the following environment variables, using the information returned by the Azure CLI when we created our Service Principal:
<ul class="">
<li id="1508" class="hj hk fo hl b hm jf hn ho hp jg hq hr hs jh ht hu hv ji hw hx hy jj hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_CLIENT_ID — the <code class="js jt ju jv jw b">appId</code> from the response.</li>
<li id="4e7d" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_CLIENT_SECRET — the <code class="js jt ju jv jw b">password</code> from the response.</li>
<li id="8a57" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_SUBSCRIPTION_ID — your subscription ID.</li>
<li id="7340" class="hj hk fo hl b hm jn hn ho hp jo hq hr hs jp ht hu hv jq hw hx hy jr hz ia ic jk jl jm gk" data-selectable-paragraph="">ARM_TENANT_ID — the <code class="js jt ju jv jw b">tenant</code> from the response.</li>
</ul>
You can set these environment variables directly on your Stack, but what we’re going to do here is create a <a class="dt id" href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="noopener">Spacelift Context</a> and attach it to our Stack. This allows us to share the same credentials with multiple Stacks if we want.
Add a new Context via the Contexts screen in Spacelift:

3-2-2

adding contexts

Once your context has been created, edit it and enter your environment variables, making sure to mark the <code class="js jt ju jv jw b">ARM_CLIENT_SECRET</code> as a secret rather than plaintext:

environment variables

Now that we’ve got a Context containing our credentials, you can attach it to your Stack. To do this, click on the Manage link next to your Stack, go to Contexts, choose your Context, and click Attach:

5-2-2

Azure stack

Now that your Context is attached, you should be able to see the it in the Environment section of your Stack:

Azure credentials

If you try to run your Azure Stack again you should get a successful plan:

running Azure Stack again

applying

If you take a look in the Azure Portal, you should now be able to see the Resource Group that was created:

Azure resource groups

Before finishing up, I wanted to just point out a few things about the Service Principal credential expiry. Although this isn’t really related to Spacelift, it’s something that you need to be aware of.
When you create your Service Principal via the Azure CLI, it automatically creates a Client Secret for you with an expiry of 1 year:

Spacelift Client Secret

What this means is that if you do absolutely nothing, your Spacelift Stacks will stop working after the credentials expire a year later. To solve this, all you need to do is generate a new Client Secret, and update the <code class="js jt ju jv jw b">ARM_CLIENT_SECRET</code> environment variable in your Context.

Hopefully in this post I’ve shown that although Spacelift doesn’t have an official Azure integration (yet!), it’s simple to manage Azure resources using Spacelift. Once you’ve performed a little bit of initial setup, you should be able to rapidly get new Stacks up and running creating Azure resources.

Importing Existing Infrastructure into Terraform - step by step

In order to allow Spacelift to manage Azure resources, we need to create a Service Principal in Azure with the correct ARM permissions to manage our resources., create a Stack in Spacelift that uses the Azure provider and give our Stack the credentials for our Service Principal.

managing-azure-resources-using-spacelift-2

Atlantis is an old and battle-tested self-hosted tool that provides a layer of automation and collaboration for Terraform. Lacking a user interface other than server <a href="https://www.runatlantis.io/docs/server-configuration.html#flags" target="_blank" rel="nofollow noopener">flags</a> and <a href="https://www.runatlantis.io/docs/server-side-repo-config.html#do-i-need-a-server-side-repo-config-file" target="_blank" rel="nofollow noopener">YAML configuration files</a>, Atlantis uses your VCS as one, listening for pull request notifications and posting feedback as comments.
Using Atlantis has two major benefits over running Terraform in a “generic” CI system. First, it strikes the right balance between specialized and flexible: <a href="https://www.runatlantis.io/docs/custom-workflows.html" target="_blank" rel="nofollow noopener">custom workflows</a> give you a level of control that’s not possible within a tool like <a href="https://www.terraform.io/docs/cloud/index.html" target="_blank" rel="nofollow noopener">Terraform Cloud</a>. Second, it runs on your infrastructure, which allows it to access resources inside a VPC or use cloud-specific dynamic credentials.
Spacelift embraces all those characteristics of Atlantis but offers them in as a modern and sophisticated SaaS product that provides:
<ul>
<li style="font-weight: 400;" aria-level="1">a more intuitive, <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">versatile</a> and <a href="https://docs.spacelift.io/concepts/run#run-state-machine" target="_blank" rel="noopener">robust</a> workflow;</li>
<li style="font-weight: 400;" aria-level="1">extremely granular access controls on <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">account</a> and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">project</a> level that work well with an <a href="https://docs.spacelift.io/integrations/single-sign-on" target="_blank" rel="noopener">identity provider of your choice</a> (SSO);</li>
<li style="font-weight: 400;" aria-level="1">an <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">automated code review</a> and <a href="https://docs.spacelift.io/concepts/policy/task-run-policy" target="_blank" rel="noopener">threat detection</a> using a consistent open-source <a href="https://www.openpolicyagent.org/" target="_blank" rel="noopener">policy-as-code framework</a>;</li>
<li style="font-weight: 400;" aria-level="1">the ability to <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">declare complex workflows</a> between projects across multiple repositories;</li>
<li style="font-weight: 400;" aria-level="1">a <a href="https://docs.spacelift.io/concepts/modules" target="_blank" rel="noopener">built-in private module registry</a> with a full CI system for modules;</li>
<li style="font-weight: 400;" aria-level="1">effortless setup and customization with <a href="https://docs.spacelift.io/concepts/environment" target="_blank" rel="noopener">per-project environment management</a> and <a href="https://docs.spacelift.io/integrations/docker" target="_blank" rel="noopener">Docker integration</a>;</li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/terraform-provider" target="_blank" rel="noopener">programmatic configuration</a> using Terraform;</li>
<li aria-level="1"><a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">drift detection;</a></li>
<li style="font-weight: 400;" aria-level="1">… and a myriad of creature comforts like <a href="https://docs.spacelift.io/concepts/context" target="_blank" rel="noopener">contexts</a>, <a href="https://docs.spacelift.io/concepts/task" target="_blank" rel="noopener">tasks</a> or <a href="https://docs.spacelift.io/concepts/stack#stack-locking" target="_blank" rel="noopener">stack locking</a>;</li>
</ul>
Let’s talk through each of these features one by one.

One of most frequent challenges while using Atlantis in practice is a non-intuitive workflow driven by its reliance on pull requests. Not only is it a lot of work to set up multiple <a href="https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#supporting-terraform-workspaces" target="_blank" rel="nofollow noopener">workspaces</a> pointing to the same Git repository but the result is fragile and nondeterministic. Since there’s no concept of mapping projects to branches or tags, anyone that can comment on an approved pull request can deploy arbitrary code to production, even if the approval was meant for a short-lived experimental environment.
Spacelift does not depend on pull requests. Sure, it will post beautiful comments there for your convenience, but it’s mostly driven by push and tag events, so it’s easy to build a sophisticated Git flow. Spacelift will report the outcome of its jobs as commit status checks, which then allows you to block merging the code on a failing Spacelift check. There are <a href="https://docs.spacelift.io/concepts/run#where-do-runs-come-from" target="_blank" rel="noopener">two main types of runs</a> &#8211; proposed and tracked. By default, each project points at a particular branch (and optionally also a path) &#8211; pushing to that branch will trigger a tracked run &#8211; one that can be applied. This behavior can be customized using <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">Git push policies</a>. Thanks to that, Spacelift can provide the same level of comfort and security to teams using one project per repository and those using monorepos with hundreds of interdependent projects. You can read more about our approach to VCS integration <a href="https://docs.spacelift.io/integrations/github#tracked-branches" target="_blank" rel="noopener">here</a>.

Another pain point when using Atlantis at scale is its access control model. By relying on comments on pull requests to drive infrastructure deployments, Atlantis effectively yields control over your infrastructure to the VCS provider. While this is usually fine when a single repository drives a single Terraform project, this becomes a huge liability for more complex scenarios.
Spacelift ships with a sophisticated mechanism allowing administrators to declare <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">who can log in</a> (and under what circumstances) and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">what should be their level of access</a> to each of the managed projects. Since it’s 2020, Spacelift ships with SSO out of the box, supporting every identity provider that speaks SAML 2.0 protocol. Even our <a href="https://docs.spacelift.io/integrations/slack" target="_blank" rel="noopener">Slack integration</a> can be subject to policy controls, allowing an admin to grant access to a project <a href="https://docs.spacelift.io/integrations/slack#managing-access-to-stacks-with-policies" target="_blank" rel="noopener">based on Slack-specific data</a> (think team, channel, user etc.). For extra protection, powerful features like Tasks have their own independent policy controls.

Both Spacelift and Atlantis let you use <a href="https://www.openpolicyagent.org/" target="_blank" rel="noopener">Open Policy Agent</a> &#8211; an open-source solution endorsed by <a href="https://www.cncf.io/" target="_blank" rel="noopener">Cloud Native Computing Foundation</a> &#8211; to ensure that your infrastructure is compliant with industry best practices and your company policies. Atlantis integrates with <a href="https://www.runatlantis.io/docs/policy-checking.html#how-it-works" target="_blank" rel="nofollow noopener">conftest</a> to achieve that and lets you verify each Terraform plan using OPA policies.
Spacelift goes a few steps further though and puts policy-as-code in the center of its value proposition and builds a consistent, <a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">robust policy framework</a>. Apart from providing a comprehensive automated change review and ensuring compliance of your <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">Terraform changes</a>, Spacelift uses the same approach to allow you to declare rules around <a href="https://docs.spacelift.io/concepts/policy/login-policy" target="_blank" rel="noopener">account</a> and <a href="https://docs.spacelift.io/concepts/policy/stack-access-policy" target="_blank" rel="noopener">project access</a>, <a href="https://docs.spacelift.io/concepts/policy/git-push-policy" target="_blank" rel="noopener">handling push notifications</a>, <a href="https://docs.spacelift.io/concepts/policy/run-initialization-policy" target="_blank" rel="noopener">starting runs</a> and <a href="https://docs.spacelift.io/concepts/policy/task-run-policy" target="_blank" rel="noopener">triggering tasks</a>, and creating <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">relationships between projects</a>. It also provides the Policy Workbench, which lets you view past executions of your policy &#8211; including the inputs and decisions that have been made &#8211; and lets you interactively edit your policy, while simulating its execution on these previous inputs. You can use our <a href="https://github.com/spacelift-io/terraform-starter" target="_blank" rel="nofollow noopener">starter repo</a> to quickly provision a bunch of policies and get a feel for how you can work with them in Spacelift.

Handling interdependencies between projects has always been Terraform’s Achilles&#8217; heel and Atlantis is no different in that regard. The usual approach to this problem is adding another layer of abstraction in the form of a Terraform wrapper like <a href="https://terragrunt.gruntwork.io/" target="_blank" rel="noopener">Terragrunt</a>. While this breaks the problem into smaller chunks, it’s only a partial solution achieved at a cost of locking yourself into another dependency.
<a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">Spacelift’s trigger policies</a> on the other hand provide a smart, declarative automation layer on top of vanilla Terraform. These allow you to plug into state changes of individual projects and declare dependencies that should be resolved following the changes that have just been applied. In fact, this is just a subset of all use cases covered by trigger policies. <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">Read more</a> to discover other exciting possibilities.

Another problem that one needs to solve externally when using Atlantis is authoring and maintaining reusable Terraform modules for your organization. Terraform is flexible in allowing modules to come from <a href="https://www.terraform.io/docs/modules/sources.html" target="_blank" rel="nofollow noopener">various sources</a> but ensuring confidential access as well as testing and versioning are left to you, the user.
Until now, the golden standard in that regard has been the <a href="https://www.terraform.io/docs/cloud/registry/index.html" target="_blank" rel="nofollow noopener">private module registry from HashiCorp</a>. But Spacelift offers that, and much more. Far from being just a glorified package manager, Spacelift adds a <a href="https://docs.spacelift.io/concepts/modules" target="_blank" rel="noopener">full CI solution for Terraform modules</a>, out of the box and free of charge. You can thus ensure that your private modules are healthy before you distribute them to the rest of your organization.

If you manage a single or a handful of rarely changing projects using Atlantis, it’s likely that you just set it up once and forget about it. But in a more dynamic environment, where microservices come and go, new environments proliferate and new product teams require their own Terraform workspaces, the need to configure Atlantis each and every time becomes a major nuisance putting a lot of pressure on your DevOps team.
Enter Spacelift. In Spacelift, much of the configuration can be handled by the project owners themselves &#8211; you can add Terraform and/or <a href="https://docs.spacelift.io/concepts/environment" target="_blank" rel="noopener">environment variables</a> and mount files (even inject Terraform code!) programmatically or through the GUI without the need for administrative privileges or changing the central server configuration. For administrators, adding new projects requires minimal hassle since there’s no need to set up webhooks or change any YAML. And it can all be done programmatically, using Terraform.

What comes as a pleasant surprise to long-time Atlantis users, Spacelift entities like stacks, contexts, modules or policies as well as their configuration can be managed in a declarative way using your favorite infra-as-code tool. Yes, that’s right &#8211; <a href="https://docs.spacelift.io/integrations/terraform-provider" target="_blank" rel="noopener">Spacelift offers a Terraform provider</a> that allows you to manage the lifecycle of its own resources programmatically.
 Administrative stacks get credential-less access to the subset of our <a href="https://docs.spacelift.io/integrations/api" target="_blank" rel="noopener">GraphQL API</a> that does not involve managing the actual infrastructure. For more sophisticated use cases, Spacelift allows you to generate API keys that are subject to the same access controls as normal users are, allowing you to create single-purpose tokens for restricted use by your internal scripts.

<a href="https://www.hashicorp.com/blog/detecting-and-managing-drift-with-terraform" target="_blank" rel="nofollow noopener">Infrastructure drift happens.</a> Either somebody will make manual changes while debugging and then forget about them, or you might have data sources whose outputs change periodically. Normally, you would notice these changes only after you execute your next plan or apply.
Spacelift has got you covered here. You can configure periodic <a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">drift detection</a> to be notified whenever drift happens and take immediate action. You can go a step further too with optional automatic <a href="https://docs.spacelift.io/concepts/stack/drift-detection#to-reconcile-or-not-to-reconcile" target="_blank" rel="noopener">reconciliation</a>, ensuring your infrastructure always resembles your Terraform configuration.

Last but not least, Spacelift puts an emphasis on great user experience, offering a myriad of creature comforts. <a href="https://docs.spacelift.io/concepts/context" target="_blank" rel="noopener">Contexts</a> for example allow you to attach entire collections of configuration to individual stacks and modules. <a href="https://docs.spacelift.io/concepts/task" target="_blank" rel="noopener">Tasks</a> provide a powerful audited way of running one-off administrative commands on an initialized Terraform environment &#8211; subject to <a href="https://docs.spacelift.io/concepts/task" target="_blank" rel="noopener">their own policy constraints</a>. <a href="https://docs.spacelift.io/concepts/stack#stack-locking" target="_blank" rel="noopener">Stack locking</a> allows a single individual to take exclusive control over a stack to ensure that nobody is able to modify its state while crucial changes are being made. The <a href="https://docs.spacelift.io/concepts/resources" target="_blank" rel="noopener">resources visualization</a> shows you all the resources you’re managing in your account, when and how they’ve last been updated, and lets you slice and dice them using a plethora of views, filters and groupings.

Spacelift helps manage Terraform state, build more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and many more.

Open Policy Agent: What Is OPA and How It Works (Examples)

The top alternatives to Atlantis include Spacelift, Heroku, Red Hat OpenShift, Hasura. Spacelift embraces all the characteristics of Atlantis but offers them in as a modern and sophisticated SaaS product

alternative-to-atlantis-2

Alternative to Atlantis

Atlantis vs. Spacelift: Key Differences and Similarities

Want to level up your Terraform knowledge? Explore our tutorials and step-by-step guides written by Terraform experts. Learn Terraform best practices and solutions to the most difficult problems.

Terraform Guides and Tutorials | Spacelift Blog

Terraform

Learning Terraform can be easy. The HCL configuration language is simple enough to start with basic resource creation straight away. 
But—
Once the code that defines your infrastructure grows in complexity, maintainability, and dependency, an uphill battle can ensue. And that is why working out an efficient deployment workflow is crucial for further development.
In this article you will see five approaches to managing Terraform workflows at scale, what their benefits and downsides are, and what problems they address.

A quick and easy way to start deploying resources using the Infrastructure as Code pattern is to use Terraform locally from the same machine you’re developing the code on. The only requirement is to have access to the target provider and an installed Terraform binary.
This approach makes it much easier to leverage commands related to day-to-day operations (such as state import, move, remove) or to get outputs than if doing it within the implementations where you have no direct access to the underlying provider.
You can also use all of the features of Terraform, such as remote state and state locking, to work with your teammates all-at-once. If you collaborate using a version control system and do not have continuous integration in place, but want to keep your code at a decent level of quality, there are tools to help you.
One of these is <a href="https://github.com/antonbabenko/pre-commit-terraform" target="_blank" rel="nofollow noopener">pre-commit-terraform</a> that will run a set of selected tools to lint the code you’re creating before you even commit it. Even if you have continuous integration available, it’s a handy way to avoid wasting time waiting for results from pull request status checks.
What are the drawbacks of this approach? Each action is manual and must be performed directly by the developer, which leads to several issues. 
In a situation where multiple people are working on the same codebase, you could often encounter a scenario such as this one:
Person A applies their changes to the environment and follows up with a pull request. At the same time, Person B would like to deploy their changes but is blocked as the codebase is not up-to-date with Person A’s changes. Applying the changes in this state would cause damage to the resources created in the previous deployment. When working in larger teams, this will often result in a lot of wasted time just waiting and rebasing on a codebase to apply the changes.
If you happen to write unit tests for <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> (and you should be), you’re sure to know they need to be executed manually from your computer. Running them every time you change something in your configuration can be tedious and time-consuming. If you do not yet write tests for your modules, you’d be well advised to start looking at <a href="https://docs.spacelift.io/vendors/terraform/module-registry#tests" target="_blank" rel="noopener">Spacelift modules tests</a> or <a href="https://terratest.gruntwork.io/docs/getting-started/quick-start/" target="_blank" rel="nofollow noopener">Terratest</a>.
Privileged access to an underlying vendor is required, which can easily result in a compromised environment. It’s much harder to keep restricted access within this model, as Terraform must be allowed to create, change, and destroy resources. The same rule applies to a scenario where access to private (internal) resources is required, e.g., every developer has to set up a VPN connection to interact with them.
Moreover, every person working with the codebase needs access to the Terraform state, which creates a security concern due to how Terraform state works. Even if you’re pulling sensitive data from an external system such as Vault, it will be stored within the state file in plaintext once used in a resource. A threat actor could simply run `terraform state pull` to access all the secrets!
Keeping all these issues in mind, let’s take a look at a more automated approach.

Implementing homegrown automation means incorporating Terraform workflow into your continuous integration / continuous deployment process. A specific example would need to directly tie to a CI/CD tool you are using, so let’s try to keep the example as generic as possible.
As the applying actor is the CI/CD system, the hard requirement of privileged access for developers is gone. You can grant this type of access to the execution layer while developers keep read-only permissions. This is true for private (internal) resources, and if your code is being executed on an agent that resides within the target infrastructure with the proper permissions having been set. In a model where everything has to go through the <a href="https://spacelift.io/blog/terraform-in-ci-cd">CI/CD pipeline</a>, visibility of changes is improved as the system logs produced during the operation execution are available in real-time. 
Linting, compliance checks, and automated unit tests can be moved to pull request status checks, and it’s up to you to configure these steps of the pipeline.
But—
If your CI/CD is executing jobs concurrently, you will end up with race conditions causing pull requests to fail non-deterministically. 
It is not difficult to imagine a situation where two operators open a pull request within several seconds of one another. The pipeline gets triggered immediately for both, and one pull request will be marked as successful, while the other will be marked as failed. This situation will definitely happen if you are using state locking (it is a best practice and you really should be using it).
The explanation is simple: the first pull request has locked the state and therefore the second one could not, so it failed. This issue can be resolved by configuring queued runs, where only one pipeline can run at a time. However, this is not so simple with some CI/CD products.
The biggest issue and concern with this approach is the effort you have to put in to build a complete pipeline. If you consider requirements such as:
<ul>
<li style="font-weight: 400;" aria-level="1">planning on pull requests</li>
<li style="font-weight: 400;" aria-level="1">linting</li>
<li style="font-weight: 400;" aria-level="1">unit tests</li>
<li style="font-weight: 400;" aria-level="1">compliance checks</li>
<li style="font-weight: 400;" aria-level="1">applying once merged</li>
<li style="font-weight: 400;" aria-level="1">periodical drift detection</li>
<li style="font-weight: 400;" aria-level="1">registry for all your versioned modules</li>
<li style="font-weight: 400;" aria-level="1">modules sharing across different parts of the organization</li>
<li style="font-weight: 400;" aria-level="1">shared variable contexts/reusable values</li>
</ul>
It might be tempting to implement solutions for those requirements by yourself. However, taking into account the time and effort required for doing it, you might want to take a look at alternatives, such as open-source tools.

One of the most popular open-source tools to use with Terraform is Atlantis, which describes itself as “Terraform pull request automation”. You can read more about it here: <a href="https://spacelift.io/blog/post/alternative-to-atlantis" target="_blank" rel="noopener">Alternative to Atlantis</a>.

Terraform Cloud is an application provided by the company behind Terraform itself—HashiCorp. It is available for free for up to 5 users. Two plans are available for more users: “Team &amp; Governance” and “Business”. As the product is created by Terraform authors themselves, any new feature that makes it to Terraform will be quickly available within Terraform Cloud as well. Neat!
<h3>Advantages of Terraform Cloud</h3>
Advanced security compliance mechanisms available through Sentinel allow you to enforce organization standards even before the code actually ships and creates resources with the chosen provider context.
Another important feature is a module registry that acts as an artifact repository for all your modules. It can be compared to JFrog Artifactory, Sonatype Nexus, or other similar software. Having every module in a separate repository and versioning allows you to control code promotion in a granular fashion or quickly roll back to if it allows you to go back to the most recent version only, then we should use “the”, but if it lets you go back to some other earlier version then we should use “a” previous version. 
Having a centralized place to store all the pieces also enables you to quickly browse through available modules, their documentation, inputs, outputs, and used resources. It is a great single pane of glass in terms of module management. However, it can only be used within the same Terraform Cloud organization and cannot be shared. This might be troublesome for operations teams that base their separation on the organization level.
There’s also the remote execution backend, which is an exclusive feature for Terraform Cloud. It allows you to work with Terraform on your local computer as you would usually do, upload your local codebase, and run a plan against it. It is a very efficient way of quickly verifying if what you are working on is actually working.
<h3>Disadvantages of Terraform Cloud</h3>
Along with so many benefits, there are some drawbacks as well. Any linting or module unit tests need to be implemented inside your own continuous integration system. This requires an additional effort the development team has to make to sustain their code quality.
Another disadvantage is that you are not able to share common values across your workspaces. Imagine a situation where you share a common set of variables like cloud region or environment name. The only way to achieve something similar to shared context is to have a dedicated workspace that will output this information as an input for all remaining workspaces, although this would be more of a workaround than an actual solution.
Compliance as code is available via Sentinel, a framework created by HashiCorp. Having a built-in solution to develop policies is nice, but operators often already use <a href="https://spacelift.io/blog/what-is-open-policy-agent-and-how-it-works">Open Policy Agent (OPA)</a> to define these within different areas of the infrastructure. This frequently ends up in a doubled effort as you need to create a similar policy in both frameworks.
The biggest shortcoming of Terraform Cloud is in my opinion the lack of periodical drift detection runs. Once something is applied and the codebase does not get updates, you will never be notified if anything has changed in comparison to the state within the repository. Scheduling a daily “plan” to verify if everything is up-to-date would be highly beneficial to keep the infrastructure as described programmatically.

And here we are. You can compare Spacelift directly to Terraform Cloud. They share a lot of functionality with each other in terms of the actual deployment execution, but also in the area of compliance.
While Terraform Cloud uses Sentinel (the HashiCorp approach to implementation for compliance as code), Spacelift leverages Open Policy Agent. It is beneficial to have a similar way of working with policies for companies that already incorporate OPA in their workflow (e.g., for Kubernetes). As there is no need to learn another syntax, operators can focus their efforts on providing compliance, instead of learning yet another tool.
Spacelift also allows you to automate your workflow even further. By having a built-in continuous integration for modules, you can shift your linters or unit tests really quickly into one place dedicated to Terraform. It helps you reduce the effort required to implement integration workflow by letting you utilize a solution that you are already using.
Sometimes you need to work with a Terraform state directly to import something, for example. In scenarios where you are leveraging Terraform Cloud, it is often impossible to do this without accessing the provider yourself. In contrast, Spacelift gives you the ability to run literally any command within the context of a particular codebase via tasks. This way, you do not need to grant any additional permissions to import, move, or remove resources directly.
By utilizing contexts, you can simply share anything you like across multiple stacks—whether it is a set of environment variables or a file. This is something that becomes much more valuable once your codebase complexity increases.
As mentioned in the previous paragraph, drift detection is something that becomes much more important once your infrastructure grows. With Spacelift, it is possible to schedule a periodic drift detection on any stack. Going even further, you can enable automated remediation if any changes are found compared to what you have in your codebase.

There are many ways of working with Terraform. Each way is different in terms of complexity, and has a different set of features. It is important to keep in mind that choosing one way or another should be based on business and technical requirements. Most times, there is no point in implementing an in-house solution as the cost and effort of building and maintaining it may often exceed its potential benefits. It is much easier and quicker to leverage platforms such as Spacelift to provide these features for you instead.

How to Provision an AWS EKS Kubernetes Cluster with Terraform

How to Get Started with Terraform on Google Cloud Platform (GCP)

This article goes through five approaches to managing Terraform workflows at scale: using Terraform locally, automating it in-house, using open source solution, using Terraform Cloud, and using Spacelift.

Infrastructure Problems that Spacelift Solves

Recent posts

Running Spacelift CI/CD workers in Kubernetes using DinD

Introducing the Spacelift Azure Integration!

How a Specialized Management Platform Can Improve Your IaC

Managing Azure Resources using Spacelift

Alternative to Atlantis – Spacelift