Infrastructure Problems that Spacelift Solves

Automating infrastructure involves far more than developing code in your favorite IaC tools such as Terraform, OpenTofu, Pulumi, and CloudFormation. It encompasses every aspect of your workflow and covers everything from making a pull request for your changes to reaching the production environments.

As soon as engineers adopt an IaC tool, they find they need some sort of platform to handle their infrastructure orchestration because their organization’s processes(compliance, security, governance, etc.) cannot be implemented without one. They usually look at generic CI/CD pipelines first because they have already used them for their application deployment processes.

Generic CI/CDs are powerful because they can accommodate many use cases, but they were not built with infrastructure in mind, so they lack certain features that could greatly improve your infrastructure orchestration capabilities. CI/CD pipelines for infrastructure take time to build, are hard to maintain, and offer visibility restricted to certain runs. This makes it harder to tie workflows across tools and processes, so vulnerabilities are hard to control.

Most IaC tools are stateful, which means you have to manage your state files, version them, and ensure you have backups in case something goes wrong. This introduces overhead and uses time your engineers could be using to increase their productivity.

Spacelift comes with a default workflow for your favorite infrastructure tools such as Terraform, OpenTofu, Pulumi, CloudFormation, Terragrunt, Kubernetes, and Ansible. It uses stacks to handle workflows that combine your source code, your tool, and the environment you want to deploy to. 

This workflow can be used as it is, but you can also easily customize it using a stack or a context (logical container for your environment variables, mounted files, and lifecycle hooks) that can be attached to multiple stacks.

Spacelift can also manage the state files for you and implement all the mechanisms you need to keep your state files safe, but you can also do that yourself if you want to:

Spacelift handles all the intricacies of building a CI/CD pipeline for your infrastructure out of the box. If you miss anything, you can easily integrate with them.

Enforcing security, governance, and compliance can be very hard, especially if you don’t have something native in your platform to do it. Spacelift uses Open Policy Agent (OPA) for policy as code, allowing you to easily:

• Restrict certain infrastructure resources or certain parameters of resources with Plan policies. This can be useful to ensure that compliance is respected and costs are kept under control.
• Define who can approve or reject a run/task and how a run/task can be approved with Approval policies. This will ensure that changes are not made without proper accountability.
• Define where to send notifications.  You can send notifications about your runs, your drift detection, and even data about your Spacelift account to a monitoring tool.
• Control what happens when a PR is open or merged. This is very useful if you want to ignore changes to a certain path or only apply a subset of changes.

OPA uses Rego as its policy language, which can have a very steep learning curve. Luckily, with Spacelift you don’t need to know Rego to create policies, as Spacelift offers its own policy library from which you can choose the policies you need for workflows.

Spacelift offers many other security features, such as SSO, MFA, private runners, user management, and more. See what makes Spacelift secure.

Inanimate objects and things have an irritating tendency not to stay as you left them. If you have ever lost your keys, you will surely relate. 

Infrastructure is no different. Unexpected situations, manual tampering, or force majeure could lead to differences between the configuration you have applied and the system’s current state.

Sometimes the reasons are understandable. For example, when an emergency forces your engineer to introduce crucial infrastructure changes manually to avoid downtime or other damages; sometimes it’s not just the result of a script or person doing what they shouldn’t. 

Spacelift introduces automatic drift detection runs. Spacelift can periodically check your infrastructure (you decide how often or trust the defaults), detect drifts, and provide you with information about the resources they impact:

• What was impacted? 
• When did this happen?

Then, if you decide that they are beneficial or required, you can properly introduce them into your state in a process called reconciliation.

If you want to learn more about the top causes of drift and their hidden impacts, read the highlighted articles.

Everyone wants to move fast, especially now that everything is served in an instant. You want to deliver your infrastructure fast, but it can be very fragile, and the smallest misconfiguration can result in hours of downtime and revenue loss.

A faster time to market doesn’t necessarily mean the experience you offer will be better than your competition’s. You need to move fast, but you still need to have control mechanisms in place to ensure everything goes smoothly. However, excessive control means you are moving too slowly, and your competition will get the upper hand.

Spacelift helps you balance speed and control by enabling you to self-serve infrastructure via our Blueprints feature.

With Blueprints, you can create pre-approved infrastructure templates that can be used by anyone on the team, and you can also ensure all the security, governance, and compliance you require. Inside the template, you can define which policies to use, enable drift detection, integrate security vulnerability scanning tools, and more. This will give platform teams the power to stay in control while enabling developers to use infrastructure when they need it without going through a cumbersome process.

Infrastructure as Code significantly improves collaboration. It is much easier to work with someone on a piece of code than on a server spinning up, but there are still a few peculiarities that you might find out about along the way. 

Take this example. Let’s assume you’ve got some configuration elements, like environment variables, that you wish to use as shared—between various groups of resources. Normally, you’d need to individually provide those values to each resource set that needs them by defining them multiple times, in different places.

To significantly ease this process, Spacelift introduces the concept of contexts—grouped declarations of variable values and useful files that you can mount to your stacks. You declare them once and use them where you need, with no security disadvantages or time wasted on useless repetition.

Another situation that you might run into, would be when you’d want to use your Terraform modules in configurations outside your organization. For example, you have five customers. Each one has a separate account, and ideally, you’d want to retain control over the configuration modules you made, to make sure no one breaks anything. But—you also want to use the configuration you already have, for repetitive tasks between the infrastructures of those customers. Spacelift makes this possible:

Just provide the names of accounts you wish to share your code with, and let those accounts build their infrastructure with your curated resources. Easy!

Still unconvinced? There’s much more. Seamless integration with popular VCS providers such as GitHub or BitBucket, push status notifications… and of course, preview runs, showing you what will change as a result of the pull request.

As you can see, there’s something here for everyone. Spacelift is developed with collaboration in mind. It blends in very well but provides features that stand out. Even if you don’t really like using external tools or technologies, you won’t be disappointed with the results.

Scaling infrastructure introduces complexity, makes your workflow hard to manage, and introduces unwanted vulnerabilities.

This happens because:

• Multiple teams are working on different projects.
• There are various environments to maintain.
• There is a growing number of resources to manage.

At scale, this lack of visibility makes detecting and resolving issues harder, organizational risks increase, and security vulnerabilities become hard to contain.

Spacelift provides the tools you need to scale your IaC operations effectively:

• Automated workflows with dependencies and output sharing
• A comprehensive policy-as-code engine
• Unparalleled visibility inside your IaC and even your configuration management resources
• Comprehensive access control mechanisms
• Real-time collaboration features

Having a single view in which you can see all the resources deployed with your account, their configurations, their health, the hosts, and how they have been configured gives you unparalleled visibility so you can make decisions accordingly.

Spacelift offers two views for this, one for the IaC resources, and the other one for configuration management.

The snippet above shows the resources view and how you can get details about them.

Now, let’s look at the configuration management view:

You can see all the tasks and roles that ran on your hosts and their statuses, and if you click on one of the tasks, you will get details about it:

This is extremely useful as you can understand at a glance everything that happened to your infrastructure resources.

Traditional approaches make linking infrastructure provisioning with configuration management and container orchestration difficult. Spacelift’s stack dependencies feature simplifies this. 

Stack dependencies let you create dependencies on as many nested levels as you want between your configurations. You can even use different tools when you are building these dependencies.

Another key power of this feature is the ability to easily share outputs between them. For example, you can have a Terraform stack that creates EC2 instances and an EKS cluster, create a dependency to an Ansible stack that builds the inventory from the EC2 instances created by Terraform, and another dependency to a Kubernetes stack that receives the cluster configuration to easily deploy your applications:

Whenever there is a change to the parent stack, runs will be queued on the children. If the parent finishes its run successfully, the children will start their runs. The process repeats on as many nested levels as you want until it reaches the lower-level child. This is very powerful because you can easily split your configurations into smaller chunks, identify errors faster, and build multi-tools workflows easily.

Everyone, at least once in their lifetime, bought something and then watched in disbelief as it diminished before their eyes. The pain of a huge Amazon Web Services bill is definitely real. 

AWS bills are often mentioned in the countdowns of some of the most massive things in existence—somewhere between cargo ships, black holes, and the NodeJS node_modules directories. Some engineers even say that you don’t really need coffee in your life if you’re provisioning AWS infrastructure—the bill will raise your blood pressure just the same. 

Spacelift integrates with Infracost, to provide you with cost estimates of your runs. This way your pull request can tell you how much it will cost to apply the configuration you have designed. It also allows you to set up proper cost barriers to make sure that you won’t spend more than expected just by misconfiguration or burn the entire monthly budget in a week by one unfortunate deployment. 

Read more about it in our Terraform Cost Estimation with Infracost article.

The setup is fairly simple, but to make it even easier you can create a context mentioned in an earlier section and attach it to the stacks you wish to monitor in terms of budget.

Infrastructure as code offers many benefits, but without proper orchestration, it can also introduce many issues. Using an infrastructure orchestration platform such as Spacelift adds tremendous value because it can help you solve many of the problems that IaC tools introduce.

If you want to learn more about Spacelift, book a demo with one of our engineers.