Self-Service Infrastructure with Spacelift

In the previous article, we discussed the benefits of self-service infrastructure and the Spacelift functionality that enables it. We’ll now move on to explore examples of self-service infrastructure, the problems it solved and the improvements Spacelift can add.

Unlike available CI/CD tools, Spacelift works as a multifunctional platform, delivering CI/CD and orchestration functionality and also operating as a self-service infrastructure platform.

How does Spacelift do all that? Here is an overview of the features that enable Spacelift’s self-service approach:

• Blueprints. These are the backbone of Spacelift’s self-service infrastructure approach. Templating is a core element of self-service, allowing teams to select the infrastructure template that fits their needs and deploy it securely. Templates are tested and managed by experts.
• Spaces. The development team doesn’t need to know exactly how the structure of cloud accounts is defined. Spaces make the organization of the accounts and environments easier.
• Policies. Policies are powerful tools for controlling blueprints usage. Using Open Policy Agent (OPA) policies to manage who can use specific templates when enhances infrastructure control.
• Dependencies. Development teams can easily create a chain of infrastructure deployment using dependencies between stacks. This makes the whole process easier to understand and doesn’t require deep knowledge of IaC.

I will now walk you through some real cases I’ve observed or worked on:

Case 1: CloudFormation for setting up environments

This project centers on the team who managed the infrastructure templates for development teams. They used AWS CloudFormation to create separated environments in multiple AWS accounts. Infrastructure created in these environments contains networks, policies, container clusters, serverless, and more.

The problem

The first problem in this case concerned knowledge gaps around AWS architecture and best practices within development teams. The second was more complex. The organization had a large number of development teams, with each team owning at least 3 AWS accounts, sometimes more. This proliferation of teams and ownership meant there was little control over the infrastructure and no single pane of glass where information about systems could be gathered.

How it was addressed

The solution at that time was to create a set of CloudFormation snippets that could be collected by the development team and combined into one big template according to their needs.

It is clear that this approach did not deliver a satisfactory solution, failing to resolve technical issues or improve the process.

How Spacelift could solve the problem

The first step is to devise an appropriate process for defining the requirements and designing the templates. With Spacelift, you don’t need to use unwieldy CloudFormation snippets: you can prepare and parameterize full templates. For optimal flexibility, the templates should be prepared per their layers. For example, you would have separate templates for the VPC, ECS cluster, and so on. Finally, the templates can be published as Blueprints.

We mentioned the complicated AWS account setup earlier. This setup can be replicated using Spaces and guarded by Policies. 

When the development team creates their stacks from blueprints, they use Stack Dependencies to connect all stacks in a logical chain of deployment.

The first diagram below presents the process used by the company. The second shows the process with Spacelift. I also show additional skills the team needs to work with these processes.

This diagram shows the process as-is. It is clear that the development team has to do a lot of manual work on infrastructure tasks to create the templates properly. The platform team doesn’t know who is using the snippets (and how).

Even worse, the development team has full access to the snippets, and they can modify them. These issues add tension and uncertainty in the process.

The second diagram shows the Spacelift approach for self-service infrastructure. We have a clear separation of responsibilities, and we use one central tool. The respective responsibilities of the platform and development teams are clearly defined. There is an established control and communication channel.

Development teams can focus on their main goal — to deliver software. The platform team controls the CloudFormation blueprints and these cannot be modified by development teams, enforce configuration best practices.

The platform team can also easily track the quality of executions and improve the blueprints when needed.

Case 2: Multicloud Kubernetes

Kubernetes shines as an agnostic tool that can be used in a range of applications. It can be deployed on AWS and Azure or AWS and on-prem, for example. Such multicloud setups are useful, especially when different workloads operate under different security guardrails.

In this example, we explore creating self-service infrastructure for such a scenario.

The problem

This is a fairly straightforward case, in which the development team wants to quickly deploy a workload on a Kubernetes cluster. Depending on security recommendations, deployment will take place in AWS or on-prem.

How it can be solved

For this exercise, we assume that the platform team is in charge of Kubernetes infrastructure and keep the solution simple, with one AWS account per environment and one Kubernetes cluster. The same setup is on-prem — one cluster per environment.

Let’s take a look at the responsibilities of platform and development teams in this scenario:

As we can see, the platform team is responsible for the underlying infrastructure. Fortunately, this responsibility can be executed through Spacelift, in the same way that development teams use it.

To avoid confusion for development teams, Spacelift administrators create Spaces which logically separate administrative work done by platform team from development work. Also, another set of Spaces can be created to separate environments and clusters (on-prem, AWS). This logical structure must be part of SDLC design.

The diagram above illustrates the potential process of this solution. It offers simple and compelling benefits:

• All decisions and deployments are made with one tool (Spacelift).
• Only one team is responsible for infrastructure.
• Developers need to know only where to deploy workload. From their perspective, the additional work is to select a defined worker pool.
• No additional knowledge is expected from development teams.
• Development teams are not blocked with their deployments.
• Spaces separate the responsibilities of platform and development teams.

Case 3: Create entire project for development teams

In this case, we look at a real-case scenario in which the platform team sets out to deliver fully functional templates for development teams to create projects. These templates must contain infrastructure on AWS, as well as GitHub repositories and pipelines.

We will approach this case in two steps. We have already created an appropriate setup in Spacelift, with Spaces, etc.

The first step is to create and configure the GitHub repository. Configuration involves assigning proper users, configuring the protected branches, and so on.

The second step is to deploy infrastructure on AWS. This step is strictly related to infrastructure.

This is how the process might be constructed:

The platform team creates the templates for all use cases. There may be just one template related to GitHub repository creation, enabling almost complete control over the way repositories are constructed and managed. This is a very important consideration when your organization is scaling. The approach allows you to keep the SDLC approach unified for the whole organization, which can be important from a regulatory perspective.

For the development team, this is a very straightforward step. They simply select the name for the repositories and who should have access to them. They are then ready to ship their code into the repository.

The second step is similar to previous scenarios. The platform team prepares and manages blueprints for infrastructure, and development teams use these blueprints to create their infrastructure for applications.

In all the scenarios we’ve outlined, self-service infrastructure and a properly designed underlying system enabled the development teams to work autonomously. These factors are very important in Agile environments. However, we cannot forget about security, best practices, proper configuration of infrastructure, cost effectiveness, and other considerations crucial for modern organizations to operate effectively. These are the responsibilities of well-informed platform teams. 

Development teams in modern organizations are expected to deliver software as efficiently as possible. We cannot allow Lean waste such as a waiting state to create blockages for development teams, who must wait for ops teams to deliver infrastructure. The self-service approach allows development teams to become independent and able to deliver their software without advancing their skillset significantly. 

Spacelift allows organizations to organize and control workloads with one tool. This is a very efficient and cost-effective way of working in the modern IT industry. With components like Blueprints and Spaces all in one place, Spacelift delivers a fully functional platform organizations can use to deliver their solutions to customers quickly and effectively.

If you would like to discover how the Spacelift platform could enhance flexibility and efficiency in your organization, try it for free or book a demo with one of our engineers.