Tianchen is a guest author at Spacelift. He is focused on Data Science, Cloud Computing, Data Engineering, DevSecOps in the world of digitalization and automation.

All posts by Tianchen Wu - DevOps & Cloud Expert

Tianchen Wu

Author title

OpenTofu is now part of the Linux Foundation 🎉

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

General | Spacelift Blog

General

Want to level up your Terraform knowledge? Explore our tutorials and step-by-step guides written by Terraform experts. Learn Terraform best practices and solutions to the most difficult problems.

Terraform Guides and Tutorials | Spacelift Blog

Terraform

Terraform is a well-known IaC tool from Hashicorp. Terragrunt is a thin wrapper of Terraform that provides extra tools for keeping your configurations DRY, working with multiple Terraform modules, and managing remote states. It is an open source project from Gruntwork, co-founded by Yevgeniy Brikman, author of Terraform up and running.
This article discusses common patterns in <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code (IaC)</a> based on Terraform and Terragrunt examples and summarizes and highlights the design principle underlying good architecture.
We will present these patterns in an abstract way. Although the examples presented in this article use Terraform and <a href="https://spacelift.io/blog/terragrunt" target="_blank" rel="noopener">Terragrunt</a>, the main concepts apply to other popular IaC tools and platforms like <a href="https://spacelift.io/blog/what-is-pulumi" target="_blank" rel="noopener">Pulumi</a>, AWS CDK, CloudFormation, <a href="https://spacelift.io/blog/what-is-terraform-cloud" target="_blank" rel="noopener">Terraform Cloud</a>, <a href="https://spacelift.io/blog/alternative-to-atlantis" target="_blank" rel="noopener">Atlantis</a>, Spacelift, and <a href="https://spacelift.io/terraform-cloud-alternative" target="_blank" rel="noopener">Terraform Cloud alternatives</a>.
To put it simply, “managing collections of infrastructure resources” is an abstract concept. The technologies used to implement this concept include:
<ul>
<li style="font-weight: 400;" aria-level="1"><a href="https://spacelift.io/blog/terraform-workspaces" target="_blank" rel="noopener">Terraform workspace</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacks.html" target="_blank" rel="nofollow noopener">Cloudformation (nested) stack</a></li>
</ul>
Git branch (allows you to create two branches, one called workspace_1, the other workspace_2, then relate remote state to each of the branches, which is actually a neat trick.)

In short, to utilize IaC is to describe your infrastructure with code. Coding infrastructure in HCL (<a href="https://github.com/hashicorp/hcl" target="_blank" rel="nofollow noopener">Hashicorp Configuration Language</a>) with Terraform is just like coding with any other programming language, like Python or Java. Programming technologies such as linter, IDE, unittest, documentation, and CICD, are all applicable to IaC. Additionally, a project layout is crucial to defining a structure to organize your code, dependencies, configuration files, etc.
One common pattern of the project layout for IaC is much like the following:

image1-3

common pattern of the project layout for IaC

The basic principle is to:
<ul>
<li style="font-weight: 400;" aria-level="1">Separate the process of IaC into two phases: development and deployment.</li>
<li style="font-weight: 400;" aria-level="1">Separate the deployment environments into prod, qa, and stage, for example.</li>
<li style="font-weight: 400;" aria-level="1">Separate the cloud architecture into components according to projects, types of infrastructure, etc. Each component can be stored in a single “gigantic” repository or multiple “micro” repositories (see Pattern: Mono-repo vs Multi-repo).</li>
</ul>

The process of IaC usually consists of two phases: development and deployment (if you are interested in this topic, the <a href="https://www.weave.works/technologies/gitops/" target="_blank" rel="noopener">GitOps and weaveworks articles</a> are good sources of information).
In the development phase, one writes code to describe and configure the desired infrastructure. In the deployment phase, the written infrastructure code is applied.
The advantages of separation:
<ul>
<li style="font-weight: 400;" aria-level="1">Separation of Concern: developers who write IaC do not need to know the details of the deployment, and can work or test on a new version of the code without influencing the actual deployed infrastructure.</li>
<li style="font-weight: 400;" aria-level="1">Flexibility: a well-designed IaC codebase is modularized and parameterized. The same code snippet can be reused and deployed under different use cases in different environments with customized parameters.</li>
</ul>

<ul>
<li><a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Deploy your Infrastructure in CI/CD using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Deploy your Infrastructure in CI/CD using Terraform</a></li>
<li><a href="https://spacelift.io/blog/5-ways-to-manage-terraform-at-scale" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Managing Terraform at Scale - Best Practices&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Managing Terraform at Scale – Best Practices</a></li>
<li><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Improve Your Infrastructure as Code using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Improve Your Infrastructure as Code using Terraform</a></li>
</ul>

Imagine you have an AWS cloud architecture for web services, including different AWS services like ECS, RDS, S3, IAM, SSM, Secret Manager, Cloudwatch, Cloudfront and so on. One option is to pack all of the components into a single Terraform remote state (main.tf). This is fine for quick and dirty prototyping, but not recommended for production.
Always cut your cloud architecture into meaningful components such as:
<ul>
<li style="font-weight: 400;" aria-level="1">Stateful components like database, filesystem, or S3.</li>
<li style="font-weight: 400;" aria-level="1">Stateless components like lambda, ECS, or API Gateway.</li>
</ul>
You can also cut it based on projects or functional areas (e.g. frontend, backend, data warehouse).
The advantages of separation:
<ol>
<li style="font-weight: 400;" aria-level="1">Avoids scanning the whole architecture if a change needs to be made only locally to one resource. One can enter the component folder which includes that resource, make changes there, and apply them. Scanning a fat cloud account can be time consuming on a large amount of cloud resources, even causing unnecessary API throttling.</li>
<li style="font-weight: 400;" aria-level="1">Avoids unnecessary touches on databases or other crucial infrastructure to prevent accidental destructive operations.</li>
<li style="font-weight: 400;" aria-level="1">Increases readability of the IaC.</li>
</ol>
Translate the above described project layout into folder structure:

image3-3

folder structure

The separation of the development and deployment process is implemented with two git repositories &#8211; env/ and modules/. Notice the .git in the folder. Here, Terragrunt.hcl is simply a configuration file for modules specifically in the language of Terragrunt. The separation of cloud architecture components is implemented with subfolders app/ , mysql/, vpc/. The separation of deployment environments is implemented with subfolders prod/ , qa/ , stage/.
Check out our <a href="https://spacelift.io/blog/terragrunt-vs-terraform" target="_blank" rel="noopener">Terragrunt vs. Terraform comparison</a>.

One of the principles of software engineering is DRY (don’t repeat yourself). The same applies to IaC. There are at least two kinds of tricks you can use to DRY IaC:
<ul>
<li style="font-weight: 400;" aria-level="1">Modularization</li>
<li style="font-weight: 400;" aria-level="1">Configuration inheritance</li>
</ul>
<h2>1) Modularization</h2>
A <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform module</a> is a way of creating a template of a cloud pattern, parameterizing, and reusing it. Like any other artifacts, modules can be stored and versioned in the registry. The official Terraform module registry is <a href="https://registry.terraform.io/" target="_blank" rel="nofollow noopener">https://registry.Terraform.io/</a>. You can also build your own private registry.
<h2></h2>

image2-3

AWS security group

<h2>2) Configuration Inheritance</h2>
<a href="https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#include" target="_blank" rel="nofollow noopener">Configuration inheritance</a> is a feature supported by Terragrunt. The concept is similar to class inheritance in object oriented programming, except that configuration inheritance serves the purpose of configuration reusability.
In the following example (also see the project layout above), we have a configuration file at the global level (global_vars.yaml) and configuration files at the environment level (environment_specific_vars.yaml). 
In global_vars.yaml, variables like account_id, region_name and repository_url can be defined, and in environment_specific_vars, these global variables will be inherited and present, additionally defining new environment level variables like vpc_id (usually we use separate vpc for different environments), environment_name, etc.

image4-3

Configuration Inheritance

In summary, configuration inheritance increases the reusability of configuration files and reduces the difficulty in maintenance.
Also, by running your <a href="https://spacelift.io/blog/terragrunt-with-spacelift" target="_blank" rel="noopener">Terragrunt commands on Spacelift</a>, you can extend the powers of Terragrunt with other features such as Spacelift <a href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="noopener">Contexts</a> and <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">Policies</a>.
With Spacelift Contexts, you can define a collection of variables and files and then attach these to Spacelift stack(s). Spacelift Policies allow you to build rules using the <a href="https://spacelift.io/blog/what-is-open-policy-agent-and-how-it-works" target="_blank" rel="noopener">Open Policy Agent (OPA)</a> to govern your infrastructure as code and its automation. For example, you may want to disallow a specific resource type or even a certain way of configuring that resource type – you can easily achieve such a thing using a <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">Plan Policy</a>. You can also use <a href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a> to mix and match Terraform, Pulumi, and CloudFormation Stacks and have them talk to one another.

Last but not least, let’s review the concept of mono-repo and multi-repo. It has been pointed out by Hashicorp as a <a href="https://www.hashicorp.com/blog/terraform-mono-repo-vs-multi-repo-the-great-debate" target="_blank" rel="nofollow noopener">major architectural decision for IaC projects</a>, a view I share.
The problem is, there are two ways to organize code. In simple terms, mono-repo stores all code into one git repository (for example with multiple subfolders), whereas multi-repo separates them into multiple repositories.
Analogously, the relationship between mono-repo and multi-repo can be compared to monolith software architecture and microservice architecture. Mono-repo is intended to perceive the architecture as a whole and as evolving together. It is usually more suitable for small projects or big projects in prototyping. Multi-repo, on the other hand, facilitates collaboration but with extra overhead. Naturally, the more repos you have, the higher the maintenance cost, but also the more autonomous and self-service each repository is.

In this article, we reviewed several patterns of IaC architecture in a technology-agnostic way. All of these patterns can be easily implemented in Spacelift, as the product works natively with both <a href="https://docs.spacelift.io/vendors/terraform/" target="_blank" rel="noopener">Terraform</a> and <a href="https://spacelift.io/blog/introducing-terragrunt-native-support" target="_blank" rel="noopener">Terragrunt</a>.
Your workflows can be extended with other features like policies, and third-party integrations and you even have the possibility to bring your own runner image. 
Spacelift’s flexibility extends to other popular IaC technologies like Pulumi, CloudFormation, Ansible, and Kubernetes, where the same patterns keep appearing in slight variations.
We rooted the principles of IaC architecture design in software engineering concepts like inheritance, DRY, project layout, parameterization, and so on. After all, Infrastructure, as Code is coding infrastructure, and recognizing common patterns, is the recommended way to learn and apply IaC.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence.

Whenever you're embarking on a new journey or seeking to refine your foundational knowledge.

frame-8479

Frame 8479

5 Ways to Manage Terraform at Scale - Best Practices

What is Multi-Cloud Infrastructure? Benefits, Best Practices

What is tfsec? How to Install, Config, Ignore Checks

See common patterns of Infrastructure as Code architecture in Terraform and Terragrunt. Learn the differences between common patterns of IaC.

Tianchen Wu

Common Patterns of Infrastructure as Code Architecture: Terraform & Terragrunt

Get the newsletter

No recent posts