Stanislaw is a guest writer at Spacelift. He is a Systems, Service Reliability Engineer, and Cloud specialist.

All posts by Stanislaw Szymanski - Terraform Expert

Stanislaw Szymanski

Author title

Spacelift's going to AWS re:Invent in Las Vegas 🚀 🚀

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

Introducing Stack Dependencies v.2  🎉 🎉

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

Infrastructure Problems that Spacelift Solves

Learn more about new features, updates, and Spacelift product announcements.

Product Features and Updates | Spacelift Blog

Product

For every issue, there is a solution.
Infrastructure is no different—due to its intricacies, complexity and the purpose it serves, troubles are guaranteed. Especially when it comes to Infrastructure as Code, as even though this concept was created to fix many extremely difficult problems it did create a few new ones.
For those, Spacelift comes to the rescue. Let’s talk about five infrastructure problems that Spacelift solves for you.

Although the modular approach is great, it presents a little quirk that you might find out about only when it’s too late. A painful one, at that. Modules are clean, efficient, granular… but not immortal. Things in the infrastructure field tend to change and some of those changes might end up breaking your modules—one unfortunate version bump, and things collapse.
Spacelift provides automated testing for the configuration modules it manages and I cannot emphasize enough how significant this is. So significant, in fact, that we would want to dedicate an entire article just to this outstanding functionality (stay tuned for an in-depth explanation of the registry testing). 
To test is to know what to expect. Knowing what to expect gives you peace of mind and confidence that things will work out correctly. Even better—you don’t have to lift a finger to run comprehensive tests for your modules and infrastructure.

image4-5

Broken Modules Tearing Down Your Configurations

Inanimate objects and things have an irritating tendency to not stay as you left them. If you have ever lost your keys, you will surely relate. 
Infrastructure is no different in this matter. Unexpected situations, manual tampering or the workings of force majeure could lead to differences between the configuration you have applied and the current state of the system.
Sometimes the reasons are understandable. For example, when in an emergency your engineer manually introduces crucial infrastructure changes to avoid downtime or other damages; sometimes it’s not just a result of a script or person doing what they shouldn’t be. 
Spacelift introduces <a href="https://docs.spacelift.io/concepts/stack/drift-detection.html" target="_blank" rel="noopener">automatic drift detection runs</a>. That’s right—periodically (how often? You decide or trust the defaults), Spacelift can check your infrastructure, detect drifts, provide you information about the resources they impact:
<ul>
<li style="font-weight: 400;" aria-level="1">What was impacted? </li>
<li style="font-weight: 400;" aria-level="1">When did this happen?</li>
</ul>
Then, you can properly introduce them into your state in a process called reconciliation if you decide that they are beneficial or required.

image5-4

Drifting Away From What You Had Defined

Security is an important factor of every infrastructure, Big or small—if insecure, it won’t stand long. Unfortunately, maintaining security and keeping compliance is cumbersome, to say the least. There are many things to take into consideration and even more that you might forget about&#8230; until they come back to haunt you. 
With some smart tricks, Spacelift makes security much easier—for example, it <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">provides support for policies</a>. And what are those? Internally, Spacelift leverages the well-known <a href="https://spacelift.io/blog/what-is-open-policy-agent-and-how-it-works" target="_blank" rel="noopener">Open Policy Agent</a> and the Rego language. This combination allows you to define policies, affecting various aspects of daily operation such as logging in, starting Tasks and Runs, or command execution.
You can enforce compliance by code. 
Another example is data handling. Data that you exchange with Spacelift is encrypted both at rest and in transit, and temporary credentials are used where needed—the secret data is protected by multiple layers of additional security. 
And when it comes to shared workers and their credentials, granting static, permanent access to your AWS, Azure Cloud or GCP wouldn’t exactly be a good practice. Thanks to Spacelift’s integration with identity management systems, dynamically-generated temporary credentials can be used to provide necessary access during runs.
Whether you don’t trust shared workers or just prefer to provide and use your own with Spacelift —this is also possible.

<ul>
<li><a href="https://spacelift.io/blog/why-devops-engineers-recommend-spacelift" target="_blank" rel="noopener">Why DevOps Engineers Recommend Spacelift</a></li>
<li><a href="https://spacelift.io/blog/spacelift-module-registry" target="_blank" rel="noopener">How to Use Spacelift Module Registry</a></li>
<li><a href="https://spacelift.io/blog/alternative-to-atlantis" target="_blank" rel="noopener">Alternative to Atlantis – Spacelift</a></li>
</ul>

<a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code</a> significantly improves collaboration. It is much easier to work with someone on a piece of code than on a server spinning up, but there are still a few peculiarities that you might find out about along the way. 
Take this example. Let’s assume you’ve got some configuration elements, like environment variables, that you wish to use as shared—between various groups of resources. Normally, you’d need to individually provide those values to each resource set that needs them by defining them multiple times, in different places. To significantly ease this process, Spacelift introduces the <a href="https://docs.spacelift.io/concepts/configuration/context.html" target="_blank" rel="noopener">concept of contexts</a>—grouped declarations of variable values and useful files that you can mount to your stacks. You declare them once and use them where you need, with no security disadvantages or time wasted on useless repetition.
Another situation that you might run into, would be when you’d want to use your <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> in configurations outside your organization. For example, you have five customers. Each one has a separate account, and ideally, you’d want to retain control over the configuration modules you made, to make sure no one breaks anything. But—you also want to use the configuration you already have, for repetitive tasks between the infrastructures of those customers. Spacelift makes this possible:

image3-6

Troublesome Collaboration

Just provide the names of accounts you wish to share your code with, and let those accounts build their infrastructure with your curated resources. Easy!
Still unconvinced? There’s much more. Seamless integration with popular VCS providers such as GitHub or BitBucket, push status notifications… and of course, preview runs, showing you what will change as a result of the pull request.

image2-6

Seamless integration with popular VCS providers

As you can see, there’s something here for everyone. Spacelift is developed with collaboration in mind. It blends in very well but provides features that stand out. Even if you don’t really like using external tools or technologies, you won’t be disappointed with the results.

Everyone at least once in their lifetime bought something and then watched in disbelief as it diminished before their eyes. The pain of a huge Amazon Web Services bill is definitely real. 
AWS bills are often mentioned in the countdowns of some of the most massive things in existence—somewhere between cargo ships, black holes, and the NodeJS node_modules directories. Some engineers even say that you don’t really need coffee in your life if you’re provisioning AWS infrastructure—the bill will raise your blood pressure just the same. 
Spacelift integrates with <a href="https://www.infracost.io/" target="_blank" rel="nofollow noopener">Infracost</a>, to provide you with <a href="https://docs.spacelift.io/vendors/terraform/infracost.html" target="_blank" rel="noopener">cost estimates of your runs</a>. This way your pull request can tell you how much it will cost to apply the configuration you have designed. It also allows you to set up proper cost barriers to make sure that you won’t spend more than expected just by misconfiguration or burn the entire monthly budget in a week by one unfortunate deployment. 
Read more about it in our <a href="https://spacelift.io/blog/terraform-cost-estimation-using-infracost" target="_blank" rel="noopener">Terraform Cost Estimation with Infracost</a> article.
The setup is fairly simple, but to make it even easier you can create a context mentioned in an earlier section and attach it to the stacks you wish to monitor in terms of budget.

image1-6

Budgets Out of Hand

As you’ve seen, while the Infrastructure as Code approach definitely solves many issues, there’s still a huge lot to go around. Every bit of assistance or automation is precious and can provide tremendous benefits.
If you wish to make the management of your infrastructure easier, less time-consuming, the infrastructure itself more robust and dependable… and also keep everything as secure and as under control as possible—<a href="https://spacelift.io/">Spacelift</a> is the way to go. I’m sure you will love it.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence.

Infrastructure as Code : Best Practices, Benefits & Examples

Managing Azure Resources using Spacelift

How to Deploy Infrastructure in CI/CD Using Terraform (Pipeline)

See what infrastructure problems Spacelift solves. Learn about Spacelift’s automated testing, auto drift detection runs, and cost estimation.

3-infrastructure_problems_that_spacelift_solves

Want to level up your Terraform knowledge? Explore our tutorials and step-by-step guides written by Terraform experts. Learn Terraform best practices and solutions to the most difficult problems.

Terraform Guides and Tutorials | Spacelift Blog

Terraform

Flavius Dinu

The native language of Terraform, called HCL, can be an extremely convenient and efficient tool, if you know it well. Although it is not a programming language (even though operators coding in HCL are often jokingly called “HCL Developers”), it does implement some structures and logic that bear resemblance to traditional programming. 
In this article, we will discuss the HCL’s core – built-in functions, expressions and loops. What are those, and what are they for? Let’s dive in.

What are Terraform Functions? <dfn>Terraform functions</dfn> are built-in, reusable code blocks that perform specific tasks within Terraform configurations. They make your code more dynamic and ensure your configuration is DRY. Functions allow you to perform various operations, such as converting expressions to different data types, calculating lengths, and building complex variables.
These functions are split into multiple categories:
<ul class="">
<li id="38a4" class="ke kf hy ji b jj jk jn jo jr kg jv kh jz ki kd kj kk kl km bi" data-selectable-paragraph="">String</li>
<li id="48e1" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Numeric</li>
<li id="686b" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Collection</li>
<li id="f342" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Date and Time</li>
<li id="bd5c" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Crypto and Hash</li>
<li id="f378" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Filesystem</li>
<li id="a78a" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">IP Network</li>
<li id="6519" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Encoding</li>
<li id="d265" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">Type Conversion</li>
</ul>
This split, however, can become overwhelming to someone who doesn’t have that much experience with Terraform. For example, the formatlist list function is considered to be a string function even though it modifies elements from a list. A list is a collection though; some may argue that this function should be considered a collection function, but still, at its core, it does change to strings.
For that particular reason, I won’t specify the function type when I’ll describe them, but just go with what you can do with them. Of course, I will not go through all of the available functions, but through the ones I am using throughout my configurations.
<h3 id="7e48" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">ToType Functions</h3>
ToType is not an actual function; rather, many functions can help you change the type of a variable to another type.
<code class="eq ls lt lu lv b">tonumber(argument)</code> &#8211; With this function, you can change a string to a number, anything else apart from another number and null will result in an error.
<code class="eq ls lt lu lv b">tostring(argument)</code> &#8211; Changes a number/bool/string/null to a string.
<code class="eq ls lt lu lv b">tobool(argument)</code> &#8211; Changes a string (only “true” or “false”)/bool/null to a bool.
<code class="eq ls lt lu lv b">tolist(argument)</code> &#8211; Changes a set to a list.
<code class="eq ls lt lu lv b">toset(argument)</code> &#8211; Changes a list to a set.
<code class="eq ls lt lu lv b">tomap(argument)</code> &#8211; Converts its argument to a map.
In Terraform, you are rarely going to need to use these types of functions, but I still thought they are worth mentioning.
<h3 id="3f71" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">format(string_format, unformatted_string)</h3>
The format function is similar to the printf function in C and works by formatting a number of values according to a specification string.
It can be used to build different strings that may be used in conjunction with other variables. Here is an example of how to use this function:

<h3 id="c6c0" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">formatlist(string_format, unformatted_list)</h3>
The formatlist function uses the same syntax as the format function but changes the elements in a list.
Here is an example of how to use this function:

<h3 id="a0bd" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">length(list / string / map)</h3>
Returns the length of a string, list, or map.

<h3 id="5a0f" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">join(separator, list)</h3>
Another useful function in Terraform is <a href="https://spacelift.io/blog/terraform-join" target="_blank" rel="noopener">“join”</a>. This function creates a string by concatenating together all elements of a list and a separator. For example, consider the following code:

The output of this code will be “a, b, c”.
<h3 id="67f7" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">try(value, fallback)</h3>
Sometimes, you may want to use a value if it is usable but fall back to another value if the first one is unusable. This can be achieved using the “try” function.
For example:

The output of this code will be “fallback”, as the expression <code class="eq ls lt lu lv b">local.map_var.test2</code> is unusable.
<h3 id="a3f4" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">can(expression)</h3>
A useful function for validating variables is “can”. It evaluates an expression and returns a boolean indicating if there is a problem with the expression. For example:

The validation in this code will give you an error: “This is not a number: 1”.
<h3 id="c78f" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">flatten(list)</h3>
In Terraform, you may work with complex data types to manage your infrastructure. In these cases, you may want to flatten a list of lists into a single list.
This can be achieved using the <a href="https://spacelift.io/blog/terraform-flatten" target="_blank" rel="noopener">Terraform “flatten” function</a>, as in this example:

The output of this code will be [1, 2, 3, 4, 5, 6].
<h3 id="fcf3" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">keys(map) &amp; values(map)</h3>
It may be useful to extract the keys or values from a map as a list. This can be achieved using the “keys” or “values” functions, respectively.
For example:

<h3 id="f98b" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">slice(list, startindex, endindex)</h3>
Slice returns consecutive elements from a list from a startindex (inclusive) to an endindex (exclusive).

The output for the above would be <code class="eq ls lt lu lv b">slice_list = [3]</code>.
<h3 id="7030" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">range</h3>
Creates a range of numbers:
<ul class="">
<li id="6080" class="ke kf hy ji b jj jk jn jo jr kg jv kh jz ki kd kj kk kl km bi" data-selectable-paragraph="">one argument(limit)</li>
<li id="6cdd" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">two arguments(initial_value, limit)</li>
<li id="0e1b" class="ke kf hy ji b jj kn jn ko jr kp jv kq jz kr kd kj kk kl km bi" data-selectable-paragraph="">three arguments(initial_value, limit, step)</li>
</ul>

The output for this code would be:

<h3 id="494d" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">lookup(map, key, fallback_value)</h3>
Retrieves a value from a map using its key. If the value is not found, it will return the default value instead.

This will return: <code class="eq ls lt lu lv b">lookup_in_a_map = "key1"</code>
Learn more about the <a href="https://spacelift.io/blog/terraform-lookup" target="_blank" rel="noopener">Terraform <code>lookup</code> function</a>.
<h3 id="49a2" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">concat(lists)</h3>
Takes two or more lists and combines them in a single one.

This will return: <code class="eq ls lt lu lv b">concat_list = [1, 2, 3, 4, 5, 6]</code>
<h3 id="b64f" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">merge(maps)</h3>
The <code class="eq ls lt lu lv b">merge</code> function takes one or more maps and returns a single map that contains all of the elements from the input maps. The function can also take objects as input, but the output will always be a map.
Let’s take a look at an example:

<h3 id="0d88" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">zipmap(key_list, value_list)</h3>
Constructs a map from a list of keys and a list of values.

<h3 id="3f49" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">expanding function argument …</h3>
This special argument works only in function calls and expands a list into separate arguments. Useful when you want to merge all maps from a list of maps.

<h3 id="468d" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">file(path_to_file)</h3>
Reads the content of a file as a string and can be used in conjunction with other functions like jsondecode / yamldecode.

The output would be the content of the file called a_file as a string.
<h3 id="c6fe" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">templatefile(path, vars)</h3>
Reads the file from the specified path and changes the variables specified in the file between the interpolation syntax ${ … } with the ones from the vars map.

This will change the ${change_me} variable to awesome_value.
<h3 id="dd1e" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">jsondecode(string)</h3>
Interprets a string as json.

<h3 id="c4e1" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">jsonencode(string)</h3>
Encodes a value to a string using json.

<h3 id="4747" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">yamldecode(string)</h3>
Parses a string as a subset of YAML and produces a representation of its value.

<h3 id="5be5" class="ks kt hy bd ku kv kw kx ky kz la lb lc jr ld le lf jv lg lh li jz lj lk ll lm bi">yamlencode(value)</h3>
Encodes a given value to a string using YAML.

You can use Terraform functions to make your life easier and to write better code. Keeping your configuration as DRY as possible improves readability and makes updating it easier, so functions are a must-have in your configurations.
These are just the functions that I am using the most, but some honorable mentions are <code class="eq ls lt lu lv b">element</code>, <code class="eq ls lt lu lv b">base64encode</code>, <code class="eq ls lt lu lv b">base64decode</code>, <code class="eq ls lt lu lv b">formatdate</code>, <code class="eq ls lt lu lv b">uuid</code>, and <code class="eq ls lt lu lv b">distinct</code>.

Expressions are the core of HCL itself – the logic muscle of the entire language. <dfn>Terraform expressions</dfn> allow you to get a value from somewhere, calculate or evaluate it. You can use them to refer to the value of something, or extend the logic of a component – for example, make one copy of the resource for each value contained within a variable, using it as an argument.
They are used pretty much everywhere – the most simple type of expression would be a literal value – so, there is a great chance that you have already used them before.
As it is an extremely extensive topic, I couldn’t possibly fit everything into this single article. Instead, I can talk about what in my humble opinion are the most notable/interesting expressions: Operators, Conditional expressions, Splat Expressions, Constraints, and Loops.
<h3>Operators</h3>
Dedicated to logical comparison and arithmetic operations, operators are mostly used for doing math and basic Bool’s algebra. If you need to know if number A equals number B, add them together, or determine if both boolean A and boolean B are “true”, Terraform offers the following operators:
<h4>Types of Terraform Operators</h4>
<ul>
<li style="font-weight: 400;" aria-level="1">Arithmetic operators – the basic ones for typical math operations (+, -, *, /) and two special ones: “X % Y” would return the remainder of dividing X by Y, and “-X”, which would return X multiplied by -1. Those can only be used with numeric values.</li>
<li style="font-weight: 400;" aria-level="1">Equality operators – “X == Y” is true, if X and Y have both the same value and type, “X != Y” would return false in this case. This one will work with any type of value.</li>
<li style="font-weight: 400;" aria-level="1">Comparison operators &#8211; “&lt;, &gt;, &lt;=, &gt;=” &#8211; exclusive to numbers, returns true or false depending on the condition. </li>
<li style="font-weight: 400;" aria-level="1">Logical operators – the Bool’s algebra part of the pack, work only with the boolean values of true and false. 
 &#8211; “X || Y” returns true if either X or Y is true, false if any of them is false. 
 &#8211; “X &amp;&amp; Y” returns true only, if both X and Y are true, false if any of them is false. 
            &#8211; “!X” is true, if X is false, false if X is true.</li>
</ul>
<h3>Conditionals</h3>
Sometimes, you might run into a scenario where you’d want the argument value to be different, depending on another value. The conditional syntax is as such:

The condition part is constructed using previously described operators. In this example, the bucket_name value is based on the “test” variable—if it’s set to true, the bucket will be named “dev” and if it’s false, the bucket will be named “prod”:

<h3>Splat Expressions</h3>
Splat expressions are used to extract certain values from complicated collections – like grabbing a list of attributes from a list of objects containing those attributes. Usually you would need an “for” expression to do this, but humans are lazy creatures who like to make complicated things simpler.
For example, if you had a list of objects such as these:

Instead of using the entire “for” expression:

you could go for the splat expression form:

And in both cases, get the same result:

Do note, that this behavior applies only if splat was used on a list, set, or tuple. Anything else (Except null) will be transformed into a tuple, with a single element inside, null will simply stay as is. This may be good or bad, depending on your use case. I would suggest you read <a href="https://www.terraform.io/docs/language/expressions/splat.html#single-values-as-lists" target="_blank" rel="nofollow noopener">the official documentation</a> on the topic.
<h3>Constraints</h3>
In simple terms, constraints regulate what can be what, and where something can, or cannot be used. There are two main types of constraints—for types and versions.
<ul>
<li style="font-weight: 400;" aria-level="1">Type constraints regulate the values of <a href="https://spacelift.io/blog/how-to-use-terraform-variables" target="_blank" rel="noopener">variables</a> and outputs. For example, a string is represented by anything enclosed in quotes, bool value is either a literal true or false, a list is always opened with square brackets [ ], a map is defined with curly brackets { }.</li>
<li style="font-weight: 400;" aria-level="1">Version constraints usually apply to modules and regulate which versions should or should not be used. See this article to learn more about it: <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">What are Terraform Modules and How to Use Them (Examples)</a></li>
</ul>

<ul>
<li><a href="https://spacelift.io/blog/5-ways-to-manage-terraform-at-scale" target="_blank" rel="noopener">5 Ways to Manage Terraform at Scale</a></li>
<li><a href="https://spacelift.io/blog/terraform-automation" target="_blank" rel="noopener">How to Automate Terraform Deployments and Infrastructure Provisioning</a></li>
<li><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener">How to Improve Your Infrastructure as Code using Terraform</a></li>
</ul>

Ah yes, the elephant in the room. Mentioned a few times, but still unexplained—until now.
<dfn>Terraform loops</dfn> are used to handle collections, and to produce multiple instances of a resource or module without repeating the code. There are three loops provided by Terraform to date:
<h3>Count</h3>
Count is the most primitive—it allows you to specify a whole number, and produces as many instances of something as this number tells it to. For example, the following would order Terraform to create ten S3 buckets:

When count is in use, each instance of a resource or module gets a separate index, representing its place in the order of creation. To get a value from a single resource created in this way, you must refer to it by its index value, e.g. if you wished to see the ID of the fifth <a href="https://spacelift.io/blog/terraform-aws-s3-bucket" target="_blank" rel="noopener">created S3 bucket</a>, you would need to call it as such:

Although this is fine for identical, or nearly identical objects, as previously mentioned, count is pretty primitive. When you need to use more distinct, complex values – count yields to for_each.
<h3>For_each</h3>
As mentioned earlier, sometimes you might want to create resources with distinct values associated with each one – such as names or parameters (memory or disk size for example). For_each will let you do just that. Merely provide a variable—map, or a set of strings, and the resources can access values contained within, via each.key and each.value:

As you can see, for_each is quite powerful, but you haven’t seen the best yet. By constructing a map of objects, you can leverage a resource or module to create multiple instances of itself, each with multiple declared variable values:

Using this approach, you don’t have to touch anything except the .tfvars file, to provide new instances of resources you have already declared in your configuration. Absolutely brilliant.
If you want to know more about Terraform count and for_each meta-arguments, see: <a href="https://spacelift.io/blog/terraform-count-for-each" target="_blank" rel="noopener">Terraform Count vs. For_Each Meta-Argument: When to Use Them</a>
<h3>For</h3>
For is made for picking out, iterating over, and operating on things from complex collections. Imagine that you have a list of words (strings), but unfortunately, all of them contain newline characters at the end which you don’t want. Like this:

To fix this problem, you could do

which would result in:

As you can see, a list comes in, a list goes out—but, this is not a must. The type of input, and the brackets which wrap the for expression, determine the type of output it produces. If you had wrapped it with curly brackets, and provided a map as an input, the output would have been a map. Quite interesting. But there’s one thing that’s even more interesting—the <a href="https://spacelift.io/blog/terraform-for-loop" target="_blank" rel="noopener">for</a><a href="https://spacelift.io/blog/terraform-for-loop" target="_blank" rel="noopener"> expression</a> can also be used to filter the input as you please. By adding an if clause, you can conditionally operate or not operate on certain values, depending on your defined terms. Take this example, making every word start with a capital letter… except for the word “teapot”:

Importing Existing Infrastructure Into Terraform - Step by Step

Provisioning AWS EKS Cluster with Terraform - Tutorial

How to Debug & Troubleshoot Terraform Projects: Tutorial

Learn what built-in Terraform functions, expressions, and loops are. See examples and definitions. Terraform functions made easy!

20-terraform-functions

terraform functions

Terraform Functions, Expressions, and Loops (Examples)

Terraform makes it easier to grow your infrastructure and keep its configuration clean. But, as the infrastructure grows, a single directory becomes hard to manage.
That’s where Terraform modules come in. 
In this post you will learn what Terraform modules are, how to use them, and what problems they solve.

A Terraform module is a collection of standard configuration files in a dedicated directory. Terraform modules encapsulate groups of resources dedicated to one task, reducing the amount of code you have to develop for similar infrastructure components.
Some say that Terraform modules are a way of extending your present Terraform configuration with existing parts of reusable code reducing the amount of code you have to develop for similar infrastructure components. Others say that the Terraform module definition is a single or many .tf files stacked together in their own directory. Both are correct.
Module blocks can also be used to force compliance on other resources—to deploy databases with encrypted disks, for example. By hard-coding the encryption configuration and not exposing it through <a href="https://spacelift.io/blog/how-to-use-terraform-variables" target="_blank" rel="noopener">variables</a>, you’re making sure that every time the module is used, the disks are going to be encrypted. 
A typical module can look like this:

As you can see, practically any Terraform configuration is already a module in itself. If you run Terraform in this directory, those configuration files would be considered a root module. It means that this configuration is the base of your operation, a core that you can expand further.
If you need more help learning about Terraform, check out our <a href="https://spacelift.io/blog/terraform-tutorial" target="_blank" rel="noopener">Getting Started With Terraform on AWS</a> tutorial.
Note: New versions of Terraform will be placed under the BUSL license, but everything created before version 1.5.x stays open-source. <a href="https://opentofu.org/" target="_blank" rel="nofollow noopener">OpenTofu</a> is an open-source version of Terraform that will expand on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6. OpenTofu retained all the features and functionalities that had made Terraform popular among developers while also introducing improvements and enhancements. OpenTofu is not going to have its own providers and modules, but it is going to use its own registry for them.

OK, now that you know what a Terraform module is, let’s take a look at a step-by-step process of creating them. So, let’s get started!
<h3>1. Declare that you want to use Terraform modules</h3>
To use a Terraform module, you have to first declare that you wish to use it in your current configuration. To do this, use the module block and provide the appropriate variable values:

As you’re adding the variables, there are a few arguments that you should keep an eye on: their source, version, and four meta-arguments.
<h4>Sources</h4>
Terraforms modules can be stored either locally or remotely. The `source` argument will change depending on their location. For example, if the module you wish to call is stored in a directory named “terraform-test-module” located in the same place as your root module directory, your root configuration would look like this:

But, if you want to keep the module in a VCS (let’s assume git and GitHub), the source will need to point to the correct repository containing the appropriate version of the module that you’re calling:

Terraform modules can also be stored in so-called registries. <a href="https://spacelift.io/blog/terraform-registry" target="_blank" rel="noopener">Registries</a> are places where you can find modules published by fellow Terraform users, or where you store the ones you have created—either privately, for your company/yourself, or publicly, for everyone to enjoy. A Terraform module called from a registry might look like this:

Please note that the URL scheme depends on the registry provider. For example, imagine that you wish to include a VPC module stored in the <a href="https://spacelift.io/blog/spacelift-module-registry" target="_blank" rel="noopener">Spacelift registry</a> in your configuration:

As you’ve probably noticed, those two examples are a bit different from one another. A big advantage of the Spacelift’s registry is that it provides an example for each module. This means that you can check whether the source argument in your configuration is correct. Additionally, you can grab the entire example and use it as a template, filling in the blanks, and starting your work right away!
<h4>Versions</h4>
Versioning enables you to control what module changes should be introduced into your infrastructure. Why is it useful? It helps prevent the damage of your infrastructure caused by unpredictable updates or faulty code. You probably know from experience that any update of software, packages, or applications on a production server can go wrong. This can also happen to Terraform modules, if they’re not kept under control.
The syntax for version constraints is simple. Simply put, a version is a string containing one or more conditions. 
The first part is an operator:
<ul>
<li style="font-weight: 400;" aria-level="1">“=” (or no operator) means “Only one version—this specific version.”</li>
<li style="font-weight: 400;" aria-level="1">“!=” translates to “Other versions are fine, except this one here”</li>
<li style="font-weight: 400;" aria-level="1">“&gt;, &gt;=, &lt;, &lt;=” are used for comparisons. For example, “Use any version newer than 1.0.0, but older than 1.1.0”</li>
<li style="font-weight: 400;" aria-level="1">“~&gt;” is quite interesting. This operator allows only the rightmost part of the version number to increment. In other words, “~&gt;2.6.0” would mean that you wish to use the newest patch version of the module (2.6.&lt;anything&gt;), but not the newer minor or major versions (2.7.0 or above).</li>
</ul>
The second part of the syntax is the version number. Although it isn’t absolutely required (except when using registries), it’s best to stick to the <a href="https://semver.org/" target="_blank" rel="nofollow noopener">Semantic Versioning convention</a>—it’s easy to understand and very transparent. One look at the version, and you know where you are.
As you might have already noticed, there is an exception to the “keep things versioned” rule—in two of the examples above, the version argument is defined. In one, it&#8217;s not. The reason for this is that Terraform considers modules loaded from the same source repository as being of the equal version to the caller. Local modules, by the way, need no version constraints whatsoever.
<h4>Meta-Arguments</h4>
Meta-arguments are special arguments that change the behavior of Terraform when parsing the declared module. Currently, there are four of them in active use:
<ul>
<li style="font-weight: 400;" aria-level="1">“Count” and “for_each” are used to create multiple instances of the same module or resource. More on this: <a href="https://spacelift.io/blog/terraform-count-for-each" target="_blank" rel="noopener">Terraform Count vs. For_Each Meta-Argument: When to Use Them</a> 
</li>
<li style="font-weight: 400;" aria-level="1">“providers”— this meta-argument allows you to directly declare which provider the module should use for its operation. This comes in handy if you have two cloud service accounts and you want to create resources bound to the secondary. In order to achieve that, you need to properly design your Terraform module so that it takes the provider as an argument and passes it along, like this:</li>
</ul>

<ul>
<li>“<a href="https://spacelift.io/blog/terraform-depends-on" target="_blank" rel="noopener">depends_on</a>”—usually, Terraform handles implicit dependencies quite well, but sometimes they aren’t enough. If you need to declare that something should be created before something else, use this meta-argument to define an explicit boundary between resources or modules.</li>
</ul>

<h3>2. Declare Module Outputs</h3>
Sometimes you might need to use the values that are available in the already created resources. A Terraform module completely encapsulates those resources, and here’s how they can be accessed. 
First, declare in your Terraform module that the selected value should be available as an output:

Then, call this value like this:

The module outputs are not passed on from the resources by default—so if you don’t declare an output, it won’t exist.
<h3>3. Taint Terraform Modules</h3>
Sometimes you might need to recreate some or even all component parts of the Terraform module. As there is no way of flagging an entire module that needs to be recreated, you will need to manually taint every resource that was created by the module. While it may seem like a downside, it is actually a feature that offers additional protection against human error. What’s more, you will rarely need to taint an entire Terraform module.
To taint the resources that were created by the example module presented above, try this: 
terraform taint module.example.aws_example_resource.example_something
<h3>4. Test Modules</h3>
Whether a blessing or a curse, Terraform’s code is still code, and should be properly tested. It is absolutely crucial. But when done manually, it can be quite a chore. Luckily, Spacelift, alongside other upgrades and extensions, provides automated module testing and you will absolutely love it. 
Terraform modules managed by the Spacelift registry get tested every time you push a new commit. They are applied, deleted, and you get all of the details on the run. If something fails along the way, the entire test fails, too. If everything goes smoothly, the test is successful. The entire process is automatic, but if you feel like playing around with it, you can even write your own tests, such as this one:

Customizable automated testing—what’s not to love here?

Alright, now that you know how to use Terraform modules and what to keep an eye on when doing it, let’s see what problems Terraform modules solve.
<h3>1. Code Repetition</h3>
As your <a href="https://spacelift.io/blog/5-ways-to-manage-terraform-at-scale" target="_blank" rel="noopener">Terraform infrastructure grows in scale</a>, the code will need to do so as well. Copy-pasting an entire stack of code whenever you need more than a single instance of something isn’t scalable or efficient. It’s a waste of many things, time in particular.
<h3>2. Lack of Code Clarity</h3>
Copy-pasting isn’t clean either. Good code is readable, and great code documents itself. A modular approach addresses all those pesky details. A configuration of connecting modules dedicated to particular tasks is much easier to read and understand. 
<h3>3. Lack of Compliance</h3>
When you create a Terraform module in compliance with appropriate standards and best practices, whenever you reuse it, it follows the same right pattern. No matter if it’s encryption, redundancy, or lifecycle policies—practices configured inside the module will be enforced, so that you won’t have to do it again personally.
<h3>4. Human Error</h3>
When you copy-paste or create a group of resources from scratch, it’s easy to make a mistake, like rewriting or renaming something. To make sure that doesn’t happen, create a single Terraform module, test it, then use it in multiple places to check whether all of the elements are correct. It’s easier to check and test what you type into a single block than scroll, jumping from one place to another, and so on. 
As you can see, there are a lot of advantages to using Terraform modules, so long as they aren’t overused. Find the right balance and keep it.

<ul>
<li>Each Terraform module should live in its own repository and versioning should be leveraged.</li>
<li>Minimal structure: main.tf, variables.tf, outputs.tf.</li>
<li>Each Terraform module should have examples inside of them.</li>
<li>Use input and output variables (outputs can be accessed with <code>module.module_name.output_name</code>).</li>
<li>Used for multiple resources, a single resource module is usually a bad practice.</li>
<li>Use defaults or optionals depending on the data type of your variables.</li>
<li>Use <a href="https://spacelift.io/blog/terraform-dynamic-blocks" target="_blank" rel="noopener">dynamic blocks</a>.</li>
<li>Use ternary operators and take advantage of Terraform built-in functions.</li>
<li>Test your modules.</li>
</ul>

The logic of Terraform modules may seem a bit complicated at first, but in the end, is a simple and efficient way of building your infrastructure quickly and reliably. If you want to step up your efforts, check out <a href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a> and the additional features it offers, which extend and perfect the modular approach.
Spacelift provides everything you need to make your module easily maintainable and usable. There is CI/CD for multiple specified versions of Terraform, which is capable of testing your module on each commit. You get an autogenerated page describing your Module and its intricacies, so your users can explore them and gather required information at a glimpse. It&#8217;s also deeply integrated with all the features <a href="https://docs.spacelift.io/concepts/stack/" target="_blank" rel="noopener">Stacks</a> use which you know and love, like <a href="https://docs.spacelift.io/concepts/configuration/environment" target="_blank" rel="noopener">Environments</a>, <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">Policies</a>, <a href="https://docs.spacelift.io/concepts/configuration/context" target="_blank" rel="noopener">Contexts</a>, and <a href="https://docs.spacelift.io/concepts/stack/stack-settings#worker-pool" target="_blank" rel="noopener">Worker Pools</a>. <a href="https://spacelift.io/free-trial" target="_blank" rel="noopener">Create a trial account and check it out for free</a>.
Cross-organization private module sharing and the above-mentioned module testing automation will help you get the most out of your configuration and make its development a lot smoother.
If you need more help with Terraform, I encourage you to check the following blog posts: <a href="https://spacelift.io/blog/terraform-automation" target="_blank" rel="noopener">How to Automate Terraform Deployments</a>, and <a href="https://spacelift.io/blog/terraform-best-practices" target="_blank" rel="noopener">12 Terraform Best Practices.</a>
Enjoy writing your code and growing your infrastructure, and have a productive day!

Spacelift helps manage Terraform state, build more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and many more.

Grab our ultimate cheat sheet PDF to master building reusable Terraform modules!

frame-8479

Frame 8479

How to Use Terraform Variables (Locals, Input, Output, Environment)

How Spacelift Can Improve Your Infrastructure as Code

Terraform Taint, Untaint, Replace - How to Use It (Examples)

Terraform modules are a way of extending your present Terraform configuration with already existing parts of reusable code, to reduce the amount of code you have to develop for similar infrastructure components. Others would say that the module definition is a single or many .tf files stacked together in their own directory. Both sides would be right.

Stanislaw Szymanski

Infrastructure Problems that Spacelift Solves

Get the newsletter

Recent posts

Terraform Functions, Expressions, Loops (Examples)

What Are Terraform Modules and How to Use Them: Tutorial