Ioannis is a Solutions Architect with a background in DevOps &amp; Site Reliability Engineering interested in Cloud Infrastructure, Automation, CI/CD Pipelines, and Containerization.

All posts by Ioannis Moustakis - Infrastructure as Code Expert

Ioannis Moustakis

Author title

Going to AWS Summit London? 🇬🇧🇬🇧

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

Announcing Enhanced VCS Integration  🎉 🎉

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

Ansible Modules: Basics & How to Use Them Efficiently

Ansible Guides and Tutorials | Spacelift Blog

Ansible

This article delves into Ansible modules, one of the core building blocks of Ansible. In this blog post, we will examine the purpose and usage of modules, along with information on how to build them and best practices. 
If you are interested in other Ansible concepts, <a href="https://spacelift.io/blog/ansible" target="_blank" rel="noopener">these Ansible tutorials</a> posted on Spacelift’s blog might be helpful for you.

Modules represent distinct units of code, each one with specific functionality. Basically, they are standalone scripts written for a particular job and are used in tasks as their main functional layer.
We build Ansible modules to abstract complexity and provide end-users with an easier way to execute their automation tasks without needing all the details. This way, some of the cognitive load of more complex tasks is abstracted away from Ansible users by leveraging the appropriate modules. 
Here’s an example of a task using the apt package manager module to install a specific version of Nginx.

Modules can be executed as well directly from the command line. Here’s an example of running the ping module against all the database hosts from the command line.

A well-designed module provides a predictable and well-defined interface that accepts arguments that make sense and are consistent with other modules. Modules take some arguments as input and return values in JSON format after execution. 
Ansible modules should follow <a href="https://docs.ansible.com/ansible/latest/reference_appendices/glossary.html#term-Idempotency" target="_blank" rel="nofollow noopener">idempotency principles</a>, which means that consecutive runs of the same module should have the same effect if nothing else changes. Well-designed modules detect if the current and desired state match and avoid making changes if that’s the case. 
We can utilize handlers to control the flow execution of modules and tasks in a <a href="https://spacelift.io/blog/ansible-playbooks" target="_blank" rel="noopener">playbook</a>. Modules can trigger additional downstream modules and tasks by notifying specific handlers.
As mentioned, modules return data structures in JSON data. We can store these return values in variables and use them in other tasks or display them to the console. Look at the <a href="https://docs.ansible.com/ansible/latest/reference_appendices/common_return_values.html" target="_blank" rel="nofollow noopener">common return values</a> for all modules to get an idea. 
For custom modules, the return values should be documented along with other useful information for the module. The command-line tool <code>ansible-doc</code> displays this information. 
Here’s an example output of running the ansible-doc command.

ansible-doc-ansible-modules

run ansible-doc

In the latest versions of Ansible, most modules are part of <a href="https://docs.ansible.com/ansible/latest/user_guide/collections_using.html" target="_blank" rel="nofollow noopener">collections</a>, a distribution format that includes roles, modules, plugins, and playbooks. Many of the core modules we use extensively are part of the <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/index.html#plugins-in-ansible-builtin" target="_blank" rel="nofollow noopener">Ansible.Builtin</a> collection. To find other available modules have a look at the <a href="https://docs.ansible.com/ansible/latest/collections/index.html#list-of-collections" target="_blank" rel="nofollow noopener">Collection docs</a>.

In this part, we explore some of the most used and helpful modules, and for each, we provide a working example. The modules in this list are picked based on their popularity within the Ansible community and functionality to perform everyday automation tasks.
<h3>Package Manager Modules yum &amp; apt</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.html" target="_blank" rel="nofollow noopener">apt module</a> is part of ansible-core and manages apt packages for Debian/Ubuntu Linux distributions. Here’s an example that updates the repository cache and updates the Nginx package to the latest version:

The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/yum_module.html" target="_blank" rel="nofollow noopener">yum module</a> is also part of ansible-core and manages packages with yum for RHEL/Centos/Fedora Linux distributions. Here’s the same example as above with the yum module:

<h3>Service Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/service_module.html" target="_blank" rel="nofollow noopener">service module</a> controls services on remote hosts and can leverage different init systems depending on their availability in a system. This module provides a nice abstraction layer for underlying service manager modules. Here’s an example of restarting the docker service:

<h3>File Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html" target="_blank" rel="nofollow noopener">file module</a> handles operations to files, symlinks, and directories. Here’s an example of using this module to create a directory with specific permissions:

<h3>Copy Module</h3>
The <a href="https://spacelift.io/blog/ansible-copy" target="_blank" rel="noopener">Ansible </a><a href="https://spacelift.io/blog/ansible-copy" target="_blank" rel="noopener">copy module</a> copies files to the remote machine and handles file transfers or moves within a remote system. Here’s an example of copying a file to the remote machine with permissions set:

<h3>Template Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html" target="_blank" rel="nofollow noopener">template module</a> assists us to template files out to target hosts by leveraging the <a href="https://jinja.palletsprojects.com/en/3.1.x/" target="_blank" rel="nofollow noopener">Jinja2 templating language</a>. Here’s an example of using a template file and some set <a href="https://spacelift.io/blog/ansible-variables" target="_blank" rel="noopener">Ansible variables</a> to generate an Nginx configuration file:

Read more about <a href="https://spacelift.io/blog/ansible-template" target="_blank" rel="noopener">how to create an Ansible template</a>.

<h3>Lineinfile &amp; Blockinfile Modules</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/lineinfile_module.html" target="_blank" rel="nofollow noopener">lineinfile module</a> adds, replaces, or ensures that a particular line exists in a file. It’s pretty common to use this module when we need to update a single line in configuration files.

The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/blockinfile_module.html" target="_blank" rel="nofollow noopener">blockinfile module</a> inserts, updates, or removes a block of lines from a file. It has the same functionality as the previous module but is used when you want to manipulate multi-line text blocks.

<h3>Cron Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/cron_module.html" target="_blank" rel="nofollow noopener">cron module</a> manages crontab entries and environment variables entries on remote hosts.

<h3>Wait_for Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_module.html" target="_blank" rel="nofollow noopener">wait_for module</a> provides a way to stop the execution of plays and wait for conditions, amount of time to pass, ports to become open, processes to finish, files to be available, strings to exist in files, etc.

<ul>
<li><a href="https://spacelift.io/blog/ansible-best-practices" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;44 Ansible Best Practices to Follow [Tips &amp; Tricks]&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">44 Ansible Best Practices to Follow</a></li>
<li><a href="https://spacelift.io/blog/spacelift-ansible-integration-beta" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to manage Ansible more securely&quot;}" data-sheets-userformat="{&quot;2&quot;:6723,&quot;3&quot;:{&quot;1&quot;:0},&quot;4&quot;:{&quot;1&quot;:2,&quot;2&quot;:16777215},&quot;9&quot;:0,&quot;12&quot;:0,&quot;14&quot;:{&quot;1&quot;:2,&quot;2&quot;:0},&quot;15&quot;:&quot;Arial&quot;}">How to Manage Ansible More Securely</a></li>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Common Infrastructure Challenges and How to Solve Them&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Common Infrastructure Challenges and How to Solve Them</a></li>
</ul>

<h3>Command &amp; Shell Modules</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/command_module.html" target="_blank" rel="nofollow noopener">command</a> and <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/shell_module.html" target="_blank" rel="nofollow noopener">shell</a> modules execute commands on remote nodes. Their main difference is that the command module bypasses the local shell, and consequently, variables like $HOSTNAME or $HOME aren’t available, and operations like “&lt;”, “&amp;” don’t work. If you need these features, you have to use the shell module. 
On the other hand, the remote local environment won’t affect the command module, so its outcome is considered more predictable and secure. 
Usually, it’s always preferred to use specialized Ansible modules to perform tasks instead of command and shell. There are cases, though, where you won’t be able to get the functionality that you need from specialized modules, and you will have to use one of these two. Use command and shell with care, and always try to check if there is a specialized module that can serve you better before relying on them.

<span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Added:Read more about the Ansible shell module.&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Read more about the <a href="https://spacelift.io/blog/ansible-shell-module" target="_blank" rel="noopener">Ansible shell module</a>.

For advanced users, there is always the option to develop custom modules if they have needs that can’t be satisfied by existing modules. Since modules always should return JSON data, they can be written in any programming language. 
Before jumping into module development, ensure that a similar module doesn&#8217;t exist to avoid unnecessary work. Αdditionally, you might be able to combine different modules to achieve the functionality you need. In this case, you might be able to replicate the behavior you want by creating a role that leverages other modules. Another option is to use <a href="https://docs.ansible.com/ansible/latest/plugins/plugins.html" target="_blank" rel="nofollow noopener">plugins</a> to enhance Ansible’s basic functionality with logic and new features accessible to all modules.
Next, we will go through an example of creating a custom module that takes as input a string that represents an epoch timestamp and converts it to its human-readable equivalent of type <a href="https://docs.python.org/3/library/datetime.html" target="_blank" rel="nofollow noopener">datetime</a> in Python. You can find the code for this tutorial <a href="https://github.com/spacelift-io-blog-posts/Blog-Technical-Content/tree/master/ansible-modules" target="_blank" rel="nofollow noopener">on this repository</a>.
First, let’s create a <code>library</code> directory on the top of our repository to place our custom module. Playbooks with a <code>./library</code> directory relative to their YAML file can add custom ansible modules that can be recognized in the ansible module path. This way, we can group custom modules and their related playbooks. 
We create our custom Python module <code>epoch_converter.py</code> inside the library directory. This simple module takes as input the argument <code>epoch_timestamp</code> and converts it to datetime type. We use another argument, <code>state_changed</code>, to simulate a change in the target system by this module.
library/epoch_converter.py

To test our module, let’s create a <code>test_custom_module.yml</code> playbook in the same directory as our <code>library</code> directory.
test_custom_module.yml

Last stop, let’s execute the playbook to test our custom module. Since we opted to set the <code>state_changed</code> argument, we expect the task state to appear as <code>changed</code> and displayed in yellow.

ansible-modules-example

ansible modules example

If you wish to contribute to an existing Ansible collection or create and publish a new one with your custom modules, look at <a href="https://docs.ansible.com/ansible/latest/dev_guide/developing_collections_distributing.html#publishing-your-collection" target="_blank" rel="nofollow noopener">Distributing collections</a> and <a href="https://docs.ansible.com/ansible/latest/community/index.html#ansible-community-guide" target="_blank" rel="nofollow noopener">Ansible Community Guide</a>, where you can find information on how to configure and distribute Ansible content.

Use specialized modules over shell or command: Although It might be tempting to use the <code>shell</code> or <code>command</code> module often, it’s considered a best practice to leverage more specific modules for each job. Specialized modules are typically recommended because they implement the concept of desired state and idempotency, have been tested, and fulfill basic standards, like error handling.
Specify arguments when it makes sense: Some module arguments have default values that can be omitted. To be more transparent and explicit, we can opt to specify some of these arguments like the <code>state</code> in our playbook definitions.
Prefer multi-tasks in a module over loops: The most efficient way of defining a list of similar tasks, like installing packages, is to use multiple tasks in a single module.

The above method should be preferred over the <code>loop</code> or defining multiple separate tasks using the same module.
Custom modules should be simple and tackle a specific job: If you decide to build your own module, focus on solving a particular problem. Each module should have a concise functionality, be as simple as possible, and perform one thing well. If what you try to achieve goes beyond the scope of a single module, consider <a href="https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_in_groups.html#developing-modules-in-groups" target="_blank" rel="nofollow noopener">developing a new collection</a>.
Custom modules should have predictable parameters: Try to enable others to use your module by defining a transparent and predictable user interface. The arguments should be well-scoped and understandable, and their structures should be as simple as possible. Follow the typical convention of parameter names in lowercase and use underscores as the word separator.
Document and test your custom modules: Every custom module should include examples, explicitly document dependencies, and describe return responses. New modules should be tested thoroughly before releasing. You can create roles and playbooks to test your custom modules and different test cases.

We encourage you also to explore how <a href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a>’s vibrant ecosystem and excellent GitOps flow can greatly assist you in managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can then easily <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">create custom workflows</a> based on pull requests and apply any necessary <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">compliance checks</a> for your organization. Another great advantage of using Spacelift is that you can manage different infrastructure tools like Ansible, Terraform, Pulumi, AWS CloudFormation, and even Kubernetes from the same place and combine their <a href="https://docs.spacelift.io/concepts/stack/" target="_blank" rel="noopener">Stacks</a> with building workflows across tools.
If you want to learn more about Spacelift working with Ansible, check our <a href="https://docs.spacelift.io/vendors/ansible/" target="_blank" rel="noopener">documentation</a>, read our <a href="https://spacelift.io/blog/spacelift-ansible-integration-beta" target="_blank" rel="noopener">Ansible guide</a>, or <a href="https://spacelift.io/schedule-demo" target="_blank" rel="noopener">book a demo with one of our engineers</a>.

We deep-dived into Ansible modules and examined their use and functionality in detail. We discussed best practices and showed practical examples of leveraging the most commonly-used modules. Lastly, we went through a complete example of developing a custom module. 
Thank you for reading, and I hope you enjoyed this article as much as I did.

Spacelift helps you manage the complexities and compliance challenges of using Ansible. It brings with it a GitOps flow, so your infrastructure repository is synced with your Ansible Stacks, and pull requests show you a preview of what they’re planning to change. It also has an extensive selection of policies, which lets you automate compliance checks and build complex multi-stack workflows.

16 DevOps Best Practices Every Developer Should Know

Ansible Tutorial for Beginners: Ultimate Playbook & Examples

Ansible Roles: Basics, Creating & Using

Modules are one of the core building blocks of Ansible. Learn how to build them and see best practices. Explore some of the most used and helpful modules.

81-ansible-modules

Ansible Modules - How To Use Them Efficiently (Examples)

Manage and orchestrate Ansible with Spacelift’s vibrant ecosystem and excellent GitOps flow. Create powerful custom workflows combining multiple IaC tools.

Ioannis is a Cloud Architect with a background in DevOps &amp; Site Reliability Engineering interested in Cloud Infrastructure, Automation, CI/CD Pipelines &amp; Containerization. In his free time, he curates a personal blog at <a href="https://devopsmadness.com" rel="noopener nofollow">devopsmadness.com</a> and <a href="https://moustakisioannis.medium.com" rel="noopener nofollow">moustakisioannis.medium.com</a>. Opinions expressed are solely his own.

DevOps Best Practices (And How Not To Do DevOps)

General | Spacelift Blog

General

Utilizing DevOps practices to maximize speed and value creation has been a hot topic in the software industry for the past decade. We have embraced these practices and changed how we work and think about development, operations, project management, code quality, observability, and continuous feedback. 
As organizations started applying these practices, we noticed many anti-patterns emerging. In this article, we will see some DevOps best practices and ways to improve our workflows, but also we will explore some of the typical DevOps anti-patterns and how to avoid them.
Some of the DevOps best practices are:
<ol>
<li><a href="https://spacelift.io/blog/devops-best-practices#1-foster-a-culture-of-collaboration-and-blameless-communication">Foster a culture of collaboration and blameless communication</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#2-adopt-continuous-integration-and-delivery-cicd">Adopt Continuous Integration and Delivery (CI/CD)</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#3-set-up-automated-testing">Set up automated testing</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#4-focus-on-observability-and-find-the-right-metrics">Focus on observability and find the right metrics</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#5-avoid-manual-work-with-automation">Avoid manual work with automation</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#6-incorporate-security-early-in-the-development-lifecycle">Incorporate security early in the development lifecycle</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#7-learn-from-incidents-and-build-processes-around-them">Learn from incidents and build processes around them</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#8-focus-first-on-concepts-then-find-the-right-tools">Focus first on concepts, then find the right tools</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#9-embrace-infrastructure-as-code-iac-and-push-for-a-selfservice-infra-model">Embrace Infrastructure as Code (IaC) and push for a self-service infra model</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#1-dont-create-separate-devops-team">Don’t create separate DevOps team</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#2-avoid-having-the-devops-hero">Avoid having the DevOps hero</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#3-dont-attempt-to-automate-and-change-everything-at-once">Don’t attempt to automate and change everything at once</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#4-avoid-chasing-new-tools">Avoid chasing new tools</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#5-dont-sacrifice-quality-for-speed">Don’t sacrifice quality for speed</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#6-dont-give-up-on-continuous-improvement">Don’t give up on Continuous Improvement</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices#7-dont-neglect-documentation-and-information-sharing">Don’t neglect documentation and information sharing</a></li>
</ol>

Probably you have seen hundreds of different definitions of DevOps by now. For me, DevOps is a set of best practices around the software development lifecycle and the effort to continually improve and deliver value more efficiently.  
On its basis, DevOps is a culture that spreads equally between developers and operations rather than a specific role. In reality, the term has been used as an umbrella term to characterize the engineering roles of cloud-savvy people who share the pains and responsibilities of devs and ops. At the same time, they strive to enable and promote DevOps practices within organizations. 
So why all this fuss? Implementing these practices has proven to improve software quality. Different software and operations teams collaborate more efficiently, reduce friction and lead time, integrate and test their code continuously, and deploy more often.
It&#8217;s all about finding the inefficiencies in our workflows and building a culture of continuous communication and trust. Other aspects of it are handling failures and unplanned work, leveraging automation, and focusing significantly on observability to get meaningful feedback.

Now that we set the foundation, let&#8217;s explore some DevOps best practices without further ado. The list isn&#8217;t supposed to be exhaustive but rather a guide with hints and pointers that will ease your journey towards adopting a healthy DevOps culture.

First and foremost, for this journey to be successful, we must focus intensely on growing a culture that allows people to collaborate freely and remove the fear of failure. Organizations and teams that promote values like trust and empathy tend to have a heads-up advantage in adopting DevOps practices. Break down the silos between teams and make them work together towards a common goal, bringing value to the company. One of the tools that deliver an <a href="https://spacelift.io" target="_blank" rel="noopener">enhanced collaboration layer for IaC</a><a href="https://spacelift.io/" target="_blank" rel="noopener"> </a>is Spacelift. At Spacelift, you can invite security and compliance teams to collaborate on and approve workflows and policies.

Integrating small batches of code frequently into a central code repository is a practice that allows developers to collaborate efficiently. With this approach, the repository is always kept in a good state since we introduce small changes that are easier to handle. Continuous Integration (CI) enables early error detection and improves code quality since these small batches of changes are validated each time with automated builds and tests. 
The next step after integrating our code is deploying it to our environment. Continuous Delivery (CD) is the practice of getting the code into a deployable state continuously for every small batch of change. This simplifies our deployments and provides our developers with an easy and automated method to push code to production.
Read more: <a href="https://spacelift.io/blog/devops-pipeline" target="_blank" rel="noopener">Building the DevOps Pipeline</a>.

A continuation of the previous point and an integral part of DevOps success is setting up and curating meaningful automated tests as part of our CI/CD pipelines. This way, we don&#8217;t rely on humans to run manual tests on our code; instead, we set up automated tests that run on every minor change introduced. 
By increasing the testing frequency and the number of tests, we reduce our chances of introducing bugs to production systems. The tests vary depending on the use case but typically could include unit testing, integration testing, end-to-end testing, load testing, smoke testing, etc.

DevOps practices are based on getting feedback and continuously improving our processes. We need to find and track the right metrics to achieve that and measure our results. Figuring out the right metrics is an arduous journey that each organization has to go through. 
These metrics will differ from organization to organization and from team to team, depending on the goals and key results targeted. Still, it&#8217;s a crucial exercise for achieving success. Some typical examples of DevOps metrics are deployment time, frequency of deploys, deployment failure rate, availability of critical services, mean time to detect, mean time to restore, cost per unit, code coverage, and change lead time. 
Moving one step forward, we also have to focus on the observability of our apps and software running in production. We have to define a strategy for effectively storing, managing, and distributing logs, traces, and metrics of our applications to quickly solve issues, improve the understandability of our systems and enable our teams to operate efficiently.

By reducing manual work and automating recurring tasks, we accelerate our processes and provide increased consistency to our results. Automation can allow us to focus on what is essential and avoid human intervention and time spent on chores. It also provides more confidence in our systems and processes, removes human errors and miscommunication, and speeds up the performance of teams. If you need an automation layer for your cloud resources, take a look at <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bdevops_best_practices%7D" target="_blank" rel="noopener">Spacelift&#8217;s self-service infrastructure</a> (automated workflow management feature especially).

Security shouldn&#8217;t be one of the last things to integrate into software development. The birth of <a href="https://spacelift.io/blog/what-is-devsecops" target="_blank" rel="noopener">DevSecOps</a> emphasizes thinking about application and infrastructure security early in the development lifecycle, incorporating security into the initial design and integrating it into the CI/CD pipelines. 
Learn more about <a href="https://spacelift.io/blog/devsecops-code-testing" target="_blank" rel="noopener">Automated and Manual Code Testing with DevSecOps</a>.
Security should be a responsibility shared among different teams and through the entire application lifecycle and be considered an integral part of the process, not an optional add-on. Recently, a strong focus has been given to securing the software supply chain due to increased malicious attacks over the last years.
In the world of infrastructure, even the tiniest of mistakes can cause major outages. That&#8217;s why Spacelift adds <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">an extra layer of </a><a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">policy</a><a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener"> that allows you to control</a> &#8211; separately from your infrastructure project &#8211; what code can be executed, what changes can be made, when and by whom. This isn&#8217;t only useful to protect yourself from the baddies but allows you to implement an automated code review pipeline<span data-key="3dc1c9be419548e2b421064b7e513810" data-slate-fragment="JTdCJTIyb2JqZWN0JTIyJTNBJTIyZG9jdW1lbnQlMjIlMkMlMjJkYXRhJTIyJTNBJTdCJTdEJTJDJTIybm9kZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJibG9jayUyMiUyQyUyMnR5cGUlMjIlM0ElMjJwYXJhZ3JhcGglMjIlMkMlMjJpc1ZvaWQlMjIlM0FmYWxzZSUyQyUyMmRhdGElMjIlM0ElN0IlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMmluJTIwdGhlJTIwd29ybGQlMjBvZiUyMGluZnJhc3RydWN0dXJlJTJDJTIwZXZlbiUyMHRoZSUyMHRpbmllc3QlMjBvZiUyMG1pc3Rha2VzJTIwY2FuJTIwY2F1c2UlMjBtYWpvciUyMG91dGFnZXMlMjAtJTIwbGlrZSUyMHRoYXQlMjB0eXBvJTIwd2UlMjBvbmNlJTIwbWFkZSUyMGluJTIwb3VyJTIwRE5TJTIwY29uZmlnLiUyMFRoYXQncyUyMHdoeSUyMFNwYWNlbGlmdCUyMGFkZHMlMjBhbiUyMGV4dHJhJTIwbGF5ZXIlMjBvZiUyMCUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMmFjZDUyMmQwMmEzODQzZWNhZTM2NTljNTY2NTE3Nzc2JTIyJTdEJTJDJTdCJTIyb2JqZWN0JTIyJTNBJTIyaW5saW5lJTIyJTJDJTIydHlwZSUyMiUzQSUyMmxpbmslMjIlMkMlMjJpc1ZvaWQlMjIlM0FmYWxzZSUyQyUyMmRhdGElMjIlM0ElN0IlMjJyZWYlMjIlM0ElN0IlMjJraW5kJTIyJTNBJTIycGFnZSUyMiUyQyUyMnBhZ2UlMjIlM0ElMjItTHhwdG9mZ080RXFWcUtzbXVpSSUyMiU3RCU3RCUyQyUyMm5vZGVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIydGV4dCUyMiUyQyUyMmxlYXZlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMmxlYWYlMjIlMkMlMjJ0ZXh0JTIyJTNBJTIycG9saWN5JTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyMjY1N2RhZDlmZWMyNGI5ODgxOWQ2MDFjYTZjYzI1MzElMjIlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjIwNWZlMzliYzE1NzQ0OTE0YjQ2Yzk2MmFkYjZkNzA0YiUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMiUyMHRoYXQlMjBhbGxvd3MlMjB5b3UlMjB0byUyMGNvbnRyb2wlMjAtJTIwc2VwYXJhdGVseSUyMGZyb20lMjB5b3VyJTIwaW5mcmFzdHJ1Y3R1cmUlMjBwcm9qZWN0ISUyMC0lMjAlMjIlMkMlMjJtYXJrcyUyMiUzQSU1QiU1RCUyQyUyMnNlbGVjdGlvbnMlMjIlM0ElNUIlNUQlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjJlZGIzYmEzZjdhMzA0MzRmYTBmNWEzZWRkMjViZmQyZCUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMmlubGluZSUyMiUyQyUyMnR5cGUlMjIlM0ElMjJsaW5rJTIyJTJDJTIyaXNWb2lkJTIyJTNBZmFsc2UlMkMlMjJkYXRhJTIyJTNBJTdCJTIycmVmJTIyJTNBJTdCJTIya2luZCUyMiUzQSUyMnBhZ2UlMjIlMkMlMjJwYWdlJTIyJTNBJTIyLUx4cHU2QjhOQ2FpbE1lOTYtamUlMjIlN0QlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMndoYXQlMjBjb2RlJTIwY2FuJTIwYmUlMjBleGVjdXRlZCUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjZiM2Q2MjRhZmI4NTRkMDI4Y2EzMDRkMjQ0OGMyYTdlJTIyJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyODliYmFjYTcwZTAyNGI3ZGJiNWY2ZGNmMDIzMjM3MDclMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJ0ZXh0JTIyJTJDJTIybGVhdmVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIybGVhZiUyMiUyQyUyMnRleHQlMjIlM0ElMjIlMkMlMjAlMjIlMkMlMjJtYXJrcyUyMiUzQSU1QiU1RCUyQyUyMnNlbGVjdGlvbnMlMjIlM0ElNUIlNUQlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjIwOGU5MWYxOTI1ZjQ0MGI5YjdhNzgwZjlkYzhmMzY1YiUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMmlubGluZSUyMiUyQyUyMnR5cGUlMjIlM0ElMjJsaW5rJTIyJTJDJTIyaXNWb2lkJTIyJTNBZmFsc2UlMkMlMjJkYXRhJTIyJTNBJTdCJTIycmVmJTIyJTNBJTdCJTIya2luZCUyMiUzQSUyMnBhZ2UlMjIlMkMlMjJwYWdlJTIyJTNBJTIyLUx4cHU4VHAxQkZhSjUtbE5CYXAlMjIlN0QlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMndoYXQlMjBjaGFuZ2VzJTIwY2FuJTIwYmUlMjBtYWRlJTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyODk2ODAxMzIwYmYzNDBiMDkyYjllZGQwZmM3NGNkYTclMjIlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjJlMDE5ZDE4ZmI5ZWI0NzYzOTM1YjliOTczMDZjNzgwYSUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMiUyQyUyMCUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjYyYzNmOGNhN2MwYzRiMWE5MjU0NWEyODQ5MGNjOTQ2JTIyJTdEJTJDJTdCJTIyb2JqZWN0JTIyJTNBJTIyaW5saW5lJTIyJTJDJTIydHlwZSUyMiUzQSUyMmxpbmslMjIlMkMlMjJpc1ZvaWQlMjIlM0FmYWxzZSUyQyUyMmRhdGElMjIlM0ElN0IlMjJyZWYlMjIlM0ElN0IlMjJraW5kJTIyJTNBJTIycGFnZSUyMiUyQyUyMnBhZ2UlMjIlM0ElMjItTHhwdHhHbTRrNnFRa0g1NTduSiUyMiU3RCU3RCUyQyUyMm5vZGVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIydGV4dCUyMiUyQyUyMmxlYXZlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMmxlYWYlMjIlMkMlMjJ0ZXh0JTIyJTNBJTIyd2hlbiUyMGFuZCUyMGJ5JTIwd2hvbSUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjVjY2JkZDBkZTdjNDRiODNiMTYwZTNkYzU3MWVlZGU4JTIyJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyMDk5NDFlZTlhMzk0NDZjNDk2ODc4YTQ3NGM2MmU2YTElMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJ0ZXh0JTIyJTJDJTIybGVhdmVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIybGVhZiUyMiUyQyUyMnRleHQlMjIlM0ElMjIuJTIwVGhpcyUyMGlzbid0JTIwb25seSUyMHVzZWZ1bCUyMHRvJTIwcHJvdGVjdCUyMHlvdXJzZWxmJTIwZnJvbSUyMHRoZSUyMGJhZGRpZXMlMkMlMjBidXQlMjBhbGxvd3MlMjB5b3UlMjB0byUyMGltcGxlbWVudCUyMGFuJTIwJTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyZWUyZTRmNzVhNjRhNDk5NDhiZmIxN2JkMjU1MmUyYmUlMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJpbmxpbmUlMjIlMkMlMjJ0eXBlJTIyJTNBJTIybGluayUyMiUyQyUyMmlzVm9pZCUyMiUzQWZhbHNlJTJDJTIyZGF0YSUyMiUzQSU3QiUyMnJlZiUyMiUzQSU3QiUyMmtpbmQlMjIlM0ElMjJhbmNob3IlMjIlMkMlMjJhbmNob3IlMjIlM0ElMjJhdXRvbWF0ZWQtY29kZS1yZXZpZXclMjIlMkMlMjJwYWdlJTIyJTNBJTIyLUx4cHU4VHAxQkZhSjUtbE5CYXAlMjIlN0QlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMmF1dG9tYXRlZCUyMGNvZGUlMjByZXZpZXclMjBwaXBlbGluZSUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjRkNWQ2ZDhmYzZhNzRmZjY4ZmFjYTc2ZmQ2YTc0MGNhJTIyJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyYzM5MTNjMzkwZDNjNGNkOWJkZWM0NzVjZDMwNzNjYTQlMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJ0ZXh0JTIyJTJDJTIybGVhdmVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIybGVhZiUyMiUyQyUyMnRleHQlMjIlM0ElMjIuJTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyM2RjMWM5YmU0MTk1NDhlMmI0MjEwNjRiN2U1MTM4MTAlMjIlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjJkZmM2OTFkMTU3NDQ0NGQwOTIxMjgyMzY3MjYwYTFjZCUyMiU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjYzMjdmOWYxOWE0YjQwNTA5ZDE0NmUxMDVlMTg5YzcwJTIyJTdE">.

Incidents are inevitable in the IT world. It doesn&#8217;t matter how well prepared your team is; eventually, there will be an incident that you will have to address. In these cases, it&#8217;s essential to focus on blameless communication, understand the issue, communicate effectively with the affected parties, and collaborate to find a solution. 
Equally crucial to fixing the issue is to have a process to log incidents and learn from them. After the incident has been tackled, spend some time with your team to craft a post-incident review, and discuss how the incident was handled. Try to find any possible improvements in the incident handling process that can help you next time.

The DevOps landscape is moving extremely fast, with new tools and services emerging daily. Instead of continuously integrating new shiny tools and services, concentrate on understanding the core concepts that allow companies to accelerate their business with DevOps practices. 
Only once you understand the concepts and prioritize the missing pieces accordingly, you will be successful in selecting the right tools for the job. Remember, you won&#8217;t be able to build everything within your team. It&#8217;s ok to rely on <a href="https://spacelift.io/blog/devops-tools" target="_blank" rel="noopener">DevOps tooling</a> and managed services when it makes sense. Be smart about how you use your team&#8217;s time, try to understand your in-house expertise and needs, put effort into building custom tooling when it makes sense, and rely on external tooling and services for the rest.

Cloud infrastructure should be considered an integral part of software development and treated equally to application code. By leveraging <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">infrastructure as code</a>, we can incorporate the best practices we use for software development, such as version control and CI/CD, into infrastructure creation. This model removes the need to manually set up and configure resources via UIs and further strengthens our automation efforts across the IT landscape. Changes are always auditable and transparent, and we can quickly roll back infrastructure systems to a previous state when there are issues. 
Thinking one step ahead, instead of adding another bottleneck of waiting for cloud infrastructure engineers to create the necessary resources, push for a self-service infrastructure model. In this model, the developers and anyone who needs infra resources can leverage some tooling to generate the required pieces. This way, we increase productivity and speed while giving autonomy to our developers, all via a single workflow. 
Check out <a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener">how </a><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener">Spacelift</a><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener"> can improve your IaC</a>, assist your team in adopting a collaborative infrastructure model and achieving operational excellence.

<ul>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">Top Most Useful CI/CD Tools for DevOps</a></li>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Common Infrastructure Challenges and How to Solve Them&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Common Infrastructure Challenges and How to Solve Them</a></li>
<li><a href="https://spacelift.io/blog/5-ways-to-manage-terraform-at-scale" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;5 Ways to Manage Terraform at Scale&quot;}" data-sheets-userformat="{&quot;2&quot;:6723,&quot;3&quot;:{&quot;1&quot;:0},&quot;4&quot;:{&quot;1&quot;:2,&quot;2&quot;:16777215},&quot;9&quot;:0,&quot;12&quot;:0,&quot;14&quot;:{&quot;1&quot;:2,&quot;2&quot;:0},&quot;15&quot;:&quot;Arial&quot;}">5 Ways to Manage Terraform at Scale</a></li>
</ul>

Along with the rise of DevOps, we have also seen many anti-patterns emerge. During the quest to adopt DevOps practices, people have misinterpreted their scope and made mistakes that lead to common anti-patterns. Let&#8217;s look at some common challenges, traps, and misconceptions companies face while implementing DevOps principles.

A common mistake companies make when adopting DevOps practices is creating a separate team to handle the DevOps transformation. Unfortunately, this adds one more silo to the process and breaks the central promise of DevOps, which is increased collaboration and shared ownership between the existing teams. 
Similarly, we see operations teams rebranding into DevOps teams without actual changes in organizations&#8217; culture, communication, and collaboration. DevOps is about bringing the different groups closer, not creating a new one.

At times, specific team members are more involved in the DevOps practices than others. That could be due to accumulated knowledge, a higher level of experience, or increased effort by a person. When this pattern emerges, it could lead quickly to the DevOps hero anti-pattern where a specific team member becomes indispensable to the team. 
This situation is highly problematic since the team&#8217;s performance and velocity depend on a single person. At the same time, this person may face an extremely high amount of work that eventually leads to burnout and potentially leaving the company. To avoid this anti-pattern, ensure the knowledge is spread across teams and team members. Divide the work equally and don&#8217;t rely on heroes but on teamwork and rigid processes to achieve results.

Starting from scratch to apply DevOps practices in an organization could be daunting at first. As with most things, attempting to tackle everything at once is not the way to go. First, analyze the current situation and processes within your company. People usually don&#8217;t happily accept many changes, so you need to think strategically. Prioritize the tasks accordingly, find quick wins, automate the stuff that will have a higher impact, and focus on one thing at a time.

As new services and tools pop up almost every day, adopting and using these new shiny toys is always tempting. It&#8217;s common for engineers to fall into this trap of introducing a new tool just because it&#8217;s trending without proper analysis of whether it&#8217;s needed or the best choice. 
Picking the right tools for the job is critical but is a process that should be reviewed meticulously. For every new service or tool we add, we should also consider its maintainability and the operational overhead, dependencies, complexity, and new cognitive load that we introduce in the process.

Since one of the main factors of DevOps success is velocity, many teams try to speed up their processes at the cost of quality and usually security. Many of the typical DevOps metrics are based on how fast we deliver, deploy and provide value, but they are not enough by themselves as they only tell half the story. Due to this disproportionate focus on speed, it&#8217;s easy to lose perspective of what is important; delivering quality software. Treat speed and quality equally, add meaningful automated tests and avoid cutting corners just to ship faster.

Applying effective DevOps practices is a dynamic process that should be curated continuously. It might be tempting to rest and relax after implementing all the DevOps best practices in the roadmap, but unfortunately, this process never stops. 
Every step of the way, we should focus on reviewing our workflows and continuously improving our systems, processes, and products. We have to set up flows of constant feedback that allow us to review and reflect on our choices and ultimately improve. New paradigms, best practices, and improved models always appear, and we should be restless if we want our teams to survive, perform, and succeed.

By definition, the successful adoption of DevOps practices relies on sharing information efficiently within an organization and creating a workplace where collaboration thrives organically. Unfortunately, neglecting documentation and efficient information sharing is an anti-pattern that occurs too often in software teams. Documentation, when done right, could be a handy tool for developers. 
Try to integrate documentation tasks on team backlogs and treat docs as first-class citizens within your organization. Docs aren&#8217;t static and should be kept up to date, created consistently, and accessible to anyone who needs them.

We have explored different DevOps best practices and paradigms and analyzed how we can incorporate them to accelerate team performance and value creation. We also saw some hidden traps and anti-patterns to be aware of and avoid while pursuing DevOps excellence. 
Thank you for reading, and I hope you enjoyed this article as much as I did.
And don&#8217;t forget to check out how to automate, audit, secure, and continuously deliver your infrastructure with <a href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a>.

<span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures.&quot;}" data-sheets-userformat="{&quot;2&quot;:769,&quot;3&quot;:{&quot;1&quot;:0},&quot;11&quot;:4,&quot;12&quot;:0}">Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures.

Transform your IaC management to scale securely, efficiently, and productively into the future.

ebook-sidebanner

ebook bottom overlay

How to Deploy Infrastructure in CI/CD Using Terraform (Pipeline)

How Spacelift Can Improve Your Infrastructure as Code

20 Terraform Best Practices to Improve your TF workflow

Explore different DevOps Best Practices and Anti-Patterns and see how you can incorporate them to accelerate your team performance and value creation.

43-devops-best-practices

AWS Guides and Tutorials | Spacelift Blog

Securely accessing cloud resources is one of the most critical requirements for IT teams. To achieve this, cloud administrators leverage web services that assist with managing access to cloud environments. 
In this article, we will explore the AWS IAM service, and more specifically, we will look into IAM policies, learn how they are structured, how to create them, and some best practices.

AWS Identity and Access Management(IAM) is a web service that assists with managing and controlling access and permissions to resources and other AWS services. Leveraging this service, we can set up the building blocks to control authentication and authorization to our AWS accounts. 
Before we continue talking about IAM, let’s define some terminology for its essential components that we will reference throughout the article. 
<ul>
<li style="font-weight: 400;" aria-level="1">Authentication: The process of validating a user’s identity. Basically, verifying who someone claims to be. </li>
<li style="font-weight: 400;" aria-level="1">Authorization: Defining the level of permissions and access rights for a given user. Authorization happens after authentication and establishes what someone is allowed to do.</li>
<li style="font-weight: 400;" aria-level="1">Principals: Persons or applications that try to perform actions on AWS. </li>
<li style="font-weight: 400;" aria-level="1">Users: Identities in IAM that correspond to users in an organization. They represent actual persons or applications and service accounts.</li>
<li style="font-weight: 400;" aria-level="1">Groups: IAM users are grouped logically with IAM groups. This way, we can assign permissions to multiple users in a bulk fashion. A common pattern is to create groups according to different roles in an organization.</li>
<li style="font-weight: 400;" aria-level="1">Roles: Roles provide temporary access to resources and aren’t associated with specific users. Instead, roles enable principals to temporarily assume a set of permissions to complete an operation.</li>
<li style="font-weight: 400;" aria-level="1">Policies: To manage access on AWS we generate IAM policies that define levels of permissions and attach them to IAM identities(users, groups, roles) or AWS resources.</li>
<li style="font-weight: 400;" aria-level="1">Requests: Principals attempt to perform actions on AWS by instantiating requests to AWS via the AWS API, Management Console, or CLI. </li>
<li style="font-weight: 400;" aria-level="1">Actions: Every request has an action definition that declares the specific operation requested. If authentication and authorization are cleared the action is approved. </li>
<li style="font-weight: 400;" aria-level="1">Resources: A resource in this context can be considered any AWS object within a service.</li>
</ul>

IAM Policies are one of the most basic blocks of access management in AWS since they define the permissions of an identity or a resource. For every request, these policies are evaluated, and based on their definition; the requests are allowed or denied. Let’s look at the different types of policies that exist in AWS.  
<ul>
<li style="font-weight: 400;" aria-level="1">Identity-based policies are policies attached to identities(users, groups, roles) and provide them with the required permissions. Identity-based policies could be either Managed policies or Inline policies. Managed policies are either prepared and managed by AWS for common use cases(AWS managed policies) or custom policies created by users(Customer managed policies) suitable for achieving fine-grained control. Inline policies are used when we need to make a policy part of a principal’s entity and maintain a strict one-to-one relationship between them. </li>
<li style="font-weight: 400;" aria-level="1">Resource-based policies are attached directly to resources and specify permissions for specific actions on the resource by some principals. </li>
<li style="font-weight: 400;" aria-level="1">IAM permissions boundaries define the maximum permissions for an IAM entity and are used as safeguards. </li>
<li style="font-weight: 400;" aria-level="1">Access control lists(ACLs) are attached to resources and control cross-account permissions for principals from other accounts.</li>
<li style="font-weight: 400;" aria-level="1">Organizations Service Control Policies(SCPs) specify the maximum level of permissions for an organization&#8217;s accounts. These policies are used to limit the permissions that can be assigned within member accounts.</li>
<li style="font-weight: 400;" aria-level="1">Session policies are advanced policies used during temporary sessions for roles or federated users.</li>
</ul>

Now that we have seen the different types of IAM policies that exist in the context of AWS, let&#8217;s see how they are structured. Most of them are stored in JSON format and are attached to resources or identities. The only type of policy that uses a different format is ACL, but we won’t focus on ACLs in this article. 
Each JSON policy document might have optional informational elements at the top of the document and must have one or more statements. A statement contains all the necessary information about a permission that we would like to declare. 
Here’s a simple identity-based policy that allows the principal who has it attached to list the objects of a single S3 bucket named `this-is-an-example-s3-bucket-name`.

IAM Policies are built using a combination of the below elements:
<ul>
<li>Version: Defines the version of the policy language. Always use the latest version.</li>
<li>Statement: This argument is used as a parent element for the different statements in the policy. </li>
<li>Sid: This is an optional element that allows us to define a statement ID.</li>
<li>Effect: This element can have the values `Allow` or `Deny`.</li>
<li>Action: The list of actions related to the policy.</li>
<li>Resource: Defines the list of resources to which the policy is applied. For resource-based policies, this is optional since the policy applies to the resource that has it attached. </li>
<li>Principal: Defines the identities that are allowed or denied access to resource-based policies. </li>
<li>Condition: Defines some conditions under which the policy applies. This element is practical when we need to achieve custom rules for fine-grained access. </li>
</ul>
There are options for resources, principals, and actions to define deny access controls by using NotPrincipal, NotAction, or NotResource. Note that these are mutually exclusive with their contrasting elements. For example, you can’t use both Resource and NotResource elements in a statement. 

If you want to learn more about these elements and the JSON policy components, check out the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html" target="_blank" rel="nofollow noopener">IAM JSON policy reference</a>.
Next, we will see an example of a more complex IAM policy with more than one statement using conditions.

In the policy above, we define two separate statements. The first defines some rules to deny access based on the requested AWS region. More specifically, it denies all actions for regions not defined in the condition, except for the actions mentioned in the NotAction element.
The second one defines a policy to deny access to AWS based on the source IP of the requests. Here, since we have two conditions, they are combined with a logical `AND`. More specifically, the policy denies all AWS actions when requests originate from an IP that doesn’t come from the corporate IP range(in this example: 192.0.1.0/24) and when an AWS service isn’t making the call. 
Note that both policies don’t actually allow any actions and are used to restrict access to AWS resources. 
Finally, here’s a resource-based policy for an S3 bucket.

By associating this policy with the bucket `this_is_an_example_s3_bucket_name` every action on the bucket is denied if the request isn’t authenticated with a multi-factor authentication mechanism. Note the use of the Principal element here since we are using a resource-based policy.

<ul>
<li><a href="https://spacelift.io/blog/introducing-spacelift-self-hosted" target="_blank" rel="noopener">SaaS not an Option? Install Spacelift Self-Hosted on AWS</a></li>
<li><a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Deploy your Infrastructure in CI/CD using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Deploy your Infrastructure in CI/CD using Terraform</a></li>
<li><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;10 Ways to Improve Your Infrastructure as Code&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">10 Ways to Improve Your Infrastructure as Code</a></li>
</ul>

There are three main ways to create an IAM policy. We can either use the AWS console, the AWS CLI, or the AWS API. For this demo, we will go through the exercise of creating an IAM customer-managed policy via the AWS Console. 
The easiest option to create a policy is to use the helpful visual editor that the AWS console provides without having to write a JSON file and care about proper syntax. After signing in to the AWS Management Console, head to <a href="https://console.aws.amazon.com/iam/" target="_blank" rel="nofollow noopener">IAM</a> and select Policies and Create Policy. 
From this screen, you can choose to either use the Visual editor or JSON.

image9-8

create policy - choose a service

Let&#8217;s replicate our first example policy from above that allows listing the objects in an S3 bucket. For Service we select S3, for Actions choose ListBucket, and for Resources use the arn of the S3 bucket arn:aws:s3:::this-is-an-example-s3-bucket-name. Then, select Next: Tags.

image7-11

Create policy - s3

Note that by default, the policy allows the actions chosen. To deny actions, you have to select Switch to deny permissions before selecting the actions.

image1-26

create policy - switch to deny premissions

You can optionally add some tags to your customer-managed identity-based policy on the next page. For example, we could add tags for name, environment, department, etc. Then, select Next: Review.

image2-25

create policy - add tags

Finally, add a Name to your policy and optionally a description for its purpose and select Create policy.

image5-18

Create policy - review policy

Nice,  we have replicated the IAM policy we introduced in the first example.
If you feel comfortable with JSON syntax, you can instead use the JSON tab to define an IAM Policy like this:

image10-8

Create policy - JSON

Another option using the AWS API under the hood would be to create your IAM resources with the Infrastructure as Code tool of your choice. Here’s an example of using Terraform to define the same policy.

When creating or editing IAM policies using the AWS Management Console, their syntax and grammar are inspected to verify they comply with the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_grammar.html" target="_blank" rel="nofollow noopener">IAM policy grammar</a>. If there is any issue or error, you get notified accordingly and have to fix the policy. 
AWS provides <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html" target="_blank" rel="nofollow noopener">IAM Access Analyzer</a> with additional policy checks and recommendations to improve policies. When creating or editing a policy using the JSON tab, a policy validation pane below the policy provides different findings for the policy. There you get information for issues categorized as Security, Errors, Warnings, and Suggestions. Update your policy accordingly to resolve the findings.
Here’s an example of a malformed policy that triggers three separate findings.

image6-12

Malformed policy example

We get an Error finding because we don’t specify a correct ARN field for the Resource element.
Next, on the Warnings tab, we get notified that we omitted the top-level Version element.

image3-22

Validating policy - Warnings tab

Finally, on the Suggestions tab, we get notified that our Action element includes no actions.

image8-11

Validating policy - Suggestions tab

Another handy tool for AWS administrators and users who manage IAM policies is the <a href="https://policysim.aws.amazon.com/home/index.jsp?#" target="_blank" rel="nofollow noopener">IAM Policy Simulator</a>. Using this simulator, you can troubleshoot issues with different policy types and try to understand why some requests are allowed or denied. 
The IAM Policy Simulator console provides a testing playground for IAM policies and an easy way to test which actions are allowed or denied to specific principals for specific resources. The simulator doesn&#8217;t actually make the requests, so it&#8217;s a safe space to experiment and get information. You can even test new policies that don&#8217;t yet exist on our account and simulate real-world scenarios by defining complex conditions. Finally, since the returned output is a message with the result and relevant information, you can identify which statement in a policy denies or allows access. 
For example, we have created an IAM user test-user and attached the policy ListBucketContents that we created earlier. We can use the AWS Policy Simulator to validate that this user can indeed list the objects of the example bucket.

image4-20

AWS Policy Simulator

After defining all the parameters for the simulation, we choose Run Simulation and check that the Permission result is allowed. Check out the detailed <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_policies.html" target="_blank" rel="nofollow noopener">Troubleshooting IAM Policies guide</a> for issues with IAM policies.

A valuable feature of IAM Policies that you should be aware of is that AWS automatically keeps five versions of each policy. Whenever you make changes to a policy and save it, a new version is created instead of overwriting the existing one. 
When the limit of five kept versions is reached, you can select which older version to remove. This feature is convenient because it allows us to quickly revert to a previous policy version in case of issues.

<ul>
<li>Follow Least Privilege Principles: When creating IAM policies grant only the necessary permissions to perform the job. Specify specific actions, resources, and principles and add custom conditions to achieve the required controls. </li>
<li>Generate policies based on access activity: Although it&#8217;s considered a best practice, it might be challenging to implement fine-grained policies that grant the least privilege when starting with IAM policies. For this reason, you can initially use policies that allow more permissions than they should and then refine them by <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_generate-policy.html" target="_blank" rel="nofollow noopener">generating a new policy based on the access activity</a> of an IAM entity. This is a more pragmatic approach to achieving the least privilege for inexperienced users that makes the journey to improving security and access management smoother. </li>
<li>Get started with managed policies: It’s totally ok to start with AWS Managed policies as you start experimenting and learning. These policies cover common use cases and help your team set up necessary permissions to start quickly. When you feel comfortable enough with IAM, consider restricting your policies by creating customer-managed policies that follow the least privileged approach.</li>
<li>Assign permissions with roles and groups &amp; avoid inline policies: It’s considered best to avoid assigning permissions directly to users or using inline policies. For easier access management and better security controls, attach policies to groups or roles. </li>
<li>Validate policies: Every time you create or edit policies, validate them using the AWS helper tools, as we have seen in the section Validating IAM Policies.</li>
<li>Use IAM Roles for EC2 instances(EC2 Instance Profiles): For apps that run on EC2 instances, attach the necessary permissions to IAM roles and specify the role as a launch parameter for the instance. </li>
<li>Use policy conditions: Leverage policy conditions to achieve complex policies with custom rules. By combining different conditions in policies, we can comply with specific requirements according to our organization’s standards. Check out the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition.html" target="_blank" rel="nofollow noopener">Policy Elements: Condition</a> reference page for more information.</li>
<li>Test IAM policies with IAM Policy Simulator: Test existing and new policies quickly and without affecting any environment with the IAM Policy Simulator. Use it to understand your policies better and troubleshoot different issues.</li>
<li>Review IAM policies on a regular basis: Cloud environments aren’t static; they evolve over time. For this reason, IAM policies should be reviewed and updated to reflect changes. Following best practices and least privilege, it’s not something that you configure once and forget but requires continuous effort. </li>
<li>Tag IAM policies: Tag your IAM policies to add metadata to them the same way you tag other AWS resources. </li>
<li>Control access to resources with tags: If you have a proper tagging strategy for your AWS resources, you can create elaborate IAM policies that control access to them based on tags. This method becomes handy when you want to separate access to resources based on owners or teams.</li>
<li>Use IaC to create your IAM policies: In this tutorial, we have seen how to create IAM policies from the AWS Console, and this might be ok as a starting point, but eventually, you should treat your IAM resources as infrastructure components and apply the same Infrastructure as Code principles as any other AWS resource.</li>
</ul>
Check out <a href="https://spacelift.io/blog/aws-iam-best-practices" target="_blank" rel="noopener">AWS IAM security best practices</a> to learn more.

We have explored IAM concepts in the context of AWS, and more specifically, we focused on IAM policies. We saw how to create and test policies, assign necessary permissions, and combine different IAM mechanisms to achieve efficient access management controls on AWS.
Here you can <a href="https://spacelift.io/blog/introducing-spacelift-self-hosted" target="_blank" rel="noopener">learn more about self-hosting Spacelift in AWS</a>, to ensure your organization’s compliance, control ingress, egress, internal traffic, and certificates, and have the flexibility to run it within GovCloud.
You can also <a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="noopener">see Spacelift integration with AWS</a>, with our Cloud Integrations section and our update to support account-level AWS integrations.
Thank you for reading, and I hope you enjoyed this article as much as I did.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence.

CI/CD Pipeline : Everything You Need To Know

Infrastructure Drift Detection and How to Fix It With IaC Tools

Look into AWS IAM policies with some best practices. Learn how they are structured, how to create them, and how to assign necessary permissions.

80-iam-policies

AWS IAM Policies and Best Practices

AWS IAM Policies : Creating an IAM Policy & Best Practices

This blog post explores the concept of Ansible roles, their structure, and how we can combine them with our playbooks. We will analyze their functionality and usage along with ways to create new roles and retrieve public shared roles from Ansible Galaxy, a public repository for Ansible resources.
If you are new to Ansible, you might also find these tutorials helpful <a href="https://spacelift.io/blog/ansible-tutorial" target="_blank" rel="noopener">Ansible Tutorial for Beginners</a>, <a href="https://spacelift.io/blog/ansible-playbooks" target="_blank" rel="noopener">Working with Ansible Playbooks</a>, and <a href="https://spacelift.io/blog/ansible-variables" target="_blank" rel="noopener">How to Use Different Types of Ansible Variables</a>.

Ansible Roles provide a well-defined framework and structure for setting your tasks, variables, handlers, metadata, templates, and other files. They enable us to reuse and share our Ansible code efficiently. This way, we can reference and call them in our playbooks with just a few lines of code while we can reuse the same roles over many projects without the need to duplicate our code.

When starting with Ansible, it’s pretty common to focus on writing playbooks to automate repeating tasks quickly. As new users automate more and more tasks with playbooks and their Ansible skills mature, they reach a point where using just Ansible playbooks is limiting. Ansible Roles to the rescue!
Since we have our code grouped and structured according to the Ansible standards, it is quite straightforward to share it with others. We will see an example of how we can accomplish that later with Ansible Galaxy. 
Organizing our Ansible content into roles provides us with a structure that is more manageable than just using playbooks. This might not be evident in minimal projects but as the number of playbooks grows, so does the complexity of our projects. 
Lastly, placing our Ansible code into roles lets us organize our automation projects into logical groupings and follow the separation of concerns design principle. Collaboration and velocity are also improved since different users can work on separate roles in parallel without modifying the same playbooks simultaneously.

Let’s have a look at the standard role directory structure. For each role, we define a directory with the same name. Inside, files are grouped into subdirectories according to their function. A role must include at least one of these standard directories and can omit any that isn’t actively used.
To assist us with quickly creating a well-defined role directory structure skeleton, we can leverage the command ansible-galaxy init &lt;your_role_name&gt;. The ansible-galaxy command comes bundled with Ansible, so there is no need to install extra packages.
Create a skeleton structure for a role named test_role:

image4-18

test_role

This command generates this directory structure:

image5-16

Directory structure

Ansible checks for main.yml files, possible variations, and relevant content in each subdirectory. It’s possible to include additional YAML files in some directories. For instance, you can group your tasks in separate YAML files according to some characteristic.
<ul>
<li aria-level="1">defaults &#8211;  Includes default values for variables of the role. Here we define some sane default variables, but they have the lowest priority and are usually overridden by other methods to customize the role.</li>
<li aria-level="1">files  &#8211; Contains static and custom files that the role uses to perform various tasks.</li>
<li aria-level="1">handlers &#8211; A set of <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_handlers.html" target="_blank" rel="nofollow noopener">handlers</a> that are triggered by tasks of the role. </li>
<li aria-level="1">meta &#8211; Includes metadata information for the role, its dependencies, the author, license, available platform, etc.</li>
<li aria-level="1">tasks &#8211; A list of tasks to be executed by the role. This part could be considered similar to the task section of a playbook.</li>
<li aria-level="1">templates &#8211; Contains <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_templating.html" target="_blank" rel="nofollow noopener">Jinja2</a> template files used by tasks of the role. (<span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Read more about how to create an Ansible template.&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Read more about <a href="https://spacelift.io/blog/ansible-template" target="_blank" rel="noopener">how to create an Ansible template</a>.)</li>
<li aria-level="1">tests &#8211; Includes configuration files related to role testing.</li>
<li aria-level="1">vars &#8211; Contains variables defined for the role. These have quite a high <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_templating.html" target="_blank" rel="nofollow noopener">precedence</a> in Ansible.</li>
</ul>
Another directory that wasn’t automatically generated by the ansible-galaxy init command but is mentioned in the official Ansible docs, and you might find helpful in some cases, is the library directory. Inside it, we define any custom modules and plugins that we have written and used by the role. Finally, we also have a preconfigured README.md file that we can fill with details and useful information about our role.

<ul>
<li><a href="https://spacelift.io/blog/using-terraform-and-ansible-together" target="_blank" rel="noopener">Using Terraform &amp; Ansible together</a></li>
<li><a href="https://spacelift.io/blog/spacelift-ansible-integration-beta" target="_blank" rel="noopener">How to better manage Ansible and overcome challenges</a></li>
<li><a href="https://spacelift.io/blog/ansible-best-practices" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;44 Ansible Best Practices to Follow [Tips &amp; Tricks]&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">44 Ansible best practices to follow</a></li>
</ul>

A common tactic is to refactor an Ansible playbook into a role. To achieve that, we have to decompose the different parts of a playbook and stitch them together into an Ansible role using the directories we’ve just seen in the previous section. 
This section will go through an example of creating a new role for installing and configuring a minimal Nginx web server from scratch. If you wish to follow along, you will need <a href="https://www.virtualbox.org/" target="_blank" rel="nofollow noopener">VirtualBox</a>, <a href="https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html" target="_blank" rel="nofollow noopener">Ansible</a>, and <a href="https://www.vagrantup.com/" target="_blank" rel="nofollow noopener">Vagrant</a> installed locally.
Ansible searches for referenced roles in common paths like the orchestrating playbook’s directory, the roles/ directory, or the configured <a href="https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-roles-path" target="_blank" rel="nofollow noopener">roles_path</a> configuration value. It’s also possible to set a custom path when referencing a role:

Using the ansible-galaxy init command, we generate the initial directory structure for a role named webserver inside a parent directory named roles. Let’s go ahead and delete the tests directory since we won’t be using it. We will see how to utilize all the other directories during our demo. 
The final structure of our role looks like this:

image2-23

The final structure of our role

First, let’s define the most fundamental part of our role, its tasks. Head to the tasks directory and edit the main.yml file.
roles/webserver/tasks/main.yml

Here, we define a handful of tasks that update the operating system, install an Nginx web server, and set up a minimal custom configuration for demo purposes.
Next, we move to the defaults directory, where we will set default values for the variables used in the tasks. If there is no other definition for these variables, they will be picked up and used by the role, but usually, they are meant to be easily overwritten.
roles/webserver/defaults/main.yml

Moving on to our vars directory, we define values with higher precedence that aren’t meant to be overridden at the play level. Here, we override the default variable that defines the Nginx custom directory.
roles/webserver/vars/main.yml

In the handlers directory, we define any handler that is triggered by our tasks. One of our tasks includes a notify keyword since it needs to trigger our Restart the Nginx service handler.
roles/webserver/handlers/main.yml

In the templates directory, we leverage a Jinja2 template file for the Nginx configuration that gets the Nginx custom directory value from one of our previously defined variables.
roles/webserver/templates/nginx.conf.j2

In the files directory, we define a static file index.html that will serve as our static demo webpage.
roles/webserver/files/index.html

We use the meta directory to add metadata and information about the role. Any role dependencies by other roles go here as well.
roles/webservers/meta/main.yml

Lastly, we update the README.md file accordingly. The autogenerated file provided by the ansible-galaxy init command includes many pointers and guidance on filling this nicely. 
roles/webserver/README.md

Once we have defined all the necessary parts of our role, it’s time to use it in plays. The classic and most obvious way is to reference a role at the play level with the roles option:

With this option, each role defined in our playbook is executed before any other tasks defined in the play.
This is an example play to try out our new webserver role. Let’s go ahead and execute this play. To follow along, you should first run the <code>vagrant up</code> command from the top directory of <a href="https://github.com/spacelift-io-blog-posts/Blog-Technical-Content/tree/master/ansible-roles" target="_blank" rel="nofollow noopener">this repository</a> to create our target remote host.

image3-20

webserver role

Sweet! All the tasks have been completed successfully. Let’s also validate that we have configured our Ngnix web server correctly.
Use the command <code>vagrant ssh host1</code> to connect to our demo Vagrant host. Then execute <code>systemctl status nginx</code> to verify that Nginx service is up and running. Finally, run the command <code>curl localhost</code> to check if the web server responds with the custom page that we configured.

image1-24

vagrant ssh host1

When using the roles option at the play level, we can override any of the default role’s variables or pass other keywords, like tags. Tags are added to all tasks within the role.

Here, we override the default variable nginx_version with another version.
Apart from defining roles at the play level with the roles option, we can use them also at the tasks level with the options include_role dynamically and import_role statically. These are useful when we would like to run our role tasks in a specific order and not necessarily before any other playbook tasks. This way, roles run in the order they are defined in plays.

Check the <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#using-roles" target="_blank" rel="nofollow noopener">official docs</a> for achieving more fine-grained control of your role’s task execution order. 
Even if we define a role multiple times, Ansible will execute it only once. Occasionally, we might want to run a role multiple times but with different parameters. Passing a different set of parameters to the same roles allows executing the role more than once. 
Example of executing the role test three times:

<a href="https://galaxy.ansible.com/" target="_blank" rel="nofollow noopener">Ansible Galaxy</a> is an online open-source, public repository of Ansible content. There, we can search, download and use any shared roles and leverage the power of its community. We have already used its client, ansible-galaxy, which comes bundled with Ansible and provides a framework for creating well-structured roles.
You can use Ansible Galaxy to browse for roles that fit your use case and save time by using them instead of writing everything from scratch. For each role, you can see its code repository, documentation, and even a rating from other users. Before running any role, check its code repository to ensure it’s safe and does what you expect. Here’s a blog post on <a href="https://www.jeffgeerling.com/blog/2019/how-evaluate-community-ansible-roles-your-playbooks" target="_blank" rel="noopener">How to evaluate community Ansible roles</a>. If you are curious about Galaxy, check out its <a href="https://galaxy.ansible.com/docs/" target="_blank" rel="nofollow noopener">official documentation</a> page for more details.
To download and install a role from Galaxy, use the ansible-galaxy install command. You can usually find the installation command necessary for the role on Galaxy. For example, look at <a href="https://galaxy.ansible.com/geerlingguy/postgresql" target="_blank" rel="nofollow noopener">this role that installs a PostgreSQL server</a>.
Install the role with:

Then use it in a playbook while overriding the default role variable postgresql_users to create an example user for us.

This section gathers some tips and tricks that might help you along your journey with Ansible roles.
<ul>
<li style="font-weight: 400;" aria-level="1">Always use descriptive names for your roles, tasks, and variables. Document the intent and the purpose of your roles thoroughly and point out any variables that the user has to set. Set sane defaults and simplify your roles as much as possible to allow users to get onboarded quickly.</li>
<li style="font-weight: 400;" aria-level="1">Never place secrets and sensitive data in your roles YAML files. Secret values should be passed to the role at execution time by the play as a variable and should never be stored in any code repository.</li>
<li style="font-weight: 400;" aria-level="1">At first, it might be tempting to define a role that handles many responsibilities. For instance, we could create a role that installs multiple components, a common anti-pattern. Try to follow the separation of concerns design principle as much as possible and separate your roles based on different functionalities or technical components.</li>
<li style="font-weight: 400;" aria-level="1">Try to keep your roles as loosely coupled as possible and avoid adding too many dependencies. </li>
<li style="font-weight: 400;" aria-level="1">To control the execution order of roles and tasks, use the import_role or Include_role tasks instead of the classic roles keyword.</li>
<li style="font-weight: 400;" aria-level="1">When it makes sense, group your tasks in separate task files for improved clarity and organization.</li>
</ul>
To learn more, check out the <a href="https://spacelift.io/blog/ansible-best-practices" target="_blank" rel="noopener">44 Ansible Best Practices to Follow</a>.

<a href="https://spacelift.io/" target="_blank" rel="noopener">Spacelift</a>’s vibrant ecosystem and excellent GitOps flow can greatly assist you in managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can then easily <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">create custom workflows</a> based on pull requests and apply any necessary <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">compliance checks</a> for your organization. Another great advantage of using Spacelift is that you can manage different infrastructure tools like Ansible, Terraform, Pulumi, AWS CloudFormation, and even Kubernetes from the same place and combine their <a href="https://docs.spacelift.io/concepts/stack/" target="_blank" rel="noopener">Stacks</a> with building workflows across tools.
If you want to learn more about Spacelift working with Ansible, check our <a href="https://docs.spacelift.io/vendors/ansible/" target="_blank" rel="noopener">documentation</a>, read our <a href="https://spacelift.io/blog/spacelift-ansible-integration-beta" target="_blank" rel="noopener">Ansible guide</a> or <a href="https://spacelift.io/schedule-demo" target="_blank" rel="noopener">book a demo with one of our engineers</a>.

We deep-dived into Ansible Roles and their utility and saw how to refactor our playbooks into roles or generate them from scratch. We went through a complete example of creating and using a role and explored how we can benefit from the Ansible Galaxy community.
Thank you for reading, and I hope you enjoyed this “Ansible Roles” blog post as much as I did.

Terraform vs. Ansible : Key Differences and Comparison of Tools

Ansible Loops : How To Use, Tutorial & Examples

Explore the concept of Ansible roles, their structure, and how we can refactor our playbooks into roles or generate them from scratch.

76-ansible-roles

Ansible Roles

Want to level up your Terraform knowledge? Explore our tutorials and step-by-step guides written by Terraform experts. Learn Terraform best practices and solutions to the most difficult problems.

Terraform Guides and Tutorials | Spacelift Blog

Terraform

Flavius Dinu

In this article, we explore best practices for managing Infrastructure as Code (IaC) with Terraform. Terraform is one of the most used tools in the <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">IaC</a> space that enables us to safely and predictably apply changes to our infrastructure. 
Starting with Terraform can feel like an intimidating task at first, but a beginner can quickly reach a basic understanding of the tool. After the initial learning period, a new user can then start running commands, creating, and refactoring Terraform code. During this process, many new users face nuances and issues around how to structure their code correctly, use advanced features, apply software development best practices in their IaC process, etc. 
Let’s check together some best practices that will assist you in pushing your Terraform skills to the next level. If you are entirely new to Terraform, look at the <a href="https://spacelift.io/blog/terraform" target="_blank" rel="noopener">Terraform Spacelift Blog</a>, where you can find a plethora of material, tutorials, and examples to boost your skills.
<h3>Terraform Best Practices:</h3>
<ol>
<li><a href="https://spacelift.io/blog/terraform-best-practices#1-use-remote-state">Use remote state</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#2-use-existing-shared-and-community-modules">Use existing shared and community modules</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#3-import-existing-infrastructure">Import existing infrastructure</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#4-avoid-variables-hardcoding">Avoid variables hard-coding</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#5-always-format-and-validate">Always format and validate</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#6-use-a-consistent-naming-convention">Use a consistent naming convention</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#7-tag-your-resources">Tag your Resources</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#8-introduce-policy-as-code">Introduce Policy as Code</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#9-implement-a-secrets-management-strategy">Implement a Secrets Management Strategy</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#10-test-your-terraform-code">Test your Terraform code</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#11-enable-debugtroubleshooting">Enable debug/troubleshooting</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#12-build-modules-wherever-possible">Build modules wherever possible</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#13-use-loops-and-conditionals">Use loops and conditionals</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#14-use-functions">Use functions</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#15-take-advantage-of-dynamic-blocks">Take advantage of Dynamic Blocks</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#16-use-terraform-workspaces">Use Terraform Workspaces</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#17-use-the-lifecycle-block">Use the Lifecycle Block</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#18-use-variables-validations">Use variables validations</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#12-leverage-helper-tools-to-make-your-life-easier">Leverage Helper tools to make your life easier</a></li>
<li><a href="https://spacelift.io/blog/terraform-best-practices#20-take-advantage-of-the-ide-extensions">Take advantage of the IDE extensions</a></li>
</ol>

In this section, we will describe some key Terraform concepts briefly. If you are already familiar with these, you can skip this section.
<h3>Terraform Configuration Language</h3>
Terraform uses its own <a href="https://www.terraform.io/language" target="_blank" rel="nofollow noopener">configuration language</a> to declare infrastructure objects and their associations. The goal of this language is to be declarative and describe the system’s state that we want to reach.
<h3>Resources</h3>
Resources represent infrastructure objects and are one of the basic blocks of the Terraform language. 
<h3>Data Sources</h3>
<a href="https://spacelift.io/blog/terraform-data-sources-how-they-are-utilised" target="_blank" rel="noopener">Data sources</a> feed our Terraform configurations with external data or data defined by separate Terraform projects.
<h3>Modules</h3>
<a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Modules</a> help us group several resources and are the primary way to package resources in Terraform for reusability purposes. 
<h3>State</h3>
Terraform keeps the information about the state of our infrastructure to store track mappings to our live infrastructure and metadata, create plans and apply new changes. 
<h3>Providers</h3>
To interact with resources on cloud providers, Terraform uses some plugins named providers.

Before moving to Terraform, first, let&#8217;s check some fundamental best practices that apply to all Infrastructure as Code projects. These should be used in your processes regardless of your tool to manage your cloud infrastructure. 
<h3>1. Use version control and prevent manual changes</h3>
This seems like an obvious statement in 2024, but it’s the basis of everything else. We should treat our infrastructure configurations as application code and apply the same best practices for managing, testing, reviewing, and bringing it to production. We should embrace a <a href="https://www.weave.works/technologies/gitops/" target="_blank" rel="nofollow noopener">GitOps</a> approach that fits our use case and implement an automated CI/CD workflow for applying changes.
<h3>2. Shift your culture to Collaborative IaC</h3>
Keeping your infrastructure in version-controlled repositories is the first step to improving your manual infrastructure management. Next, we should strive to enable usage across teams with self-service infrastructure, apply policies and compliance according to our organization’s standards, and access relevant insights and information. Thankfully, <a href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a> can assist you along the way to achieving these.

This section will explore some strategies for structuring our Terraform projects. In the world of Terraform, there is no right or wrong way to structure our configurations, and most of the suggested structures that you will find online are heavily opinionated.
When deciding how to set up your Terraform configuration, the most important thing is to understand your infrastructure needs and your use case and craft a solution that fits your team and the project.
If we are dealing with a small project with limited infrastructure components, it’s not a bad idea to keep our Terraform configuration as simple as possible. In these cases, we can configure only the necessary files for our root module, which are the configuration files that exist in the root directory. A small project can contain just these files main.tf, variables.tf, README.md. Some other files that you might find handy to use are the outputs.tf to define the output values of your project, versions.tf to gather any pinned versions for the configurations, and providers.tf to configure options related to the providers you use, especially if there are multiple. 
Our primary entry point is main.tf, and in simple use cases, we can add all our resources there. We define our <a href="https://spacelift.io/blog/how-to-use-terraform-variables" target="_blank" rel="noopener">variables</a> in variables.tf and assign values to them in terraform.tfvars. We use the file outputs.tf to declare <a href="https://spacelift.io/blog/terraform-output" target="_blank" rel="noopener">output values</a>. You can find a similar example project structure <a href="https://github.com/spacelift-io-blog-posts/Blog-Technical-Content/tree/master/terraform-best-practices/small-terraform-project-structure" target="_blank" rel="nofollow noopener">here</a>.

image2-21

smal Terraform project structure

When dealing with larger projects, things get a bit more complicated, and we have to take a step back to figure out the best structure for our projects. 
We first have to break down our Terraform code into reusable components that abstract details from consumers and different teams can use and customize accordingly. We can achieve this by creating separate modules for pieces of infrastructure that should be reused in <a href="https://spacelift.io/blog/terraform-environments" target="_blank" rel="noopener">different environments</a>, projects, and teams. 
A common practice is to separate our modules according to ownership and responsibility, rate of change, and ease of management. For every module, we need to define its inputs and outputs and document them thoroughly to enable consumers to use them effectively. We can then leverage outputs and terraform_remote_state to reference values across modules or even different Terraform states. 
Beware that using the terraform_remote_state data source implies access to the entire state snapshot and this might post a security issue. Another option to share parameters between different states is to <a href="https://www.terraform.io/language/state/remote-state-data#alternative-ways-to-share-data-between-configurations" target="_blank" rel="nofollow noopener">leverage an external tool</a> for publishing and consuming the data like Amazon SSM Parameter Store or HashiCorp Consul.
We have to make the next decision to either keep all our Terraform code in a single repository (monorepo) or to separate our Terraform configurations in multiple code repositories. This is considered a <a href="https://www.hashicorp.com/blog/terraform-mono-repo-vs-multi-repo-the-great-debate" target="_blank" rel="nofollow noopener">great debate</a> since both approaches have drawbacks and benefits. There is a tendency in the industry to avoid gigantic monorepos and use separate configurations to enable faster module development and flexibility. Personally, this is the approach I prefer too.
Usually, we have to deal with a plethora of different Infrastructure environments, and there are multiple ways to handle this in Terraform. A good and easy practice to follow is to separate Terraform configurations for different environments. This way, different environments have their own state and can be tested and managed separately, while shared behavior is achieved with shared or remote modules. 
One option is using a separate directory per environment and keeping a separate state for each directory. Another option would be to keep all the Terraform configurations in the same directory and pass different environment variables per environment to parametrize the configuration accordingly. Check out the <a href="https://www.youtube.com/watch?v=wgzgVm7Sqlk" target="_blank" rel="nofollow noopener">Evolving Your Infrastructure with Terraform</a> and <a href="https://www.youtube.com/watch?v=Nr5Km_xGLVs" target="_blank" rel="nofollow noopener">How I Manage More Environments with Less Code in Terraform </a>talks for some inspiration around structuring your projects. 
<a href="https://github.com/spacelift-io-blog-posts/Blog-Technical-Content/tree/master/terraform-best-practices/separate-environments-project-structure" target="_blank" rel="nofollow noopener">Here</a>, you can find an example structure for three different environments per directory: production, staging, and test. Each environment has its own state and is managed separately from the others while leveraging common or shared modules.  Although this approach comes with some code duplication, we gain improved clarity, environment isolation, and scalability.

image1-21

Separate environments project structure

As a general rule, we want to define Terraform configurations with limited scope and blast radius with specific owners. To minimize risk, we should try to decompose our projects into small workspaces/stacks and segment access to them using Role-based access control(RBAC).

<ul>
<li><a href="https://spacelift.io/blog/terraform-automation" target="_blank" rel="noopener">How to Automate Terraform Deployments and Infrastructure Provisioning</a></li>
<li><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener">How to Improve Your Infrastructure as Code using Terraform</a></li>
<li><a href="https://spacelift.io/blog/github-actions-terraform" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Managing Terraform at Scale with GitHub Actions&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Managing Terraform at Scale with GitHub Actions</a></li>
</ul>

Alright, in the previous sections, we talked about some generic IaC best practices. We explored some options for optimizing our Terraform code according to our organizational structure and needs. 
In this part, we are deep-diving into specific points that will take our Terraform code to the next level. This list isn’t exhaustive, and some parts are opinionated based on my personal preferences and experiences. 
The goal here is to give you hints and guidance on experimenting, researching, and implementing the practices that make sense to your use case.

Learn some best practices that will assist you in pushing your Terraform skills to the next level. See how they can make your IaC management easier.

Ioannis Moustakis

Education

License and certifications

Ansible Modules – How To Use Them Efficiently (Examples)

Get the newsletter

Recent posts

16 DevOps Best Practices Every Developer Should Know

AWS IAM Policies : Creating an IAM Policy & Best Practices

Ansible Roles: Basics, Creating & Using

20 Terraform Best Practices to Improve your TF workflow

Terraform Output Values : Complete Guide & Examples

How to Use Different Types of Ansible Variables (Examples)

Ansible Playbooks: Complete Guide with Examples

Ansible Tutorial for Beginners: Ultimate Playbook & Examples