Ioannis is a DevOps &amp; Site Reliability Engineer interested in Cloud Infrastructure, Automation, CI/CD Pipelines &amp; Containerization.

All posts by Ioannis Moustakis - Infrastructure as Code Expert

Ioannis Moustakis

Author title

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

Ansible Guides and Tutorials | Spacelift Blog

Ansible

Ansible is one of the most used open-source software tools for configuration management, software provisioning, and application deployment of cloud or on-premises environments. This article will look into best practices for setting up Ansible projects and suggest approaches to deal with Ansible’s internals effectively. 
If you are new to Ansible, take a look at this <a href="https://spacelift.io/blog/ansible-tutorial">Ansible Tutorial for Beginners</a>.

In this part, we will examine and discuss general best practices and recommendations for organizing your projects and getting the most out of Ansible.
<h3>Generic Best Practices</h3>
Prefer YAML instead of JSON: Although Ansible allows JSON syntax, using YAML is preferred and improves the readability of files and projects. 
Use consistent whitespaces: To separate things nicely and improve readability, consider leaving a blank line between blocks, tasks, or other components. 
Use a consistent tagging strategy: <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_tags.html" target="_blank" rel="nofollow noopener">Tagging</a> is a powerful concept in Ansible since it allows us to group and manage tasks more granularly.  Tags provide us with the option to add fine-grained controls to the execution of tasks. 
Add comments: When you think a further explanation is needed, feel free to add a comment explaining the purpose and the reason behind plays, tasks, variables, etc. 
Use a consistent naming strategy: Before starting to set up your Ansible projects, consider applying a consistent naming convention for your tasks (always name them), plays, variables, roles, and modules. 
Define a style guide: Following a style guide might be helpful to be consistent with the above suggestions. After all, how can we be consistent if we don’t have a style guide to follow and enforce? If you are searching for inspiration, look at <a href="https://github.com/openshift/openshift-ansible/blob/master/docs/style_guide.adoc" target="_blank" rel="nofollow noopener">this style guide by OpenShift</a>.
Keep it simple: Ansible provides many options and advanced features, but that doesn’t mean we have to use all of them. Find the Ansible parts and mechanics that fit your use case and keep your Ansible projects as simple as possible. For example, begin with a simple playbook and static inventory and add more complex structures or refactor later according to your needs.
Store your projects in a Version Control System (VCS): Keep your Ansible files in a code repository and commit any new changes regularly. 
Don&#8217;t store sensitive values in plain text: For secrets and sensitive values, use <a href="https://docs.ansible.com/ansible/latest/user_guide/vault.html" target="_blank" rel="nofollow noopener">Ansible Vault</a> to encrypt variables and files and protect any sensitive information. 
Test your ansible projects: Use linting tools like <a href="https://github.com/ansible/ansible-lint" target="_blank" rel="nofollow noopener">Ansible Lint</a> and add testing steps in your CI/CD pipelines for your Ansible repositories. For testing Ansible roles, have a look at <a href="https://molecule.readthedocs.io/en/latest/" target="_blank" rel="nofollow noopener">Molecule</a>. To test inputs or verify custom expressions, you can use the <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/assert_module.htmls" target="_blank" rel="nofollow noopener">assert module</a>.

<h3>Directory Organization</h3>
Take a look at this example of how a well-organized Ansible directory structure looks:

<h3>Inventory Best Practices</h3>
Use inventory groups: Group hosts based on common attributes they might share (geography, purpose, roles, environment).
Separate Inventory per Environment: Define a separate inventory file per environment (production, staging, testing, etc.) to isolate them from each other and avoid mistakes by targeting the wrong environments. 
Dynamic Inventory: When working with cloud providers and ephemeral or fast-changing environments, maintaining static inventories might quickly become complex. Instead, set up a mechanism to <a href="https://docs.ansible.com/ansible/2.8/user_guide/intro_dynamic_inventory.html">synchronize the inventory dynamically with your cloud providers</a>. 
Leverage Dynamic Grouping at runtime: We can create dynamic groups using the `group_by` module based on a specific attribute. For example, group hosts dynamically based on their operating system and run different tasks on each without defining such groups in the inventory.

In this part, we will list and discuss best practices regarding using plays &amp; playbooks, two of the most basic components of Ansible projects.
Always mention the state of tasks: To make your tasks more understandable, explicitly set the state parameter even though it might not be necessary due to the default value.
Place every task argument in its own separate line:  This point is in line with the general approach of striving for readability in our Ansible files. Check the examples below. 
This works but isn’t readable enough:

Use this syntax instead, which improves a lot the readability and understandability of the tasks and their arguments:

Use top-level playbooks to orchestrate other lower-level playbooks: You can logically group tasks, plays, and roles into low-level playbooks and use other top-level playbooks to import them and set up an orchestration layer according to your needs. Have a look at <a href="https://docs.ansible.com/ansible/2.8/user_guide/playbooks_best_practices.html#top-level-playbooks-are-separated-by-role" target="_blank" rel="nofollow noopener">this example</a> for inspiration.
Use block syntax to group tasks: Tasks that relate to each other and share common attributes or tags can be grouped using the block option. Another advantage of this option is easier rollbacks for tasks under the same block.

Use handlers for tasks that should be triggered: Handlers allow a task to be executed after something has changed. This handler will be triggered when there are changes to <code>index.html</code> from the above example.

Check out the <a href="https://spacelift.io/blog/ansible-playbooks" target="_blank" rel="nofollow noopener">Working with Ansible Playbooks</a> blog post for more details about playbooks.

Variables allow users to parametrize different Ansible components and store values we can reuse throughout projects. Let’s look at some best practices and tips on using Ansible variables.
Always provide sane defaults for your variables: Set default values for all groups under <code>group_vars/all</code>. For every role, set default role variables in <code>roles/&lt;role_name&gt;/defaults.main.yml</code>.
Use groups_vars and host_vars directories: To keep your inventory file clean, prefer setting group and hosts variables in the <code>groups_vars</code> and <code>host_vars</code> directories.
Add the role’s name as a prefix to variables: Try to be explicit when defining variable names for your roles by adding a prefix with the role name.

Keep your variable’s setup simple: <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable" target="_blank" rel="nofollow noopener">Many options are available for setting variables in Ansible</a>. Using all of them isn’t probably the way to go unless you have specific needs. Pick the ones more appropriate for your use case and keep it as simple as possible.
Check out the <a href="https://spacelift.io/blog/ansible-variables" target="_blank" rel="nofollow noopener">How to Use Different Types of Ansible Variables</a> blog post for more pointers and tips.

This part will display some tips and best practices for using Ansible modules efficiently in tasks. 
Keep local modules close to playbooks: Use each Ansible project&#8217;s <code>./library</code> directory to store relevant custom modules. Playbooks that have a <code>./library</code> directory relative to their path can directly reference any modules inside it.
Avoid command and shell modules: It’s considered a best practice to limit the usage of <code>command</code> and <code>shell</code> modules only when there isn&#8217;t another option. Instead, prefer specialized modules that provide idempotency and proper error handling. 
Specify module arguments when it makes sense: Default values can be omitted in many module arguments. To be more transparent and explicit, we can opt to specify some of these arguments, like the <code>state</code> in our playbook definitions.
Prefer multi-tasks in a module over loops: The most efficient way of defining a list of similar tasks, like installing packages, is to use multiple tasks in a single module.

Document and test your custom modules: Every custom module should include examples, explicitly document dependencies, and describe return responses. New modules should be tested thoroughly before releasing. You can create testing roles and playbooks to test your custom modules and validate different test cases. 
For more details about using modules and writing your own custom modules, check the <a href="https://spacelift.io/blog/ansible-modules" target="_blank" rel="nofollow noopener">Ansible Modules &#8211; How to Use Them Efficiently</a> blog post.

Ansible roles enable reusability and sharing of code efficiently, while they provide a well-structured framework for configuring and setting our projects. This part will examine some best practices and tips for creating well-defined roles.
Follow the Ansible Galaxy Role Directory structure: Leverage the <code>ansible-galaxy init &lt;role_name&gt;</code> command to generate a default role directory layout according to Ansible Galaxy’s standards.

ansible-galaxy

ansible galaxy

Keep your roles single-purposed: Each role should have a separate responsibility and distinct functionality to conform with the separation of concerns design principle. Separate your roles based on different functionalities or technical domains. 
Try to limit role dependencies: By avoiding many dependencies in your roles, you can keep them loosely coupled, develop them independently, and use them without managing complex dependencies between them.
Prefer import_role or include_role: To better control the execution order of roles and tasks, prefer using <code>import_role</code> or <code>include_role</code> over the classic <code>roles</code> option.
Do your due diligence for Ansible Galaxy Roles: When downloading and using content and roles from Galaxy, do your due diligence, validate their content, and pick roles from trustworthy contributors.
Store Galaxy roles used locally: To avoid depending on the upstream of Ansible Galaxy, you can store any roles from Galaxy to your code repositories and manage them as part of your project.

Ansible provides many controls and options to orchestrate execution against hosts. In this part, we will explore tips and tricks on optimally controlling Ansible execution based on our needs.
Test changes in staging first: Having a staging or testing environment to test your tasks before production is a great way to validate that your changes have the expected outcome.
Limit task execution to specific hosts: If you want to run a playbook against specific hosts, you can use the <code>--limit</code> flag.
Limit tasks execution to specific tasks based on tags: In case you need to run only specific tasks from a playbook based on tags, you can define which tags to be executed with the <code>--tags</code> flag.
Validate which tasks will run before executing: You can use the <code>--list-tasks</code> flag to confirm which tasks would be run without actually running them.
Validate against which hosts the playbook will run: You can use the <code>--list-hosts</code> flag to confirm which hosts will be affected by the playbook without running it.
Validate which changes will happen without making them: Leverage the <code>--check</code> flag to predict any changes that may occur. Combine it with <code>--diff</code> flag to show differences in changed files.
Start at a specific task: Use the <code>--start-at-task</code> flag to start executing your playbook at a particular task.
Use Rolling Updates to control the number of target machines: By default, Ansible attempts to run the play against all hosts in parallel. To achieve a rolling update setup, you can leverage the <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html#setting-the-batch-size-with-serial" target="_blank" rel="nofollow noopener">serial</a> keyword. Using this keyword, you can define how many to hosts the changes can be performed in parallel.   
Control playbook execution strategy: By default, Ansible finishes the execution of each task on all hosts before moving to the next task. if you wish to select another execution strategy, have a <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html#controlling-playbook-execution-strategies-and-more" target="_blank" rel="nofollow noopener">look at this guide</a>.

<a href="https://spacelift.io/" target="_blank" rel="noopener">Spacelift</a>’s vibrant ecosystem and excellent GitOps flow can greatly assist you in managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can then easily <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">create custom workflows</a> based on pull requests and apply any necessary <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">compliance checks</a> for your organization. Another great advantage of using Spacelift is that you can manage different infrastructure tools like Ansible, Terraform, Pulumi, AWS CloudFormation, and even Kubernetes from the same place and combine their <a href="https://docs.spacelift.io/concepts/stack/" target="_blank" rel="noopener">Stacks</a> with building workflows across tools.
Spacelift is currently running a closed beta of Ansible. If you want to learn more, look <a href="https://docs.spacelift.io/vendors/ansible/" target="_blank" rel="noopener">here</a>.
If you wish to be part of the beta version testing and provide feedback, <a href="https://spacelift.io/ansible-beta" target="_blank" rel="noopener">sign up here</a>!

In this blog post, we delved into best practices, tips, and tricks for operating and configuring Ansible projects. We explored approaches for structuring our Ansible projects and roles and set different configuration options regarding the inventory and variables. Lastly, we examined various tips for controlling our playbook’s execution and deployments. 
Thank you for reading, and I hope you enjoyed this article as much as I did.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence. It supports Terraform, Cloudformation, Pulumi, and Kubernetes, with initial support for Ansible.

Terraform vs. Ansible : Key Differences and Comparison of Tools

16 DevOps Best Practices Every Developer Should Know

How Spacelift Can Improve Your Infrastructure as Code

This article will look into best practices for setting up Ansible projects and suggest approaches to deal with Ansible’s internals effectively. 

108-ansible-best-practices

Ansible best practices

44 Ansible Best Practices to Follow [Tips & Tricks]

Want to level up your Terraform knowledge? Explore our tutorials and step-by-step guides written by Terraform experts. Learn Terraform best practices and solutions to the most difficult problems.

Terraform Guides and Tutorials | Spacelift Blog

Terraform

This blog post will look into two Infrastructure as Code (IaC) automation tools, Atlantis and Terraform Cloud/Enterprise and analyze their similarities and differences. 
Atlantis allows users to orchestrate <a href="https://spacelift.io/blog/terraform-automation" target="_blank" rel="noopener">Terraform automation</a> through pull requests by using comments, and it’s a great tool suited for small-scale projects and casual usage. Terraform Cloud provides a specialized CI/CD platform for Terraform automation and a great remote backend solution. Terraform Cloud is more scalable than Atlantis but offers fewer extensibility options.

<a href="https://www.runatlantis.io/" target="_blank" rel="nofollow noopener">Atlantis</a> is an open-source and self-hosted Terraform &#8220;pull request-based&#8221; automation tool. It offers an easy way to automate the Terraform workflow using pull request comments. On every new pull request, Atlantis automatically runs the <code>terraform plan</code> command and comments the output back on the pull request. After the suggested changes have been reviewed, a team member can leave a pull request comment with a special meaning to apply the changes. 
A great benefit of the Atlantis workflow is that it doesn&#8217;t add a new user interface (UI) for operators and developers but integrates nicely with your choice&#8217;s version control system (VCS) provider. It provides the option to perform code reviews and Terraform operations via the same graphical user interface. Users don&#8217;t need access credentials for the infrastructure provider, and errors can be caught during the code review step. With the Atlantis model, each pull request contains a detailed audit log of changes made via Terraform.
Atlantis self-hosted runners can be given an identity native to your cloud (e.g., AWS instance profile) for access without credentials to the state and managed resources. They can also be configured to run inside the <a href="https://aws.amazon.com/vpc/" target="_blank" rel="nofollow noopener">Virtual Private Cloud (VPC)</a> to access local resources (e.g., VPC-internal database) but need inbound connectivity from the VCS provider to receive webhooks. Its configuration is primarily done using environment variables passed to the statically linked binary and the <a href="https://www.runatlantis.io/docs/configuring-atlantis.html" target="_blank" rel="nofollow noopener">YAML file</a>.
Atlantis is stateless, but one of its main drawbacks is that it doesn&#8217;t support a high-availability setup or any scaling and queueing support. To accommodate scaling and highly available setups, a substantial engineering effort and creativity are required to build a custom in-house solution. 
Flexibility is one of the core advantages of Atlantis, as it allows easy integration with other Terraform-helper tools(e.g., <a href="https://github.com/aquasecurity/tfsec" target="_blank" rel="nofollow noopener">tfsec</a>, <a href="https://www.checkov.io/" target="_blank" rel="nofollow noopener">checkov</a>, <a href="https://github.com/infracost/infracost-atlantis" target="_blank" rel="nofollow noopener">Infracost</a>, or <a href="https://www.terratag.io/" target="_blank" rel="nofollow noopener">Terratag</a>). It can work with Terraform wrappers, such as <a href="https://terragrunt.gruntwork.io/" target="_blank" rel="nofollow noopener">Terragrunt</a>, out of the box and even add some of Terragrunt&#8217;s features to vanilla Terraform &#8211; like <a href="https://www.runatlantis.io/docs/custom-workflows.html" target="_blank" rel="nofollow noopener">before and after hooks</a> for every execution stage (init, plan, apply, etc.). 
Atlantis has a vibrant and active community, with new versions being released often. Something to note here is that although the development is active and there are regular contributions, the efforts aren&#8217;t focused on new major features since the lead contributor <a href="https://medium.com/runatlantis/joining-hashicorp-200ee9572dc5" target="_blank" rel="nofollow noopener">moved to Hashicorp</a>.
Overall, it&#8217;s a great tool suited for small-scale operations and infrastructure topologies. It is much appreciated by its user community and offers a flexible automation solution for occasional use. Having said that, it is strongly limited by its architecture, and scaling it isn&#8217;t straightforward. If your company has large-scale infrastructure needs, other more robust and mature solutions exist.

<a href="https://cloud.hashicorp.com/products/terraform" target="_blank" rel="nofollow noopener">Terraform Cloud</a> is a more comprehensive infrastructure provisioning tool that works exclusively for Terraform, developed by Hashicorp. It provides a scalable solution to automate infrastructure delivery, handle compliance, and manage resources in a cloud-agnostic way, utilizing Terraform. It is Hashicorp’s SaaS managed service offering targeting the Terraform workflow.
One of Its main offerings is a specialized CI/CD platform to standardize Terraform deployments and reduce their time. It supports an excellent remote state backend and an API for remote Terraform operations and integration with existing workflows. It integrates with VCS providers and allows fully automated, or manual approval checks for infrastructure provisioning flows. 
Interaction with Terraform Cloud can be achieved with the command-line interface (CLI), UI, API, or CI jobs. The <a href="https://www.terraform.io/language/settings/backends/remote" target="_blank" rel="nofollow noopener">remote or enhanced backend</a> allows teams to run the Terraform binary from their laptops or a third-party CI job, but the operation is executed on a remote machine. This is especially useful for one-off administrative tasks like tainting or migrating resources &#8211; things that are not trivial with Atlantis and may require dedicated solutions like <a href="https://github.com/minamijoyo/tfmigrate" target="_blank" rel="nofollow noopener">tfmigrate</a>.
Terraform Cloud offers basic security essentials such as RBAC with custom workspace permissions and different access levels for different types of users. Its integration with single sign-on (SSO) allows administrators easy user onboarding and management.
Unlike Atlantis, Terraform Cloud architecture is highly scalable, so it will take a while to outgrow it. It offers a shared state, distributed execution, concurrent runs, notifications for workspace events, and VCS integrations to support its scalability. 
Teams can leverage Terraform Cloud’s rich API imperatively from external scripts or declaratively from Terraform itself, using <a href="https://registry.terraform.io/providers/hashicorp/tfe/latest/docs" target="_blank" rel="nofollow noopener">their provider</a>. Managing Terraform with Terraform is often a secret to managing IaC at scale in dynamic organizations. It also supports exporting audit logs to external systems via its API. 
One drawback of Terraform Cloud compared to Atlantis is that it is less extensible. While Atlantis lets you execute arbitrary shell commands as part of your Terraform job, Terraform Cloud depends on clever hacks like the <a href="https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource" target="_blank" rel="nofollow noopener">null resource</a> or an external wrapper. For example, suppose you are a Terragrunt user. In that case, you may want to have a CI job (e.g., Jenkins or GitHub Actions) trigger Terragrunt that’s later shelling out to Terraform, which executes the job on your remote Terraform Cloud environment. This extra layer complicates the architecture and workflow and introduces another party to a sensitive flow.
Terraform Cloud offers some native integrations and third-party tools to incorporate into the Terraform workflow, like their proprietary policy-as-code framework, Sentinel. Leveraging Sentinel, you can create security and compliance guardrails. The disadvantage of Sentinel is that it’s not an industry standard and open-source like <a href="https://spacelift.io/blog/what-is-open-policy-agent-and-how-it-works" target="_blank" rel="noopener">Open Policy Agent (OPA)</a>. HashiCorp recently announced publishing <a href="https://www.hashicorp.com/blog/introducing-sentinel-policies-to-the-terraform-registry" target="_blank" rel="nofollow noopener">reusable Sentinel policies in their public Terraform Registry</a>, which may give Sentinel a new lease of life in the Terraform ecosystem.
Last but not least, Terraform Cloud recently announced a <a href="https://www.hashicorp.com/blog/terraform-cloud-adds-drift-detection-for-infrastructure-management" target="_blank" rel="nofollow noopener">drift detection feature</a>, which allows you to monitor the synchronization between your resources and their respective Terraform definitions. This feature can, to some extent, be replicated using /plan and /apply HTTP endpoints from Atlantis, but it’s a far cry from the native, built-in solution that Terraform Cloud offers.

If for any reason (compliance, regulatory needs, etc.), your organization needs an on-premise version of Terraform Cloud, you can use Terraform Enterprise. <a href="https://www.terraform.io/enterprise" target="_blank" rel="nofollow noopener">Terraform Enterprise</a> is a self-hosted distribution of Terraform Cloud.
It provides organizations with a private environment installation of the Terraform Cloud instance and enterprise-grade features like single sign-on, compliance enforcement with policies, and audit logging. 
If you plan on hosting your own Terraform Enterprise distribution, have a look at the <a href="https://www.terraform.io/enterprise/requirements/credentials" target="_blank" rel="nofollow noopener">requirements</a>, <a href="https://www.terraform.io/enterprise/reference-architecture" target="_blank" rel="nofollow noopener">reference architectures</a> for common cloud providers, and the <a href="https://www.terraform.io/enterprise/install/pre-install-checklist" target="_blank" rel="nofollow noopener">installation and configuration guide</a>.

<h3>On-premise Support</h3>
Both Atlantis and Terraform Enterprise offer the possibility to host your own installation of the tools. 
<h3>Integration with VCS providers</h3>
Most of the standard VCS providers are supported and integrated seamlessly with Atlantis and Terraform Cloud/Enterprise.
<h3>Integration with CI/CD</h3>
Both tools can be incorporated into your organization&#8217;s existing CI/CD flows and work in parallel with existing continuous integration jobs.

terraform-cloud-vs-atlantis

terraform cloud vs atlantis

<h3>SaaS Offering</h3>
Terraform Cloud/Enterprise is a managed service SaaS offering, while Atlantis doesn’t have a similar offering.
<h3>User Interface</h3>
Atlantis uses the same UI as the VCS provider that you are using and allows operators to trigger automation jobs from pull requests. Terraform Cloud/Enterprise comes with its own UI and portal.
<h3>Open Source Availability</h3>
Atlantis is entirely open-source and free, while Terraform Cloud/Enterprise is a proprietary solution, although it offers a free version.
<h3>Remote State Backend</h3>
As part of its offering, Terraform Cloud provides an excellent backend for the Terraform state, while Atlantis doesn’t.
<h3>High Availability</h3>
Terraform Cloud/Enterprise is designed for scale and offers a highly available setup. Scaling and building highly available setups with Atlantis requires additional effort.
<h3>Flexibility/Extensibility</h3>
Atlantis is very flexible and can integrate with other helper tools easily. Extending Terraform Cloud functionality is a bit more cumbersome.
<h3>Security Features</h3>
Atlantis keeps all the infrastructure changes and ties them to pull requests that can be used as audit logs. Terraform Cloud offers more elaborate security essentials like RBAC, single sign-on with SAML, and an audit log.
<h3>Drift Detection</h3>
Terraform Cloud offers <a href="https://spacelift.io/blog/drift-detection" target="_blank" rel="noopener">drift detection</a>, whereas Atlantis doesn’t by default, although similar functionality can be replicated with additional effort.
<h3>Cost Calculation</h3>
Terraform Cloud offers cost estimation, whereas Atlantis doesn’t by default, although similar functionality can be replicated by adding external tooling.

Actually, Atlantis and Terraform Cloud can be used together since <a href="https://www.runatlantis.io/docs/terraform-cloud.html" target="_blank" rel="nofollow noopener">Atlantis integrates seamlessly with Terraform Cloud/Enterprise</a>. It doesn’t matter which flavor or Terraform Cloud/Enterprise your team uses since Atlantis can work with all of them.
If that’s up your alley, you can have the “pull request-based” flow with some of the benefits of a managed solution, like history, access to policies with Sentinel, stopping runs, secret storage, etc. At this point, any generic CI tool would likely do the trick, so there may not be a point in maintaining a self-hosted installation of Atlantis to hand over the work to Terraform Cloud.

Terraform Cloud has been one of the first players in the space, but it’s not the most feature-rich platform anymore. Atlantis is great for small projects, but missing features and scaling might cause headaches.
If you’re choosing between Atlantis and Terraform Cloud, why not give a chance to <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Batlantis_terraform%7D" target="_blank" rel="noopener">Spacelift</a>, a modern collaborative infrastructure delivery tool with a great focus on user experience? It works with Terraform, Terragrunt, and many other IaC frameworks, supports self-hosted on-prem workers, workflow customization, drift detection, and much more.
For more differences between the tools, I encourage you to check the article <a href="https://spacelift.io/blog/alternative-to-atlantis" target="_blank" rel="noopener">Spacelift vs. Atlantis</a> and <a href="https://spacelift.io/terraform-cloud-alternative" target="_blank" rel="noopener">Spacelift vs. Terraform Cloud.</a>
Spacelift provides a more mature way of automating the whole infrastructure provisioning lifecycle. Its <a href="https://docs.spacelift.io/concepts/run/" target="_blank" rel="noopener">flexible and robust workflow</a> allows teams to get up to speed quickly and collaborate efficiently. Spacelift is highly extensible and will enable teams to enhance the Terraform workflow with custom providers, linters, security tools, and any other custom tooling they see fit.
Spacelift connects directly to the version control system of your choice and provides a truly GitOps native approach. It can support setups with multiple repositories or massive monorepos and leverages the APIs of the VCS provider to give you visibility. 
Spacelift has a built-in CI/CD functionality for developing custom modules allowing teams to incorporate testing, checks, and linting early into the development phase of modules. Another benefit of using Spacelift is its flexible workflow management. It provides a policy-based process to handle dependencies between projects and deployments with <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">Trigger Policies</a>. 
Spacelift provides a plethora of <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">Policies</a> to allow teams to define and automate rules governing the infrastructure as code. By utilizing <a href="https://www.openpolicyagent.org/" target="_blank" rel="noopener">Open Policy Agent</a>, users can create their own custom policies and ensure the compliance of Terraform resources.
Check out the <a href="https://docs.spacelift.io/" target="_blank" rel="noopener">Documentation</a> and start automating your infrastructure delivery seamlessly!

We have looked into two infrastructure automation and delivery tools, Atlantis and Terraform Cloud. We analyzed each of them and discussed their strengths and weaknesses, along with a feature comparison. Finally, we saw how a modern collaborative infrastructure delivery tool like Spacelift could be used as an alternative.
Thank you for reading, and I hope you enjoyed this article as much as I did.

Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures.

Alternative to Atlantis – Spacelift

Achieving the Atlantis Workflow with Spacelift

Atlantis or Terraform Cloud ? Take a look at two infrastructure automation and delivery tools, along with a feature comparison.

101-atlantis-vs-terraform-cloud

Atlantis vs Terraform Cloud

Atlantis vs. Terraform Cloud / Enterprise - Comparison

Terraform provides a well-defined and concise way to deploy infrastructure resources and changes. The typical workflow involves manual steps and checks that aren’t easily scalable and depend on human intervention to complete successfully. 
This article will look into different approaches to automating infrastructure provisioning with Terraform and the pros and cons of each one.

Terraform is one of the most prominent tools in the Infrastructure as Code space. Part of its success lies in the straightforward and easy-to-operate workflow it provides. If you aren’t familiar with Terraform, check the various <a href="https://spacelift.io/blog/terraform" target="_blank" rel="noopener">Terraform articles on Spacelift’s blog</a> to get an idea. 
The core Terraform workflow consists of three concrete stages. First, we generate the infrastructure as code configuration files representing our environment&#8217;s desired state. Next, we check the output of the generated plan based on our manifests. After carefully reviewing the changes, we apply the plan to provision infrastructure resources.

terrafrom-automation-diagram

terrafrom automation - diagram

Typically, this workflow involves some manual steps that are easily automatable. For example, using the Terraform CLI we have to run the command <a href="https://spacelift.io/blog/terraform-plan" target="_blank" rel="noopener"><code>terraform plan</code></a> to check the effect of our newly prepared configuration files. Similarly, we have to execute the command <a href="https://spacelift.io/blog/terraform-apply" target="_blank" rel="noopener"><code>terraform apply</code></a> to propagate the changes to the live environment. 
If you want to have all of the important Terraform commands in one place, take a look at our <a href="https://spacelift.io/blog/terraform-commands-cheat-sheet" target="_blank" rel="noopener">Terraform Cheat Sheet</a>.
For individual contributors and small teams, operating Terraform with the typical workflow and with manual steps to plan and apply the code might work perfectly fine. When teams get bigger, though, and we want to scale Terraform&#8217;s usage across organizations, we quickly reach bottlenecks and issues.

As Terraform’s usage across teams matures, adding some kind of deployment automation is beneficial. Let’s look into some of the approaches to running Terraform in automation.
<h3>1) Terraform Orchestration with Custom Tooling</h3>
Some teams continue running Terraform locally, but they add custom tooling, pre-commit hooks, and wrappers (e.g., <a href="https://terragrunt.gruntwork.io/" target="_blank" rel="nofollow noopener">Terragrunt</a>) to enhance the core Terraform workflow. There are different wrapper tools to choose from that provide extra functionalities, such as <a href="https://spacelift.io/blog/terragrunt-with-spacelift" target="_blank" rel="noopener">keeping your configuration DRY</a>, managing remote state, and managing different environments. Other teams prefer writing their own custom wrapper scripts to prepare Terraform working directories according to some standards. 
This semi-manual approach is more native to the core Terraform workflow and allows direct access to output and running operational commands (e.g., <a href="https://spacelift.io/blog/importing-exisiting-infrastructure-into-terraform" target="_blank" rel="noopener">terraform import</a>). On the other hand, since it involves manual steps, it is error-prone and requires human intervention. One more thing to note is that usually, this approach requires privileged access to the underlying infrastructure provider as well as the Terraform state file, which might be a security risk.
One step ahead, some teams develop their own custom platform on top of Terraform manifests that allow end users to provision infrastructure resources via tweaking some configuration changes on a UI. 
This approach abstracts any unnecessary level of detail and Terraform-specific knowledge from end users and allows them to manage infrastructure on demand without another team blocking them. On the flip side, this path requires substantial engineering effort to develop a useful platform and adds a maintenance overhead to the platform team. 
<h3>2) Building infrastructure provisioning pipelines</h3>
The most common approach for running Terraform in automation is to build infrastructure provisioning pipelines with a CI/CD tool. With this method, teams can enforce best practices, add tests and checks, and integrate the Terraform workflow to any CI/CD tool they are familiar with. 
Building infrastructure delivery pipelines in <a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">CI/CD tools</a> brings several challenges as it needs to adjust the core Terraform workflow for non-interactive environments. 
The first step for automating Terraform deployments is to embrace Infrastructure as Code and GitOps and store your manifests in the version control system of your choice. Having versioned repositories as the source of truth for automating infrastructure delivery is a core requirement. 
Next, the typical Terraform workflow is adjusted for running in remote environments. Since the run might be triggered in ephemeral environments, we have to initialize the Terraform working directory, run any custom checks or tests and produce a planned output for changing resources. 
A common tactic is integrating these steps as part of every proposed code change (e.g., Pull Requests). Once other team members review the proposed changes and find the produced plan acceptable, they can approve and merge the Pull Request. Merging new code to the branch that is considered the source of truth triggers a terraform apply to provision the latest changes. 
<h3>10 things to consider for building your own automated Terraform delivery pipelines:</h3>
<ul>
<li style="font-weight: 400;" aria-level="1">Your code should be stored in a version control system.</li>
<li style="font-weight: 400;" aria-level="1">Leverage the <code>-input=false</code> flag to disable interactive prompts. The command line, environment variables, or configuration files should provide any necessary input.</li>
<li style="font-weight: 400;" aria-level="1">Use a backend that supports remote <a href="https://spacelift.io/blog/terraform-state" target="_blank" rel="noopener">Terraform State</a> to allow runs on different machines and state locking for safety against race conditions. </li>
<li style="font-weight: 400;" aria-level="1">Prepare an environment to run Terraform with any dependencies pre-installed. To avoid downloading the provider plugins every time with the init command, use the flag <code>-plugin-dir</code> to provide the path of preconfigured plugins on the automation system.</li>
<li style="font-weight: 400;" aria-level="1">To allow changing the default backend configuration to deploy with different permissions or to different environments, you can utilize the <code>-backend-config=path</code> flag when initializing. If you only need to run checks on the Terraform files that don&#8217;t require initializing the backend (e.g., terraform validate), consider using the flag <code>-backend=false</code>.</li>
<li style="font-weight: 400;" aria-level="1">Integrate Terraform formatting, validating, linting, checking policies, and custom testing to the <a href="https://spacelift.io/blog/ci-cd-pipeline" target="_blank" rel="noopener">CI/CD pipelines</a> to ensure your code conforms to your organization’s standards. </li>
<li style="font-weight: 400;" aria-level="1">Usually, CI/CD pipelines run on distributed systems. To ensure that we will apply the correct plan, we can output the plan to a file and package the whole terraform working directory after each plan. These artifacts will be stored somewhere to be fetched by the apply step to avoid accidentally applying different changes to the ones reviewed. </li>
<li style="font-weight: 400;" aria-level="1">Optionally, use the flag <code>-auto-approve</code> to apply the changes without human intervention. </li>
<li style="font-weight: 400;" aria-level="1">Use environment variables prefixed with <code>TF_VAR_</code> to pass any necessary values using the CI/CD tool mechanisms. </li>
<li style="font-weight: 400;" aria-level="1">Set the environment variable <code>TF_IN_AUTOMATION</code> to indicate that Terraform is running in automation mode. This adjusts the output of some commands to avoid outputting messages that are misleading in an automation environment.</li>
</ul>
Integrating the Terraform workflow in CI/CD infrastructure provisioning pipelines is a great way to automate infrastructure delivery. Running <a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener">Terraform in CI/CD</a> automation eliminates the need for people’s privileged access, enforces a consistent workflow and way of working, and removes any human intervention.
On the other hand, we have to take into account considerations for running on distributed systems, spend substantial engineering time and put a lot of effort into building a custom pipeline that satisfies our team’s needs

A more robust approach to automating your Terraform workflows end-to-end would be to use <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bterraform_automation%7D" target="_blank" rel="noopener">Spacelift</a>, a collaborative infrastructure delivery tool. Spacelift provides a more mature way of automating the whole infrastructure provisioning lifecycle. Its <a href="https://docs.spacelift.io/concepts/run/" target="_blank" rel="noopener">flexible and robust workflow</a> allows teams to get up to speed quickly and collaborate efficiently.
Spacelift connects directly to the version control system of your choice and provides a truly GitOps native approach. It can support setups with multiple repositories or massive monorepos and leverages the APIs of the VCS provider to give you visibility. 
The Spacelift runners are fully customizable Docker containers. This allows teams to enhance the Terraform workflow with custom providers, linters, security tools, and any other custom tooling you might see fit. 
Spacelift has a built-in CI/CD functionality for developing custom modules allowing teams to incorporate testing, checks, and linting early into the development phase of modules. Another benefit of using Spacelift is its flexible workflow management. It provides a policy-based process to handle dependencies between projects and deployments with <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">Trigger Policies</a>. 
Sometimes, the actual state of live environments drifts from the desired state, a concept known as configuration drift. Spacelift can assist you in <a href="https://docs.spacelift.io/concepts/stack/drift-detection" target="_blank" rel="noopener">automatically detecting and, if desired, reconciling configuration drift</a>.
Spacelift provides a plethora of <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">Policies</a> to allow teams to define and automate rules governing the infrastructure as code. By utilizing <a href="https://www.openpolicyagent.org/" target="_blank" rel="noopener">Open Policy Agent</a>, users can create their own custom policies and ensure the compliance of Terraform resources.
Check out the <a href="https://docs.spacelift.io/" target="_blank" rel="noopener">Documentation</a> and start automating your infrastructure delivery easily!

In this blog post, we discussed different approaches and strategies to automate Terraform deployments and provision infrastructure in an automated fashion. We looked into the typical Terraform workflow and saw how we can enhance it with orchestration tools. Finally, we saw how Spacelift greatly assists us in bringing our Terraform automation to the next level.
Thank you for reading, and I hope you enjoyed this article as much as I did.

Automate your infrastructure provisioning, build more complex workflows based on Terraform using policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and many more.

5 Ways to Manage Terraform at Scale - Best Practices

Infrastructure Drift Detection and How to Fix It With IaC Tools

12 Terraform Best Practices to Improve your TF workflow

Learn different approaches to Terraform automation and provision infrastructure in an automated fashion. See how to enhance it with orchestration tools.

104-terraform-automation

Terraform automation

How to Automate Infrastructure Provisioning with Terraform

Want to level up your Kubernetes knowledge? Explore our tutorials and step-by-step guides written by Kubernetes experts. Learn Kubernetes best practices and solutions to the most difficult problems.

Kubernetes Guides and Tutorials | Spacelift Blog

Kubernetes

This article will compare two of the most dominant tools in the cloud infrastructure space, Terraform and Kubernetes. These two tools share some similarities but are built to serve different purposes. Terraform is a tool focused on infrastructure provisioning and operates in the Infrastructure as code space. Kubernetes focuses on running container workloads and operates in the container orchestration space.
We will briefly take a look at each one of them and discuss their similarities and differences.
To learn more about these two foundational cloud infrastructure technologies, check the multiple tutorials on Spacelift’s blog around <a href="https://spacelift.io/blog/kubernetes" target="_blank" rel="noopener" data-saferedirecturl="https://www.google.com/url?q=https://spacelift.io/blog/kubernetes&amp;source=gmail&amp;ust=1661233733361000&amp;usg=AOvVaw0U55gyVSYyB8fmHIbefmpF">Kubernetes</a> and <a href="https://spacelift.io/blog/terraform" target="_blank" rel="noopener" data-saferedirecturl="https://www.google.com/url?q=https://spacelift.io/blog/terraform&amp;source=gmail&amp;ust=1661233733361000&amp;usg=AOvVaw3AGtv2eOlNveyd8H80rYk_">Terraform</a>.

<a href="https://spacelift.io/blog/what-is-terraform" target="_blank" rel="noopener">Terraform</a> is an open-source software tool that allows us to safely and predictably manage infrastructure at scale using cloud-agnostic and infrastructure as code principles. It is a powerful tool developed by Hashicorp that enables infrastructure provisioning both on the cloud and on-premises. 
Terraform is written in a declarative configuration language, Hashicorp Configuration Language (HCL), and facilitates the automation of infrastructure management in any environment. It allows IT professionals to collaborate and perform changes safely on cloud environments and scale them on-demand according to the business needs. 
<a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Modules</a> provide excellent reusability and code-sharing opportunities to boost the collaboration and productivity of teams operating on the cloud. <a href="https://spacelift.io/blog/terraform-providers" target="_blank" rel="noopener">Providers</a> are plugins that offer integration and interaction with different APIs and are one of the main ways to extend Terraform’s functionality. 
Terraform keeps an internal <a href="https://www.terraform.io/language/state" target="_blank" rel="nofollow noopener">state</a> of the managed infrastructure, which represents resources, configuration, metadata, and their relationships. The state is actively maintained by Terraform and utilized to create plans, track changes, and enable modifications of infrastructure environments. The state should be stored remotely to allow teamwork and collaboration as a best practice. 
The core Terraform workflow consists of three concrete stages. First, we generate the infrastructure as code configuration files representing our environment&#8217;s desired state. Next, we check the output of the generated plan based on our manifests. After carefully reviewing the changes, we apply the plan to provision infrastructure resources.

terraform-vs-kubernetes-terraform-workflow

terraform vs kubernetes - terraform workflow

<a href="https://kubernetes.io/" target="_blank" rel="nofollow noopener">Kubernetes</a> (K8s) is an open-source system for container orchestration, automating deployments, and managing containerized apps. Its powerful orchestration system enables applications to scale seamlessly and achieve high availability. It has been designed and developed by Google, leveraging its vast experience in running and maintaining critical workloads in production.
Kubernetes strives to be cloud agnostic at its core, providing great flexibility in running workloads across cloud and on-premises environments. Additionally, it is designed with extensibility in mind, providing the option to add features and custom tooling to clusters easily.  
One of its main benefits is the self-healing capabilities it provides. Containers that fail are automatically restarted and rescheduled, nodes can be configured to be automatically replaced, and traffic is served only by healthy components based on health checks.  
Rollouts are handled progressively, and Kubernetes provides smart mechanisms to monitor application health during deployments. Rolling back problematic changes happens automatically if the application health doesn’t report a healthy status after a new deployment. Keeping the application running while rolling out new software versions has been a hot topic in the Kubernetes ecosystem over the past years, with many possible deployment strategies.
Kubernetes handles service discovery and load-balances traffic between similar pods natively without the need for complex external solutions. It has extendable built-in mechanisms to manage configuration and secrets for your applications. Scaling your applications has never been easier since it provides autoscaling options, scaling through commands, or via a UI. 
Kubernetes provides a cluster of nodes, a group of worker machines that run containerized applications. Each node hosts pods that hold application workload containers. The brain of the whole system is the control plane. Each cluster consists of several components that manage the worker nodes and pods and guarantee operational continuity. 
The API server is the component exposing the Kubernetes API and operates as the front-end of the control plane by handling all the communication between other parts. The etcd component is used to store all cluster data and state. The scheduler manages how pods are assigned to nodes and takes all the workload scheduling decisions. The controller manager components run different controller processes to ensure that the cluster&#8217;s desired state matches its current state. The cloud controller manager integrates Kubernetes clusters with external cloud providers, embeds their logic, and links the Kubernetes API with the Cloud Provider’s API. On each node, the kubelet is the agent responsible for running containers in pods, and kube-proxy is the component that adds the necessary networking capabilities for communication between pods and nodes.

terraform-vs-kubernetes-kubernetes-cluster

terraform vs kubernetes - kubernetes cluster

Check out this article to learn more about the <a href="https://spacelift.io/blog/kubernetes-cluster" target="_blank" rel="noopener">Key Kubernetes Cluster Components</a>.

These two modern technologies have many similarities but also fundamental differences. Let’s look into some of them in more detail.
<h3>1) Area of Focus</h3>
First and foremost, Terraform and Kubernetes have different purposes and try to solve different problems. Terraform focuses on provisioning infrastructure components and targets the Infrastructure as Code space. Kubernetes aims to enable us to run container workloads and targets the container orchestration space. 
<h3>2) Configuration Language and CLI</h3>
Manifests in Terraform are written in HCL language, while in Kubernetes in YAML or JSON. Each tool has its own command line utility and tool-specific internals to understand before being productive. 
<h3>3) Tool workflow</h3>
The Terraform workflow is generally considered easy to understand and provides a welcoming experience for new users. On the other hand, to be effective in running applications in Kubernetes, one has to understand a lot of cluster internal components and mechanics, and usually, it takes more time for users to get up to speed with Kubernetes. 
<h3>4) Configuration Drift &amp; Planning Phase</h3>
Terraform provides a native way to detect and inform you about configuration drift and unwanted changes by leveraging the planning phase of the typical workflow. In contrast, Kubernetes doesn’t support this functionality out of the box.

terraform-vs-kubernetes-2

terraform vs kubernetes

<h3>1) DevOps Tools</h3>
Both tools operate in the DevOps space and are typically set up and configured by the same type of IT practitioners; Site Reliability, DevOps, and Cloud engineers. 
<h3>2) Open Source &amp; Cloud Agnostic</h3>
Both tools are open source with various contributions by their online communities. They also take a similar approach to striving to be as cloud, platform, and API agnostic as possible to accommodate workloads across different environments. Even though they try to keep the core of the projects agnostic to external providers, both tools have mature and actively maintained integrations with the most common cloud providers.
<h3>3) Declarative Configuration</h3>
Although they use different languages, Terraform and Kubernetes take a similar approach conceptually to define the configuration. Manifests in both tools are written with the declarative approach. 
<h3>4) State Management</h3>
The notion of the state exists in both tools, although it is implemented differently. Terraform and Kubernetes apply some logic to reconcile the desired state configured in declarative configuration files with the running state.
<h3>5) Extensibility</h3>
Both tools are highly extensible by leveraging external plugins, connecting to external APIs, or defining custom resources if necessary. 
<h3>6) Well-Suited for Scale</h3>
Terraform and Kubernetes are battle-tested technologies that can support huge scale since they are designed and architected with scaling considerations for modern cloud-native environments.
<h3>7) CI/CD Compatibility</h3>
Since both tools offer highly and easily automatable workflows, they can be integrated and combined with CI/CD pipelines to automate their lifecycle.

By decomposing all the information we discussed, we realize that Kubernetes and Terraform complement each other since they operate at two different levels and can be utilized in parallel. 
A typical model that cloud practitioners adopt is to use Terraform to provision infrastructure resources (e.g. Kubernetes clusters) and use Kubernetes to manage the containerized apps that run on top of the clusters. 
Terraform’s approach simplifies and standardizes the complex task of provisioning Kubernetes clusters. Terraform, in this case, enables a unified flow for provisioning Kubernetes clusters across providers with a declarative approach that is preferred over command line utilities. This approach works great, but users must use separate flows to manage infrastructure and application resources. 
Another approach is to use Terraform to manage Kubernetes-specific application components as well. This model has the advantage of adding the Terraform workflow to Kubernetes components. This way, IT operators can detect configuration drift on Kubernetes and manage infrastructure and application resources with the same workflow and configuration language. 
This approach has a significant disadvantage since Terraform requires a well-defined schema for each managed resource. Thus each Kubernetes resource needs to be translated into a Terraform schema to be available. This dependency makes maintaining Kubernetes resources through Terraform cumbersome at times.

Spacelift supports both <a href="https://docs.spacelift.io/vendors/terraform" target="_blank" rel="noopener">Terraform</a> and <a href="https://docs.spacelift.io/vendors/kubernetes" target="_blank" rel="noopener">Kubernetes</a> and enables users to create <a href="https://docs.spacelift.io/concepts/stack" target="_blank" rel="noopener">stacks</a> based on them. Leveraging Spacelift, you can build CI/CD pipelines to combine them and get the best of each tool. This way, you will use a single tool to manage your Terraform and Kubernetes resources lifecycle, allow your teams to collaborate easily, and add some necessary security controls to your workflows.
You could, for example, deploy Kubernetes clusters with Terraform stacks and then, on separate Kubernetes stacks, deploy your containerized applications to your clusters. With this approach, you can easily integrate drift detection into your Kubernetes stacks and enable your teams to <a href="https://docs.spacelift.io/concepts/stack/organizing-stacks" target="_blank" rel="noopener">manage all your stacks from a single place</a>. 
To take this one step further, you could add <a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">custom policies</a> to harden the security and reliability of your configurations and deployments. Spacelift provides different types of policies and workflows easily customizable to fit every use case. You could, for instance, add <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">plan policies</a> to restrict or warn about security or compliance violations or <a href="https://docs.spacelift.io/concepts/policy/approval-policy" target="_blank" rel="noopener">approval policies</a> to add an approval step during deployments. The possibilities are endless with Spacelift since it provides a great way to blend Terraform and Kubernetes and enhance their capabilities with extra functionality. 
Take a look at the <a href="https://docs.spacelift.io/getting-started" target="_blank" rel="noopener">Getting Started Guide</a> to liftoff with Spacelift!

We delved into two of the most used modern DevOps tools, Kubernetes and Terraform. We discovered what makes each of them appealing and what functionalities they provide to IT operators and developers. We discussed their similarities, differences, and synergies and explored ways to combine them with Spacelift.
Thank you all for reading, and I hope you enjoyed this as much as I did.

Spacelift allows you to automate, audit, secure, and continuously deliver your infrastructure.  It helps overcome common state management issues and adds several must-have features for infrastructure management.

Provision a Kubernetes Cluster with Spacelift and Terraform

Terraform vs. AWS CloudFormation : The Ultimate Comparison

See the comparison of Terraform and Kubernetes, the most dominant tools in the cloud infrastructure space. Learn about their similarities and differences. 

78-terraform-vs-kubernetes

Terraform vs Kubernetes

Terraform vs. Kubernetes : Key Differences and Comparison

Ansible Modules: Basics & How to Use Them Efficiently

This article delves into Ansible modules, one of the core building blocks of Ansible. In this blog post, we will examine the purpose and usage of modules, along with information on how to build them and best practices. 
If you are interested in other Ansible concepts, <a href="https://spacelift.io/blog/ansible" target="_blank" rel="noopener">these Ansible tutorials</a> posted on Spacelift’s blog might be helpful for you.

Modules represent distinct units of code, each one with specific functionality. Basically, they are standalone scripts written for a particular job and are used in tasks as their main functional layer.
We build Ansible modules to abstract complexity and provide end-users with an easier way to execute their automation tasks without needing all the details. This way, some of the cognitive load of more complex tasks is abstracted away from Ansible users by leveraging the appropriate modules. 
Here’s an example of a task using the apt package manager module to install a specific version of Nginx.

Modules can be executed as well directly from the command line. Here’s an example of running the ping module against all the database hosts from the command line.

A well-designed module provides a predictable and well-defined interface that accepts arguments that make sense and are consistent with other modules. Modules take some arguments as input and return values in JSON format after execution. 
Ansible modules should follow <a href="https://docs.ansible.com/ansible/latest/reference_appendices/glossary.html#term-Idempotency" target="_blank" rel="nofollow noopener">idempotency principles</a>, which means that consecutive runs of the same module should have the same effect if nothing else changes. Well-designed modules detect if the current and desired state match and avoid making changes if that’s the case. 
We can utilize handlers to control the flow execution of modules and tasks in a <a href="https://spacelift.io/blog/ansible-playbooks" target="_blank" rel="noopener">playbook</a>. Modules can trigger additional downstream modules and tasks by notifying specific handlers.
As mentioned, modules return data structures in JSON data. We can store these return values in variables and use them in other tasks or display them to the console. Look at the <a href="https://docs.ansible.com/ansible/latest/reference_appendices/common_return_values.html" target="_blank" rel="nofollow noopener">common return values</a> for all modules to get an idea. 
For custom modules, the return values should be documented along with other useful information for the module. The command-line tool <code>ansible-doc</code> displays this information. 
Here’s an example output of running the ansible-doc command.

ansible-doc-ansible-modules

run ansible-doc

In the latest versions of Ansible, most modules are part of <a href="https://docs.ansible.com/ansible/latest/user_guide/collections_using.html" target="_blank" rel="nofollow noopener">collections</a>, a distribution format that includes roles, modules, plugins, and playbooks. Many of the core modules we use extensively are part of the <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/index.html#plugins-in-ansible-builtin" target="_blank" rel="nofollow noopener">Ansible.Builtin</a> collection. To find other available modules have a look at the <a href="https://docs.ansible.com/ansible/latest/collections/index.html#list-of-collections" target="_blank" rel="nofollow noopener">Collection docs</a>.

In this part, we explore some of the most used and helpful modules, and for each, we provide a working example. The modules in this list are picked based on their popularity within the Ansible community and functionality to perform everyday automation tasks.
<h3>Package Manager Modules yum &amp; apt</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.html" target="_blank" rel="nofollow noopener">apt module</a> is part of ansible-core and manages apt packages for Debian/Ubuntu Linux distributions. Here’s an example that updates the repository cache and updates the Nginx package to the latest version:

The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/yum_module.html" target="_blank" rel="nofollow noopener">yum module</a> is also part of ansible-core and manages packages with yum for RHEL/Centos/Fedora Linux distributions. Here’s the same example as above with the yum module:

<h3>Service Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/service_module.html" target="_blank" rel="nofollow noopener">service module</a> controls services on remote hosts and can leverage different init systems depending on their availability in a system. This module provides a nice abstraction layer for underlying service manager modules. Here’s an example of restarting the docker service:

<h3>File Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html" target="_blank" rel="nofollow noopener">file module</a> handles operations to files, symlinks, and directories. Here’s an example of using this module to create a directory with specific permissions:

<h3>Copy Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html" target="_blank" rel="nofollow noopener">copy module</a> copies files to the remote machine and handles file transfers or moves within a remote system. Here’s an example of copying a file to the remote machine with permissions set:

<h3>Template Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html" target="_blank" rel="nofollow noopener">template module</a> assists us to template files out to target hosts by leveraging the <a href="https://jinja.palletsprojects.com/en/3.1.x/" target="_blank" rel="nofollow noopener">Jinja2 templating language</a>. Here’s an example of using a template file and some set Ansible variables to generate an Nginx configuration file:

<h3>Lineinfile &amp; Blockinfile Modules</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/lineinfile_module.html" target="_blank" rel="nofollow noopener">lineinfile module</a> adds, replaces, or ensures that a particular line exists in a file. It’s pretty common to use this module when we need to update a single line in configuration files.

The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/blockinfile_module.html" target="_blank" rel="nofollow noopener">blockinfile module</a> inserts, updates, or removes a block of lines from a file. It has the same functionality as the previous module but is used when you want to manipulate multi-line text blocks.

<h3>Cron Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/cron_module.html" target="_blank" rel="nofollow noopener">cron module</a> manages crontab entries and environment variables entries on remote hosts.

<h3>Wait_for Module</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_module.html" target="_blank" rel="nofollow noopener">wait_for module</a> provides a way to stop the execution of plays and wait for conditions, amount of time to pass, ports to become open, processes to finish, files to be available, strings to exist in files, etc.

<h3>Command &amp; Shell Modules</h3>
The <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/command_module.html" target="_blank" rel="nofollow noopener">command</a> and <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/shell_module.html" target="_blank" rel="nofollow noopener">shell</a> modules execute commands on remote nodes. Their main difference is that the command module bypasses the local shell, and consequently, variables like $HOSTNAME or $HOME aren’t available, and operations like “&lt;”, “&amp;” don’t work. If you need these features, you have to use the shell module. 
On the other hand, the remote local environment won’t affect the command module, so its outcome is considered more predictable and secure. 
Usually, it’s always preferred to use specialized Ansible modules to perform tasks instead of command and shell. There are cases, though, where you won’t be able to get the functionality that you need from specialized modules, and you will have to use one of these two. Use command and shell with care, and always try to check if there is a specialized module that can serve you better before relying on them.

For advanced users, there is always the option to develop custom modules if they have needs that can’t be satisfied by existing modules. Since modules always should return JSON data, they can be written in any programming language. 
Before jumping into module development, ensure that a similar module doesn&#8217;t exist to avoid unnecessary work. Αdditionally, you might be able to combine different modules to achieve the functionality you need. In this case, you might be able to replicate the behavior you want by creating a role that leverages other modules. Another option is to use <a href="https://docs.ansible.com/ansible/latest/plugins/plugins.html" target="_blank" rel="nofollow noopener">plugins</a> to enhance Ansible’s basic functionality with logic and new features accessible to all modules.
Next, we will go through an example of creating a custom module that takes as input a string that represents an epoch timestamp and converts it to its human-readable equivalent of type <a href="https://docs.python.org/3/library/datetime.html" target="_blank" rel="nofollow noopener">datetime</a> in Python. You can find the code for this tutorial <a href="https://github.com/spacelift-io-blog-posts/Blog-Technical-Content/tree/master/ansible-modules" target="_blank" rel="nofollow noopener">on this repository</a>.
First, let’s create a <code>library</code> directory on the top of our repository to place our custom module. Playbooks with a <code>./library</code> directory relative to their YAML file can add custom ansible modules that can be recognized in the ansible module path. This way, we can group custom modules and their related playbooks. 
We create our custom Python module <code>epoch_converter.py</code> inside the library directory. This simple module takes as input the argument <code>epoch_timestamp</code> and converts it to datetime type. We use another argument, <code>state_changed</code>, to simulate a change in the target system by this module.
library/epoch_converter.py

To test our module, let’s create a <code>test_custom_module.yml</code> playbook in the same directory as our <code>library</code> directory.
test_custom_module.yml

Last stop, let’s execute the playbook to test our custom module. Since we opted to set the <code>state_changed</code> argument, we expect the task state to appear as <code>changed</code> and displayed in yellow.

ansible-modules-example

ansible modules example

If you wish to contribute to an existing Ansible collection or create and publish a new one with your custom modules, look at <a href="https://docs.ansible.com/ansible/latest/dev_guide/developing_collections_distributing.html#publishing-your-collection" target="_blank" rel="nofollow noopener">Distributing collections</a> and <a href="https://docs.ansible.com/ansible/latest/community/index.html#ansible-community-guide" target="_blank" rel="nofollow noopener">Ansible Community Guide</a>, where you can find information on how to configure and distribute Ansible content.

Use specialized modules over shell or command: Although It might be tempting to use the <code>shell</code> or <code>command</code> module often, it’s considered a best practice to leverage more specific modules for each job. Specialized modules are typically recommended because they implement the concept of desired state and idempotency, have been tested, and fulfill basic standards, like error handling.
Specify arguments when it makes sense: Some module arguments have default values that can be omitted. To be more transparent and explicit, we can opt to specify some of these arguments like the <code>state</code> in our playbook definitions.
Prefer multi-tasks in a module over loops: The most efficient way of defining a list of similar tasks, like installing packages, is to use multiple tasks in a single module.

The above method should be preferred over the <code>loop</code> or defining multiple separate tasks using the same module.
Custom modules should be simple and tackle a specific job: If you decide to build your own module, focus on solving a particular problem. Each module should have a concise functionality, be as simple as possible, and perform one thing well. If what you try to achieve goes beyond the scope of a single module, consider <a href="https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_in_groups.html#developing-modules-in-groups" target="_blank" rel="nofollow noopener">developing a new collection</a>.
Custom modules should have predictable parameters: Try to enable others to use your module by defining a transparent and predictable user interface. The arguments should be well-scoped and understandable, and their structures should be as simple as possible. Follow the typical convention of parameter names in lowercase and use underscores as the word separator.
Document and test your custom modules: Every custom module should include examples, explicitly document dependencies, and describe return responses. New modules should be tested thoroughly before releasing. You can create roles and playbooks to test your custom modules and different test cases.

We deep-dived into Ansible modules and examined their use and functionality in detail. We discussed best practices and showed practical examples of leveraging the most commonly-used modules. Lastly, we went through a complete example of developing a custom module. 
Thank you for reading, and I hope you enjoyed this article as much as I did.

Ansible Tutorial for Beginners: Ultimate Playbook & Examples

Ansible Roles : Basics & How to Combine Them With Playbooks

Modules are one of the core building blocks of Ansible. Learn how to build them and see best practices. Explore some of the most used and helpful modules.

81-ansible-modules

Ansible Modules - How To Use Them Efficiently (Examples)

DevOps Best Practices (And How Not To Do DevOps)

General | Spacelift Blog

General

Utilizing DevOps practices to maximize speed and value creation has been a hot topic in the software industry for the past decade. We have embraced these practices and changed how we work and think about development, operations, project management, code quality, observability, and continuous feedback. 
As organizations started applying these practices, we noticed many anti-patterns emerging. In this article, we will see some DevOps best practices and ways to improve our workflows, but also we will explore some of the typical DevOps anti-patterns and how to avoid them.

Probably you have seen hundreds of different definitions of DevOps by now. For me, DevOps is a set of best practices around the software development lifecycle and the effort to continually improve and deliver value more efficiently.  
On its basis, DevOps is a culture that spreads equally between developers and operations rather than a specific role. In reality, the term has been used as an umbrella term to characterize the engineering roles of cloud-savvy people who share the pains and responsibilities of devs and ops. At the same time, they strive to enable and promote DevOps practices within organizations. 
So why all this fuss? Implementing these practices has proven to improve software quality. Different software and operations teams collaborate more efficiently, reduce friction and lead time, integrate and test their code continuously, and deploy more often.
It&#8217;s all about finding the inefficiencies in our workflows and building a culture of continuous communication and trust. Other aspects of it are handling failures and unplanned work, leveraging automation, and focusing significantly on observability to get meaningful feedback.

Now that we set the foundation let&#8217;s explore some DevOps best practices without further ado. The list isn&#8217;t supposed to be exhaustive but rather a guide with hints and pointers that will ease your journey towards adopting a healthy DevOps culture. 
<h3>1. Foster a Culture of Collaboration and Blameless Communication</h3>
First and foremost, for this journey to be successful, we must focus intensely on growing a culture that allows people to collaborate freely and remove the fear of failure. Organizations and teams that promote values like trust and empathy tend to have a heads-up advantage in adopting DevOps practices. Break down the silos between teams and make them work together towards a common goal, bringing value to the company. One of the tools that deliver an <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bdevops_best_practices%7D" target="_blank" rel="noopener">enhanced collaboration layer for IaC</a><a href="https://spacelift.io/" target="_blank" rel="noopener"> </a>is Spacelift. At Spacelift, you can invite security and compliance teams to collaborate on and approve workflows and policies.
<h3>2. Adopt Continuous Integration and Delivery (CI/CD)</h3>
Integrating small batches of code frequently into a central code repository is a practice that allows developers to collaborate efficiently. With this approach, the repository is always kept in a good state since we introduce small changes that are easier to handle. Continuous Integration (CI) enables early error detection and improves code quality since these small batches of changes are validated each time with automated builds and tests. 
The next step after integrating our code is deploying it to our environment. Continuous Delivery (CD) is the practice of getting the code into a deployable state continuously for every small batch of change. This simplifies our deployments and provides our developers with an easy and automated method to push code to production.
<h3>3. Set up Automated Testing</h3>
A continuation of the previous point and an integral part of DevOps success is setting up and curating meaningful automated tests as part of our CI/CD pipelines. This way, we don&#8217;t rely on humans to run manual tests on our code; instead, we set up automated tests that run on every minor change introduced. 
By increasing the testing frequency and the number of tests, we reduce our chances of introducing bugs to production systems. The tests vary depending on the use case but typically could include unit testing, integration testing, end-to-end testing, load testing, smoke testing, etc. 
<h3>4. Focus on Observability and Find the Right Metrics</h3>
DevOps practices are based on getting feedback and continuously improving our processes. We need to find and track the right metrics to achieve that and measure our results. Figuring out the right metrics is an arduous journey that each organization has to go through. 
These metrics will differ from organization to organization and from team to team, depending on the goals and key results targeted. Still, it&#8217;s a crucial exercise for achieving success. Some typical examples of DevOps metrics are deployment time, frequency of deploys, deployment failure rate, availability of critical services, mean time to detect, mean time to restore, cost per unit, code coverage, and change lead time. 
Moving one step forward, we also have to focus on the observability of our apps and software running in production. We have to define a strategy for effectively storing, managing, and distributing logs, traces, and metrics of our applications to quickly solve issues, improve the understandability of our systems and enable our teams to operate efficiently. 
<h3>5. Avoid Manual Work with Automation</h3>
By reducing manual work and automating recurring tasks, we accelerate our processes and provide increased consistency to our results. Automation can allow us to focus on what is essential and avoid human intervention and time spent on chores. It also provides more confidence in our systems and processes, removes human errors and miscommunication, and speeds up the performance of teams. If you need an automation layer for your cloud resources, take a look at <a href="https://spacelift.io/?utm_source=blog&amp;utm_medium=text&amp;utm_id=blogpost&amp;utm_content=%7Bdevops_best_practices%7D" target="_blank" rel="noopener">Spacelift&#8217;s self-service infrastructure</a> (automated workflow management feature especially).
<h3>6. Incorporate Security Early in the Development Lifecycle</h3>
Security shouldn&#8217;t be one of the last things to integrate into software development. The birth of DevSecOps emphasizes thinking about application and infrastructure security early in the development lifecycle, incorporating security into the initial design and integrating it into the CI/CD pipelines. 
Security should be a responsibility shared among different teams and through the entire application lifecycle and be considered an integral part of the process, not an optional add-on. Recently, a strong focus has been given to securing the software supply chain due to increased malicious attacks over the last years.
In the world of infrastructure, even the tiniest of mistakes can cause major outages. That&#8217;s why Spacelift adds <a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">an extra layer of </a><a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">policy</a><a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener"> that allows you to control</a> &#8211; separately from your infrastructure project &#8211; what code can be executed, what changes can be made, when and by whom. This isn&#8217;t only useful to protect yourself from the baddies but allows you to implement an automated code review pipeline<span data-key="3dc1c9be419548e2b421064b7e513810" data-slate-fragment="JTdCJTIyb2JqZWN0JTIyJTNBJTIyZG9jdW1lbnQlMjIlMkMlMjJkYXRhJTIyJTNBJTdCJTdEJTJDJTIybm9kZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJibG9jayUyMiUyQyUyMnR5cGUlMjIlM0ElMjJwYXJhZ3JhcGglMjIlMkMlMjJpc1ZvaWQlMjIlM0FmYWxzZSUyQyUyMmRhdGElMjIlM0ElN0IlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMmluJTIwdGhlJTIwd29ybGQlMjBvZiUyMGluZnJhc3RydWN0dXJlJTJDJTIwZXZlbiUyMHRoZSUyMHRpbmllc3QlMjBvZiUyMG1pc3Rha2VzJTIwY2FuJTIwY2F1c2UlMjBtYWpvciUyMG91dGFnZXMlMjAtJTIwbGlrZSUyMHRoYXQlMjB0eXBvJTIwd2UlMjBvbmNlJTIwbWFkZSUyMGluJTIwb3VyJTIwRE5TJTIwY29uZmlnLiUyMFRoYXQncyUyMHdoeSUyMFNwYWNlbGlmdCUyMGFkZHMlMjBhbiUyMGV4dHJhJTIwbGF5ZXIlMjBvZiUyMCUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMmFjZDUyMmQwMmEzODQzZWNhZTM2NTljNTY2NTE3Nzc2JTIyJTdEJTJDJTdCJTIyb2JqZWN0JTIyJTNBJTIyaW5saW5lJTIyJTJDJTIydHlwZSUyMiUzQSUyMmxpbmslMjIlMkMlMjJpc1ZvaWQlMjIlM0FmYWxzZSUyQyUyMmRhdGElMjIlM0ElN0IlMjJyZWYlMjIlM0ElN0IlMjJraW5kJTIyJTNBJTIycGFnZSUyMiUyQyUyMnBhZ2UlMjIlM0ElMjItTHhwdG9mZ080RXFWcUtzbXVpSSUyMiU3RCU3RCUyQyUyMm5vZGVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIydGV4dCUyMiUyQyUyMmxlYXZlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMmxlYWYlMjIlMkMlMjJ0ZXh0JTIyJTNBJTIycG9saWN5JTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyMjY1N2RhZDlmZWMyNGI5ODgxOWQ2MDFjYTZjYzI1MzElMjIlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjIwNWZlMzliYzE1NzQ0OTE0YjQ2Yzk2MmFkYjZkNzA0YiUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMiUyMHRoYXQlMjBhbGxvd3MlMjB5b3UlMjB0byUyMGNvbnRyb2wlMjAtJTIwc2VwYXJhdGVseSUyMGZyb20lMjB5b3VyJTIwaW5mcmFzdHJ1Y3R1cmUlMjBwcm9qZWN0ISUyMC0lMjAlMjIlMkMlMjJtYXJrcyUyMiUzQSU1QiU1RCUyQyUyMnNlbGVjdGlvbnMlMjIlM0ElNUIlNUQlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjJlZGIzYmEzZjdhMzA0MzRmYTBmNWEzZWRkMjViZmQyZCUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMmlubGluZSUyMiUyQyUyMnR5cGUlMjIlM0ElMjJsaW5rJTIyJTJDJTIyaXNWb2lkJTIyJTNBZmFsc2UlMkMlMjJkYXRhJTIyJTNBJTdCJTIycmVmJTIyJTNBJTdCJTIya2luZCUyMiUzQSUyMnBhZ2UlMjIlMkMlMjJwYWdlJTIyJTNBJTIyLUx4cHU2QjhOQ2FpbE1lOTYtamUlMjIlN0QlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMndoYXQlMjBjb2RlJTIwY2FuJTIwYmUlMjBleGVjdXRlZCUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjZiM2Q2MjRhZmI4NTRkMDI4Y2EzMDRkMjQ0OGMyYTdlJTIyJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyODliYmFjYTcwZTAyNGI3ZGJiNWY2ZGNmMDIzMjM3MDclMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJ0ZXh0JTIyJTJDJTIybGVhdmVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIybGVhZiUyMiUyQyUyMnRleHQlMjIlM0ElMjIlMkMlMjAlMjIlMkMlMjJtYXJrcyUyMiUzQSU1QiU1RCUyQyUyMnNlbGVjdGlvbnMlMjIlM0ElNUIlNUQlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjIwOGU5MWYxOTI1ZjQ0MGI5YjdhNzgwZjlkYzhmMzY1YiUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMmlubGluZSUyMiUyQyUyMnR5cGUlMjIlM0ElMjJsaW5rJTIyJTJDJTIyaXNWb2lkJTIyJTNBZmFsc2UlMkMlMjJkYXRhJTIyJTNBJTdCJTIycmVmJTIyJTNBJTdCJTIya2luZCUyMiUzQSUyMnBhZ2UlMjIlMkMlMjJwYWdlJTIyJTNBJTIyLUx4cHU4VHAxQkZhSjUtbE5CYXAlMjIlN0QlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMndoYXQlMjBjaGFuZ2VzJTIwY2FuJTIwYmUlMjBtYWRlJTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyODk2ODAxMzIwYmYzNDBiMDkyYjllZGQwZmM3NGNkYTclMjIlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjJlMDE5ZDE4ZmI5ZWI0NzYzOTM1YjliOTczMDZjNzgwYSUyMiU3RCUyQyU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMiUyQyUyMCUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjYyYzNmOGNhN2MwYzRiMWE5MjU0NWEyODQ5MGNjOTQ2JTIyJTdEJTJDJTdCJTIyb2JqZWN0JTIyJTNBJTIyaW5saW5lJTIyJTJDJTIydHlwZSUyMiUzQSUyMmxpbmslMjIlMkMlMjJpc1ZvaWQlMjIlM0FmYWxzZSUyQyUyMmRhdGElMjIlM0ElN0IlMjJyZWYlMjIlM0ElN0IlMjJraW5kJTIyJTNBJTIycGFnZSUyMiUyQyUyMnBhZ2UlMjIlM0ElMjItTHhwdHhHbTRrNnFRa0g1NTduSiUyMiU3RCU3RCUyQyUyMm5vZGVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIydGV4dCUyMiUyQyUyMmxlYXZlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMmxlYWYlMjIlMkMlMjJ0ZXh0JTIyJTNBJTIyd2hlbiUyMGFuZCUyMGJ5JTIwd2hvbSUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjVjY2JkZDBkZTdjNDRiODNiMTYwZTNkYzU3MWVlZGU4JTIyJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyMDk5NDFlZTlhMzk0NDZjNDk2ODc4YTQ3NGM2MmU2YTElMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJ0ZXh0JTIyJTJDJTIybGVhdmVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIybGVhZiUyMiUyQyUyMnRleHQlMjIlM0ElMjIuJTIwVGhpcyUyMGlzbid0JTIwb25seSUyMHVzZWZ1bCUyMHRvJTIwcHJvdGVjdCUyMHlvdXJzZWxmJTIwZnJvbSUyMHRoZSUyMGJhZGRpZXMlMkMlMjBidXQlMjBhbGxvd3MlMjB5b3UlMjB0byUyMGltcGxlbWVudCUyMGFuJTIwJTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyZWUyZTRmNzVhNjRhNDk5NDhiZmIxN2JkMjU1MmUyYmUlMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJpbmxpbmUlMjIlMkMlMjJ0eXBlJTIyJTNBJTIybGluayUyMiUyQyUyMmlzVm9pZCUyMiUzQWZhbHNlJTJDJTIyZGF0YSUyMiUzQSU3QiUyMnJlZiUyMiUzQSU3QiUyMmtpbmQlMjIlM0ElMjJhbmNob3IlMjIlMkMlMjJhbmNob3IlMjIlM0ElMjJhdXRvbWF0ZWQtY29kZS1yZXZpZXclMjIlMkMlMjJwYWdlJTIyJTNBJTIyLUx4cHU4VHAxQkZhSjUtbE5CYXAlMjIlN0QlN0QlMkMlMjJub2RlcyUyMiUzQSU1QiU3QiUyMm9iamVjdCUyMiUzQSUyMnRleHQlMjIlMkMlMjJsZWF2ZXMlMjIlM0ElNUIlN0IlMjJvYmplY3QlMjIlM0ElMjJsZWFmJTIyJTJDJTIydGV4dCUyMiUzQSUyMmF1dG9tYXRlZCUyMGNvZGUlMjByZXZpZXclMjBwaXBlbGluZSUyMiUyQyUyMm1hcmtzJTIyJTNBJTVCJTVEJTJDJTIyc2VsZWN0aW9ucyUyMiUzQSU1QiU1RCU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjRkNWQ2ZDhmYzZhNzRmZjY4ZmFjYTc2ZmQ2YTc0MGNhJTIyJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyYzM5MTNjMzkwZDNjNGNkOWJkZWM0NzVjZDMwNzNjYTQlMjIlN0QlMkMlN0IlMjJvYmplY3QlMjIlM0ElMjJ0ZXh0JTIyJTJDJTIybGVhdmVzJTIyJTNBJTVCJTdCJTIyb2JqZWN0JTIyJTNBJTIybGVhZiUyMiUyQyUyMnRleHQlMjIlM0ElMjIuJTIyJTJDJTIybWFya3MlMjIlM0ElNUIlNUQlMkMlMjJzZWxlY3Rpb25zJTIyJTNBJTVCJTVEJTdEJTVEJTJDJTIya2V5JTIyJTNBJTIyM2RjMWM5YmU0MTk1NDhlMmI0MjEwNjRiN2U1MTM4MTAlMjIlN0QlNUQlMkMlMjJrZXklMjIlM0ElMjJkZmM2OTFkMTU3NDQ0NGQwOTIxMjgyMzY3MjYwYTFjZCUyMiU3RCU1RCUyQyUyMmtleSUyMiUzQSUyMjYzMjdmOWYxOWE0YjQwNTA5ZDE0NmUxMDVlMTg5YzcwJTIyJTdE">.
<h3>7. Learn from Incidents and Build Processes around Them</h3>
Incidents are inevitable in the IT world. It doesn&#8217;t matter how well prepared your team is; eventually, there will be an incident that you will have to address. In these cases, it&#8217;s essential to focus on blameless communication, understand the issue, communicate effectively with the affected parties, and collaborate to find a solution. 
Equally crucial to fixing the issue is to have a process to log incidents and learn from them. After the incident has been tackled, spend some time with your team to craft a post-incident review, and discuss how the incident was handled. Try to find any possible improvements in the incident handling process that can help you next time. 
<h3>8. Focus first on Concepts, then Find the Right Tools</h3>
The DevOps landscape is moving extremely fast, with new tools and services emerging daily. Instead of continuously integrating new shiny tools and services, concentrate on understanding the core concepts that allow companies to accelerate their business with DevOps practices. 
Only once you understand the concepts and prioritize the missing pieces accordingly, you will be successful in selecting the right tools for the job. Remember, you won&#8217;t be able to build everything within your team. It&#8217;s ok to rely on <a href="https://spacelift.io/blog/infrastructure-as-code#infrastructure-as-code-tooling" target="_blank" rel="noopener">DevOps tooling</a> and managed services when it makes sense. Be smart about how you use your team&#8217;s time, try to understand your in-house expertise and needs, put effort into building custom tooling when it makes sense, and rely on external tooling and services for the rest. 
<h3>9. Embrace Infrastructure as Code (IaC) and Push for a Self-Service Infra Model</h3>
Cloud infrastructure should be considered an integral part of software development and treated equally to application code. By leveraging <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">infrastructure as code</a>, we can incorporate the best practices we use for software development, such as version control and CI/CD, into infrastructure creation. This model removes the need to manually set up and configure resources via UIs and further strengthens our automation efforts across the IT landscape. Changes are always auditable and transparent, and we can quickly roll back infrastructure systems to a previous state when there are issues. 
Thinking one step ahead, instead of adding another bottleneck of waiting for cloud infrastructure engineers to create the necessary resources, push for a self-service infrastructure model. In this model, the developers and anyone who needs infra resources can leverage some tooling to generate the required pieces. This way, we increase productivity and speed while giving autonomy to our developers, all via a single workflow. 
Check out <a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener">how </a><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener">Spacelift</a><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener"> can improve your IaC</a>, assist your team in adopting a collaborative infrastructure model and achieving operational excellence.

Along with the rise of DevOps, we have also seen many anti-patterns emerge. During the quest to adopt DevOps practices, people have misinterpreted their scope and made mistakes that lead to common anti-patterns. Let&#8217;s look at some common challenges, traps, and misconceptions companies face while implementing DevOps principles.
<h3>1. Don&#8217;t Create Separate DevOps Team</h3>
Common mistake companies make when adopting DevOps practices is creating a separate team to handle the DevOps transformation. Unfortunately, this adds one more silo to the process and breaks the central promise of DevOps, which is increased collaboration and shared ownership between the existing teams. 
Similarly, we see operations teams rebranding into DevOps teams without actual changes in organizations&#8217; culture, communication, and collaboration. DevOps is about bringing the different groups closer, not creating a new one. 
<h3>2. Avoid Having the DevOps Hero</h3>
At times, specific team members are more involved in the DevOps practices than others. That could be due to accumulated knowledge, a higher level of experience, or increased effort by a person. When this pattern emerges, it could lead quickly to the DevOps hero anti-pattern where a specific team member becomes indispensable to the team. 
This situation is highly problematic since the team&#8217;s performance and velocity depend on a single person. At the same time, this person may face an extremely high amount of work that eventually leads to burnout and potentially leaving the company. To avoid this anti-pattern, ensure the knowledge is spread across teams and team members. Divide the work equally and don&#8217;t rely on heroes but on teamwork and rigid processes to achieve results. 
<h3>3. Don&#8217;t Attempt to Automate and Change Everything at Once</h3>
Starting from scratch to apply DevOps practices in an organization could be daunting at first. As with most things, attempting to tackle everything at once is not the way to go. First, analyze the current situation and processes within your company. People usually don&#8217;t happily accept many changes, so you need to think strategically. Prioritize the tasks accordingly, find quick wins, automate the stuff that will have a higher impact, and focus on one thing at a time.
<h3>4. Avoid Chasing New Tools</h3>
As new services and tools pop up almost every day, adopting and using these new shiny toys is always tempting. It&#8217;s common for engineers to fall into this trap of introducing a new tool just because it&#8217;s trending without proper analysis of whether it&#8217;s needed or the best choice. 
Picking the right tools for the job is critical but is a process that should be reviewed meticulously. For every new service or tool we add, we should also consider its maintainability and the operational overhead, dependencies, complexity, and new cognitive load that we introduce in the process.
<h3>5. Don&#8217;t Sacrifice Quality for Speed</h3>
Since one of the main factors of DevOps success is velocity, many teams try to speed up their processes at the cost of quality and usually security. Many of the typical DevOps metrics are based on how fast we deliver, deploy and provide value, but they are not enough by themselves as they only tell half the story. Due to this disproportionate focus on speed, it&#8217;s easy to lose perspective of what is important; delivering quality software. Treat speed and quality equally, add meaningful automated tests and avoid cutting corners just to ship faster.
<h3>6. Don&#8217;t Give up on Continuous Improvement</h3>
Applying effective DevOps practices is a dynamic process that should be curated continuously. It might be tempting to rest and relax after implementing all the DevOps best practices in the roadmap, but unfortunately, this process never stops. 
Every step of the way, we should focus on reviewing our workflows and continuously improving our systems, processes, and products. We have to set up flows of constant feedback that allow us to review and reflect on our choices and ultimately improve. New paradigms, best practices, and improved models always appear, and we should be restless if we want our teams to survive, perform, and succeed. 
<h3>7. Don&#8217;t Neglect Documentation and Information Sharing</h3>
By definition, successful adoption of DevOps practices relies on sharing information efficiently within an organization and creating a workplace where collaboration thrives organically. Unfortunately, neglecting documentation and efficient information sharing is an anti-pattern that occurs too often in software teams. Documentation, when done right, could be a handy tool for developers. 
Try to integrate documentation tasks on team backlogs and treat docs as first-class citizens within your organization. Docs aren&#8217;t static and should be kept up to date, created consistently, and accessible to anyone who needs them.

We have explored different DevOps best practices and paradigms and analyzed how we can incorporate them to accelerate team performance and value creation. We also saw some hidden traps and anti-patterns to be aware of and avoid while pursuing DevOps excellence. 
Thank you for reading, and I hope you enjoyed this article as much as I did.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence

How to Deploy your Infrastructure in CI/CD using Terraform

Explore different DevOps Best Practices and Anti-Patterns and see how you can incorporate them to accelerate your team performance and value creation.

43-devops-best-practices

Securely accessing cloud resources is one of the most critical requirements for IT teams. To achieve this, cloud administrators leverage web services that assist with managing access to cloud environments. 
In this article, we will explore the AWS IAM service, and more specifically, we will look into IAM policies, learn how they are structured, how to create them, and some best practices.

AWS Identity and Access Management(IAM) is a web service that assists with managing and controlling access and permissions to resources and other AWS services. Leveraging this service, we can set up the building blocks to control authentication and authorization to our AWS accounts. 
Before we continue talking about IAM, let’s define some terminology for its essential components that we will reference throughout the article. 
<ul>
<li style="font-weight: 400;" aria-level="1">Authentication: The process of validating a user’s identity. Basically, verifying who someone claims to be. </li>
<li style="font-weight: 400;" aria-level="1">Authorization: Defining the level of permissions and access rights for a given user. Authorization happens after authentication and establishes what someone is allowed to do.</li>
<li style="font-weight: 400;" aria-level="1">Principals: Persons or applications that try to perform actions on AWS. </li>
<li style="font-weight: 400;" aria-level="1">Users: Identities in IAM that correspond to users in an organization. They represent actual persons or applications and service accounts.</li>
<li style="font-weight: 400;" aria-level="1">Groups: IAM users are grouped logically with IAM groups. This way, we can assign permissions to multiple users in a bulk fashion. A common pattern is to create groups according to different roles in an organization.</li>
<li style="font-weight: 400;" aria-level="1">Roles: Roles provide temporary access to resources and aren’t associated with specific users. Instead, roles enable principals to temporarily assume a set of permissions to complete an operation.</li>
<li style="font-weight: 400;" aria-level="1">Policies: To manage access on AWS we generate IAM policies that define levels of permissions and attach them to IAM identities(users, groups, roles) or AWS resources.</li>
<li style="font-weight: 400;" aria-level="1">Requests: Principals attempt to perform actions on AWS by instantiating requests to AWS via the AWS API, Management Console, or CLI. </li>
<li style="font-weight: 400;" aria-level="1">Actions: Every request has an action definition that declares the specific operation requested. If authentication and authorization are cleared the action is approved. </li>
<li style="font-weight: 400;" aria-level="1">Resources: A resource in this context can be considered any AWS object within a service.</li>
</ul>

IAM Policies are one of the most basic blocks of access management in AWS since they define the permissions of an identity or a resource. For every request, these policies are evaluated, and based on their definition; the requests are allowed or denied. Let’s look at the different types of policies that exist in AWS.  
<ul>
<li style="font-weight: 400;" aria-level="1">Identity-based policies are policies attached to identities(users, groups, roles) and provide them with the required permissions. Identity-based policies could be either Managed policies or Inline policies. Managed policies are either prepared and managed by AWS for common use cases(AWS managed policies) or custom policies created by users(Customer managed policies) suitable for achieving fine-grained control. Inline policies are used when we need to make a policy part of a principal’s entity and maintain a strict one-to-one relationship between them. </li>
<li style="font-weight: 400;" aria-level="1">Resource-based policies are attached directly to resources and specify permissions for specific actions on the resource by some principals. </li>
<li style="font-weight: 400;" aria-level="1">IAM permissions boundaries define the maximum permissions for an IAM entity and are used as safeguards. </li>
<li style="font-weight: 400;" aria-level="1">Access control lists(ACLs) are attached to resources and control cross-account permissions for principals from other accounts.</li>
<li style="font-weight: 400;" aria-level="1">Organizations Service Control Policies(SCPs) specify the maximum level of permissions for an organization&#8217;s accounts. These policies are used to limit the permissions that can be assigned within member accounts.</li>
<li style="font-weight: 400;" aria-level="1">Session policies are advanced policies used during temporary sessions for roles or federated users.</li>
</ul>

Now that we have seen the different types of IAM policies that exist in the context of AWS, let&#8217;s see how they are structured. Most of them are stored in JSON format and are attached to resources or identities. The only type of policy that uses a different format is ACL, but we won’t focus on ACLs in this article. 
Each JSON policy document might have optional informational elements at the top of the document and must have one or more statements. A statement contains all the necessary information about a permission that we would like to declare. 
Here’s a simple identity-based policy that allows the principal who has it attached to list the objects of a single S3 bucket named `this-is-an-example-s3-bucket-name`.

IAM Policies are built using a combination of the below elements:
<ul>
<li>Version: Defines the version of the policy language. Always use the latest version.</li>
<li>Statement: This argument is used as a parent element for the different statements in the policy. </li>
<li>Sid: This is an optional element that allows us to define a statement ID.</li>
<li>Effect: This element can have the values `Allow` or `Deny`.</li>
<li>Action: The list of actions related to the policy.</li>
<li>Resource: Defines the list of resources to which the policy is applied. For resource-based policies, this is optional since the policy applies to the resource that has it attached. </li>
<li>Principal: Defines the identities that are allowed or denied access to resource-based policies. </li>
<li>Condition: Defines some conditions under which the policy applies. This element is practical when we need to achieve custom rules for fine-grained access. </li>
</ul>
There are options for resources, principals, and actions to define deny access controls by using NotPrincipal, NotAction, or NotResource. Note that these are mutually exclusive with their contrasting elements. For example, you can’t use both Resource and NotResource elements in a statement. 

If you want to learn more about these elements and the JSON policy components, check out the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html" target="_blank" rel="nofollow noopener">IAM JSON policy reference</a>.
Next, we will see an example of a more complex IAM policy with more than one statement using conditions.

In the policy above, we define two separate statements. The first defines some rules to deny access based on the requested AWS region. More specifically, it denies all actions for regions not defined in the condition, except for the actions mentioned in the NotAction element.
The second one defines a policy to deny access to AWS based on the source IP of the requests. Here, since we have two conditions, they are combined with a logical `AND`. More specifically, the policy denies all AWS actions when requests originate from an IP that doesn’t come from the corporate IP range(in this example: 192.0.1.0/24) and when an AWS service isn’t making the call. 
Note that both policies don’t actually allow any actions and are used to restrict access to AWS resources. 
Finally, here’s a resource-based policy for an S3 bucket.

By associating this policy with the bucket `this_is_an_example_s3_bucket_name` every action on the bucket is denied if the request isn’t authenticated with a multi-factor authentication mechanism. Note the use of the Principal element here since we are using a resource-based policy.

There are three main ways to create an IAM policy. We can either use the AWS console, the AWS CLI, or the AWS API. For this demo, we will go through the exercise of creating an IAM customer-managed policy via the AWS Console. 
The easiest option to create a policy is to use the helpful visual editor that the AWS console provides without having to write a JSON file and care about proper syntax. After signing in to the AWS Management Console, head to <a href="https://console.aws.amazon.com/iam/" target="_blank" rel="nofollow noopener">IAM</a> and select Policies and Create Policy. 
From this screen, you can choose to either use the Visual editor or JSON.

image9-8

create policy - choose a service

Let&#8217;s replicate our first example policy from above that allows listing the objects in an S3 bucket. For Service we select S3, for Actions choose ListBucket, and for Resources use the arn of the S3 bucket arn:aws:s3:::this-is-an-example-s3-bucket-name. Then, select Next: Tags.

image7-11

Create policy - s3

Note that by default, the policy allows the actions chosen. To deny actions, you have to select Switch to deny permissions before selecting the actions.

image1-26

create policy - switch to deny premissions

You can optionally add some tags to your customer-managed identity-based policy on the next page. For example, we could add tags for name, environment, department, etc. Then, select Next: Review.

image2-25

create policy - add tags

Finally, add a Name to your policy and optionally a description for its purpose and select Create policy.

image5-18

Create policy - review policy

Nice,  we have replicated the IAM policy we introduced in the first example.
If you feel comfortable with JSON syntax, you can instead use the JSON tab to define an IAM Policy like this:

image10-8

Create policy - JSON

Another option using the AWS API under the hood would be to create your IAM resources with the Infrastructure as Code tool of your choice. Here’s an example of using Terraform to define the same policy.

When creating or editing IAM policies using the AWS Management Console, their syntax and grammar are inspected to verify they comply with the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_grammar.html" target="_blank" rel="nofollow noopener">IAM policy grammar</a>. If there is any issue or error, you get notified accordingly and have to fix the policy. 
AWS provides <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html" target="_blank" rel="nofollow noopener">IAM Access Analyzer</a> with additional policy checks and recommendations to improve policies. When creating or editing a policy using the JSON tab, a policy validation pane below the policy provides different findings for the policy. There you get information for issues categorized as Security, Errors, Warnings, and Suggestions. Update your policy accordingly to resolve the findings.
Here’s an example of a malformed policy that triggers three separate findings.

image6-12

Malformed policy example

We get an Error finding because we don’t specify a correct ARN field for the Resource element.
Next, on the Warnings tab, we get notified that we omitted the top-level Version element.

image3-22

Validating policy - Warnings tab

Finally, on the Suggestions tab, we get notified that our Action element includes no actions.

image8-11

Validating policy - Suggestions tab

Another handy tool for AWS administrators and users who manage IAM policies is the <a href="https://policysim.aws.amazon.com/home/index.jsp?#" target="_blank" rel="nofollow noopener">IAM Policy Simulator</a>. Using this simulator, you can troubleshoot issues with different policy types and try to understand why some requests are allowed or denied. 
The IAM Policy Simulator console provides a testing playground for IAM policies and an easy way to test which actions are allowed or denied to specific principals for specific resources. The simulator doesn&#8217;t actually make the requests, so it&#8217;s a safe space to experiment and get information. You can even test new policies that don&#8217;t yet exist on our account and simulate real-world scenarios by defining complex conditions. Finally, since the returned output is a message with the result and relevant information, you can identify which statement in a policy denies or allows access. 
For example, we have created an IAM user test-user and attached the policy ListBucketContents that we created earlier. We can use the AWS Policy Simulator to validate that this user can indeed list the objects of the example bucket.

image4-20

AWS Policy Simulator

After defining all the parameters for the simulation, we choose Run Simulation and check that the Permission result is allowed. Check out the detailed <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_policies.html" target="_blank" rel="nofollow noopener">Troubleshooting IAM Policies guide</a> for issues with IAM policies.

A valuable feature of IAM Policies that you should be aware of is that AWS automatically keeps five versions of each policy. Whenever you make changes to a policy and save it, a new version is created instead of overwriting the existing one. 
When the limit of five kept versions is reached, you can select which older version to remove. This feature is convenient because it allows us to quickly revert to a previous policy version in case of issues.

<ul>
<li>Follow Least Privilege Principles: When creating IAM policies grant only the necessary permissions to perform the job. Specify specific actions, resources, and principles and add custom conditions to achieve the required controls. </li>
<li>Generate policies based on access activity: Although it&#8217;s considered a best practice, it might be challenging to implement fine-grained policies that grant the least privilege when starting with IAM policies. For this reason, you can initially use policies that allow more permissions than they should and then refine them by <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_generate-policy.html" target="_blank" rel="nofollow noopener">generating a new policy based on the access activity</a> of an IAM entity. This is a more pragmatic approach to achieving the least privilege for inexperienced users that makes the journey to improving security and access management smoother. </li>
<li>Get started with managed policies: It’s totally ok to start with AWS Managed policies as you start experimenting and learning. These policies cover common use cases and help your team set up necessary permissions to start quickly. When you feel comfortable enough with IAM, consider restricting your policies by creating customer-managed policies that follow the least privileged approach.</li>
<li>Assign permissions with roles and groups &amp; avoid inline policies: It’s considered best to avoid assigning permissions directly to users or using inline policies. For easier access management and better security controls, attach policies to groups or roles. </li>
<li>Validate policies: Every time you create or edit policies, validate them using the AWS helper tools, as we have seen in the section Validating IAM Policies.</li>
<li>Use IAM Roles for EC2 instances(EC2 Instance Profiles): For apps that run on EC2 instances, attach the necessary permissions to IAM roles and specify the role as a launch parameter for the instance. </li>
<li>Use policy conditions: Leverage policy conditions to achieve complex policies with custom rules. By combining different conditions in policies, we can comply with specific requirements according to our organization’s standards. Check out the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition.html" target="_blank" rel="nofollow noopener">Policy Elements: Condition</a> reference page for more information.</li>
<li>Test IAM policies with IAM Policy Simulator: Test existing and new policies quickly and without affecting any environment with the IAM Policy Simulator. Use it to understand your policies better and troubleshoot different issues.</li>
<li>Review IAM policies on a regular basis: Cloud environments aren’t static; they evolve over time. For this reason, IAM policies should be reviewed and updated to reflect changes. Following best practices and least privilege, it’s not something that you configure once and forget but requires continuous effort. </li>
<li>Tag IAM policies: Tag your IAM policies to add metadata to them the same way you tag other AWS resources. </li>
<li>Control access to resources with tags: If you have a proper tagging strategy for your AWS resources, you can create elaborate IAM policies that control access to them based on tags. This method becomes handy when you want to separate access to resources based on owners or teams.</li>
<li>Use IaC to create your IAM policies: In this tutorial, we have seen how to create IAM policies from the AWS Console, and this might be ok as a starting point, but eventually, you should treat your IAM resources as infrastructure components and apply the same Infrastructure as Code principles as any other AWS resource.</li>
</ul>

We have explored IAM concepts in the context of AWS, and more specifically, we focused on IAM policies. We saw how to create and test policies, assign necessary permissions, and combine different IAM mechanisms to achieve efficient access management controls on AWS.
<a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="noopener">Here</a> you can learn more about Spacelift integration with AWS, and <a href="https://docs.spacelift.io/concepts/policy" target="_blank" rel="noopener">here</a> about the usage of policies with Spacelift.
Thank you for reading, and I hope you enjoyed this article as much as I did.

Spacelift is a sophisticated SaaS product for Infrastructure as Code that helps DevOps develop and deploy new infrastructures or changes quickly and with confidence.

CI/CD Pipeline : Everything You Need To Know with Examples

Look into AWS IAM policies with some best practices. Learn how they are structured, how to create them, and how to assign necessary permissions.

80-iam-policies

AWS IAM Policies and Best Practices

AWS IAM Policies : Best Practices & Creating an IAM Policy

This blog post explores the concept of Ansible roles, their structure, and how we can combine them with our playbooks. We will analyze their functionality and usage along with ways to create new roles and retrieve public shared roles from Ansible Galaxy, a public repository for Ansible resources.
If you are new to Ansible, you might also find these tutorials helpful <a href="https://spacelift.io/blog/ansible-tutorial" target="_blank" rel="noopener">Ansible Tutorial for Beginners</a>, <a href="https://spacelift.io/blog/ansible-playbooks" target="_blank" rel="noopener">Working with Ansible Playbooks</a>, and <a href="https://spacelift.io/blog/ansible-variables" target="_blank" rel="noopener">How to Use Different Types of Ansible Variables</a>.

When starting with Ansible, it’s pretty common to focus on writing playbooks to automate repeating tasks quickly. As new users automate more and more tasks with playbooks and their Ansible skills mature, they reach a point where using just Ansible playbooks is limiting. Ansible Roles to the rescue!
Roles enable us to reuse and share our Ansible code efficiently. They provide a well-defined framework and structure for setting your tasks, variables, handlers, metadata, templates, and other files. This way, we can reference and call them in our playbooks with just a few lines of code while we can reuse the same roles over many projects without the need to duplicate our code.
Since we have our code grouped and structured according to the Ansible standards, it is quite straightforward to share it with others. We will see an example of how we can accomplish that later with Ansible Galaxy. 
Organizing our Ansible content into roles provides us with a structure that is more manageable than just using playbooks. This might not be evident in minimal projects but as the number of playbooks grows, so does the complexity of our projects. 
Lastly, placing our Ansible code into roles lets us organize our automation projects into logical groupings and follow the separation of concerns design principle. Collaboration and velocity are also improved since different users can work on separate roles in parallel without modifying the same playbooks simultaneously.

Let’s have a look at the standard role directory structure. For each role, we define a directory with the same name. Inside, files are grouped into subdirectories according to their function. A role must include at least one of these standard directories and can omit any that isn’t actively used.
To assist us with quickly creating a well-defined role directory structure skeleton, we can leverage the command ansible-galaxy init &lt;your_role_name&gt;. The ansible-galaxy command comes bundled with Ansible, so there is no need to install extra packages.
Create a skeleton structure for a role named test_role:

image4-18

test_role

Explore the concept of Ansible roles, their structure, and how we can refactor our playbooks into roles or generate them from scratch.

Ioannis Moustakis

44 Ansible Best Practices to Follow [Tips & Tricks]

Get the newsletter

Recent posts

Atlantis vs. Terraform Cloud / Terraform Enterprise – Comparison

How to Automate Infrastructure Provisioning with Terraform

Terraform vs. Kubernetes : Key Differences and Comparison

Ansible Modules – How To Use Them Efficiently (Examples)

16 DevOps Best Practices Every Developer Should Know

AWS IAM Policies : Best Practices & Creating an IAM Policy

Ansible Roles : Basics & How to Combine Them With Playbooks

12 Terraform Best Practices to Improve your TF workflow

Terraform Output Values : Complete Guide & Examples

How to Use Different Types of Ansible Variables (Examples)

Working with Ansible Playbooks – Tips & Tricks with Examples

Ansible Tutorial for Beginners: Ultimate Playbook & Examples