Ansible is an open-source and battle-tested automation tool with simplicity and powerful capabilities. These qualities make it an excellent choice for configuration management, infrastructure provisioning, and application deployment use cases.
Leveraging Ansible to manage and provision cloud infrastructure on AWS enables operations to streamline and automate various tasks. It enables cross-platform automation and orchestration at scale and is considered an excellent option for configuration management, infrastructure provisioning, and application deployment use cases.
What we will cover:
Ansible is used in AWS for various purposes, including configuration management, CI/CD and application deployment, and cloud provisioning and management. It also supports network automation, security and compliance automation, disaster recovery automation, and complex workflow automation, making it a versatile tool for automating and streamlining various AWS operations.
Ansible AWS modules
Ansible offers a wide range of modules specifically designed to simplify the automation and management of AWS services. These modules enable you to efficiently handle tasks such as instance provisioning, security group configuration, and resource management.
| AWS service example | Ansible module example | |
| Compute and scaling | Autoscaling Groups, Elastic Cloud Compute (EC2) | autoscaling_group module, ec2_instance module |
| Networking and security | Virtual Private Cloud (VPC), Security Groups, Identity Access Manager (IAM) | ec2_vpc_subnet module, iam_policy module |
| Storage and databases | Simple Storage Service (S3), DynamoDB, ElastiCache, Relational Database Service (RDS) | s3_bucket module, rds_instance module |
| Monitoring and logging | CloudWatch, CloudTrail | cloudtrail module |
| Infrastructure as Code | CloudFormation | cloudformation module |
| Serverless computing | AWS Lambda | lambda module |
Read more on Ansible modules and how to use them.
Ansible’s robust automation and orchestration capabilities at scale make it a valuable tool in every Cloud Engineer’s toolbelt. Below are the most important benefits of using Ansible in AWS:
- The idempotent nature of Ansible ensures that tasks are executed consistently and predictably, reducing the risk of operations.
- Ansible does not require additional software to be installed on the managed nodes, making it lightweight and easy to get started.
- Ansible’s modular architecture enables extensibility and customization to cover even the most demanding needs by developing custom Ansible modules.
In this section, we’ll show you how to configure and use Ansible with AWS. We’ll guide you through the necessary steps to set up Ansible, connect it to your AWS environment, and automate various AWS tasks.
1. Connect to AWS services
To use Ansible and target resources on AWS, you must first set up authentication. The most common methods include using environment variables, AWS CLI configuration and profiles, Ansible Vault, and costing credentials in the Ansible Playbooks.
To set environment variables for authentication purposes, export the AWS secret access key and AWS access key ID:
export AWS_ACCESS_KEY_ID='<YOUR ACCESS_KEY_ID>'
export AWS_SECRET_ACCESS_KEY='>YOUR_SECRET_ACCESS_KEY>'Another option is to use the official AWS CLI to configure a profile and persist these values in a local AWS credentials file. After installing the AWS CLI, check out the Set up the AWS CLI to configure this based on your preferred method.
Let’s say we want to configure some long-term credentials locally in this case. Type aws configure and specify values for access key ID, secrets access key, and AWS region. Alternatively, we can request short-term credentials, assume an AWS IAM role, or use the IAM Identity Center (SSO).
You could also store credentials inside the Ansible Playbooks with variables. Make sure to avoid hardcoding AWS secrets and keys inside playbooks. If you wish to go down that way, leverage Ansible Vault to encrypt these values before using them.
For example, let’s say we store our secrets in secrets.yml:
secrets.yml
aws_access_key_id: my-aws-access-key-id
aws_secret_key: my-aws-secret-access-keyThen, you can encrypt this file with the following:
ansible-vault encrypt secrets.yamlTo use this encrypted file within a playbook:
playbook.yml
- hosts: localhost
vars_files:
- secrets.yml
tasks:
- ec2_instance_info:
aws_access_key_id: "{{ aws_access_key_id }}"
aws_secret_key: "{{ aws_secret_key }}"`and trigger the Ansible playbook while passing the password when prompted:
ansible-playbook playbook.yml --ask-vault-pass2. Provision the AWS infrastructure
While we know that it is not a best practice to provision infrastructure using Ansible, it is still possible.
Typically, provisioning AWS infrastructure with Ansible involves creating and managing different resources via Ansible playbooks.
In this example, we are creating an EC2 instance to deploy a web application. We would also need a VPC, a subnet, security groups, and an Internet Gateway attached to the VPC.
ec2_playbook.yml
- name: Create EC2 and necessary AWS resources
hosts: localhost
gather_facts: no
vars:
region: us-east-1
instance_type: t2.micro
ami_id: ami-04e5276ebb8451442
vpc_cidr_block: 10.0.0.0/16
subnet_cidr_block: 10.0.1.0/24
security_group_cidr_ingress: 0.0.0.0/0
security_group_cidr_egress: 0.0.0.0/0
security_group_ports:
- 80
- 443
tasks:
- name: Create VPC
ec2_vpc_net:
name: MyVPC
cidr_block: “{{ vpc_cidr_block }}”
region: "{{ region }}"
tags:
Name: MyVPC
register: my_vpc
- name: Output VPC ID
debug:
msg: "VPC ID is {{ my_vpc.vpc.id }}"
- name: Create subnet
ec2_vpc_subnet:
state: present
vpc_id: "{{ my_vpc.vpc.id }}"
cidr: “{{ subnet_cidr_block }}”
region: "{{ region }}"
tags:
Name: MySubnet
register: my_subnet
- name: Output Subnet ID
debug:
msg: "Subnet ID is {{ my_subnet.subnet.id }}"
- name: Create internet gateway
ec2_vpc_igw:
vpc_id: "{{ my_vpc.vpc.id }}"
region: "{{ region }}"
tags:
Name: MyIGW
register: igw
- name: Output Internet Gateway ID
debug:
msg: "Internet Gateway ID is {{ igw.gateway_id }}"
- name: Create security group
ec2_group:
name: "MySc"
description: "My security group"
vpc_id: "{{ my_vpc.vpc.id }}"
region: "{{ region }}"
rules:
- proto: tcp
ports: “ {{ security_group_ports }”
cidr_ip: “{{ security_group_cidr_ingress }}”
rules_egress:
- proto: all
cidr_ip: “{{ security_group_cidr_egress }}”
register: security_group
- name: Output Security Group ID
debug:
msg: "Security Group ID is {{ security_group.group_id }}"
- name: Launch instance
ec2_instance:
name: "MyInstance"
instance_type: "{{ instance_type }}"
region: "{{ region }}"
image_id: "{{ ami_id }}"
subnet_id: "{{ my_subnet.subnet.id }}"
wait: yes
security_group: "{{ security_group.group_id }}"
network:
assign_public_ip: true
tags:
Environment: Testing
register: ec2
- name: Output Instance Details
debug:
msg: "Instance ID is {{ ec2.instances[0].instance_id }}"To go ahead and create these AWS resources, run the playbook.
ansible-playbook ec2_playbook.yml
PLAY [Create EC2 and necessary AWS resources] **********************************************************************************************************************************************************************************************************************************
TASK [Create VPC] **************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Output VPC ID] ***********************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "VPC ID is vpc-008c5e6b9d3f31028"
}
TASK [Create subnet] ***********************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Output Subnet ID] ********************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Subnet ID is subnet-00557cf19209f323e"
}
TASK [Create internet gateway] *************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Output Internet Gateway ID] **********************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Internet Gateway ID is igw-0371eefa567f9fdbe"
}
TASK [Create security group] ***************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Output Security Group ID] ************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Security Group ID is sg-0806f59420e25294f"
}
TASK [Launch instance] *********************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Output Instance Details] *************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "Instance ID is i-09f60c41a4a3bc583"
}
PLAY RECAP *********************************************************************************************************************************************************************************************************************************************************************
localhost : ok=10 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=03. Configure dynamic host inventory in AWS
An Ansible inventory is a collection of managed hosts we want to manage with Ansible for various automation and configuration management tasks. Typically, when starting with Ansible, we define a static list of hosts known as the inventory.
As many modern environments are dynamic, cloud-based, and possibly spread across multiple providers, maintaining a static list of managed nodes is time-consuming, manual, and error-prone. Ansible’s dynamic inventory feature allows you to automatically pull inventory from external sources, such as cloud providers like AWS.
To track and target a dynamic set of hosts, you can use the aws_ec2 inventory plugin. This way, you can query AWS for instances and organize them into groups that can be targeted in your playbooks. The dynamic inventory plugin connects to AWS, fetches details about the instances, and outputs these details in a JSON format that Ansible can understand.
dynamic_inventory_aws_ec2.yml
plugin: aws_ec2
regions:
- us-east-1
hostnames: tag:Name
keyed_groups:
- key: tags['Environment']
prefix: env
- key: tags['Role']
prefix: roleand in your ansible.cfg file add:
ansible.cfg
[inventory]
enable_plugins = aws_ec2
Then you can view the hosts of the dynamic inventory grouped as we specified with:
ansible-inventory -i dynamic_inventory_aws_ec2.yml --graph
@all:
|--@ungrouped:
|--@aws_ec2:
| |--worker1
| |--MyInstance
| |--db1
| |--Web1
|--@env_Testing:
| |--worker1
|--@role_Worker:
| |--worker1
|--@env_Production:
| |--MyInstance
| |--Web1
|--@role_Web:
| |--MyInstance
| |--Web1
|--@env_Staging:
| |--db1
|--@role_Database:
| |--db1If you want to filter for specific tags, use the filters argument. This example will only show you the instances tagged with Environment: Testing:
# You can include only instances with specific tags, or all instances
filters:
tag:Environment: Testing # Only include instances with these tags`When running your playbook, you can specify the inventory file:
ansible-playbook -i dynamic_inventory_aws_ec2.yml playbook.ymlTo target only a specific group of instances, for example, only the Production instances, use the --limit argument:
ansible-playbook -i dynamic_inventory_aws_ec2.yml playbook.yml --limit ‘env_Production’4. Leverage Ansible for configuration management at scale
Another way we can leverage Ansible in AWS is for configuration management needs of our fleet of instances or other cloud services. When it comes to managing large-scale cloud infrastructures, Ansible’s simplicity is a significant benefit.
Ansible can run commands on multiple hosts simultaneously or in batches. There is an option for long-running tasks to allow asynchronous execution, allowing other tasks to proceed without waiting for the previous one.
When you need to deploy a web server across many EC2 instances, you could create an Ansible playbook that helps you configure them.
Here’s an example of a playbook you could use to install Nginx to Ubuntu instances.
configure_web_server.yml
- name: Create EC2 and necessary AWS resources
hosts: all
become: true
gather_facts: no
vars:
nginx_version: 1.24.0-2ubuntu7
nginx_custom_directory: /home/ubuntu/nginx
tasks:
- name: Update and upgrade apt
ansible.builtin.apt:
update_cache: yes
cache_valid_time: 3600
upgrade: yes
- name: "Install Nginx to version {{ nginx_version }}"
ansible.builtin.apt:
name: "nginx={{ nginx_version }}"
state: present
- name: Copy the Nginx configuration file to the host
template:
src: nginx.conf.j2
dest: /etc/nginx/sites-available/default
- name: Create link to the new config to enable it
file:
dest: /etc/nginx/sites-enabled/default
src: /etc/nginx/sites-available/default
state: link
- name: Create Nginx directory
ansible.builtin.file:
path: "{{ nginx_custom_directory }}"
state: directory
- name: Copy index.html to the Nginx directory
copy:
src: index.html
dest: "{{ nginx_custom_directory }}/index.html"
notify: Restart the Nginx service
handlers:
- name: Restart the Nginx service
service:
name: nginx
state: restartedOnce the playbook is configured, run the below Ansible command to target all the EC2 instances that are tagged with “Environment: Staging” using the dynamic inventory concept that we introduced above:
ansible-playbook -i dynamic_inventory_aws_ec2.yml configure_web_server.yml -- limit env_Staging5. Use Ansible for AWS image building
After configuring an EC2 instance with Ansible, you can also register an EC2 AMI image with the configuration persisted. To create an AMI from a running or stopped EC2 image, leverage the amazon.aws.ec2_ami module.
For example, at the end of our playbook ec2_playbook.yml introduced above to create an EC2 instance, add these two tasks:
- pause:
minutes: 5
- name: Creating the AMI from of the instance
ec2_ami:
instance_id: "{{ ec2.instances[0].instance_id }}"
wait: yes
name: "{{ ami_name }}"
tags:
Name: "{{ ami_name }}"Here, we added a custom pause task to wait for the EC2 instance be be in running state before creating the AMI. To create the AMI run:
ansible-playbook ec2_playbook.yml6. Run Ansible playbooks with the AWS systems manager
AWS Systems Manager gathers all the operational sub-services for your AWS applications and resources for secure, end-to-end management solutions for hybrid and multicloud environments at scale.
The Systems Manager provides robust support for running Ansible playbooks with the integration of an SSM document called AWS-ApplyAnsiblePlaybooks. This feature supports fetching Ansible manifests from the GitHub repository or S3 buckets and complex playbooks.
To use a playbook or a role with the Systems Manager, we have to create a State Manager association, which is basically a configuration that defines the state that you want to maintain on your resources.
For this example, we created an S3 bucket and uploaded our configure_web_server.yml there. Then, using the command below and targeting our AWS account, we can register this association to run our playbook.
Change the placeholder <your_S3_bucket_name> in the following command with your S3 bucket name. We are targeting the us-east-1 region and only the EC2 instances with the tag role:webserver.
aws ssm create-association --name "AWS-ApplyAnsiblePlaybooks" --parameters '{"SourceType":["S3"], "SourceInfo":["{\"path\": \"https://<your_S3_bucket_name>.s3.amazonaws.com/configure_web_server.yml\"}"], "InstallDependencies":["True"], "PlaybookFile":["ec2_playbook.yml"], "ExtraVariables":["myregion=us-east-1"], "Check":["False"], "Verbose":["-v"]}' --targets '[{"Key":"tag:role","Values":["webserver"]}]' --max-concurrency "50" --max-errors "0"With this SSM association generated, we can automate Ansible playbook deployment via Systems Manager on desired instances on demand or at a predefine schedule.
7. Clean up
To prevent unintentional AWS costs, ensure you terminate any EC2 instances, the Internet Gateway, and the SSM association you launched through the examples of this blog post.
You can do this manually through the console or create a separate playbook to clean up and delete the AWS resources when they are no longer needed. Including provisioning and de-provisioning tasks in your Ansible playbooks is a good practice.
This section presents key best practices for using Ansible with AWS for your automation and cloud management needs.
Security
- Avoid hardcoding sensitive information in your playbooks. Use environment variables or the AWS credentials file.
- For EC2 instances running Ansible, consider using IAM roles that provide necessary permissions without using static credentials.
- Don’t store sensitive values in plain text; for secrets and sensitive values, use Ansible Vault to encrypt variables and files and protect sensitive information.
Inventory
- Leverage the Dynamic Inventory option. Instead of hardcoding hosts in your Ansible inventory, use the AWS EC2 dynamic inventory. This allows Ansible to automatically query AWS for running instances and use those instances based on tags, regions, and other attributes. It keeps your inventory updated as the state of AWS resources changes.
- Leverage Dynamic Grouping at runtime. Using the
group_bymodule based on a specific attribute, we can create dynamic groups. For example, group hosts dynamically based on their operating system and run different tasks on each without defining such groups in the inventory.
Automation & execution
- Leverage the specific AWS Cloud Modules; Ansible includes a wide range of modules specifically designed for AWS, such as EC2, S3, RDS, and more. These Ansible AWS modules are designed to interact efficiently with AWS services, reducing the need to script these operations manually.
- Ensure your playbooks are idempotent. Running them multiple times on the same system will produce the same result without unintended side effects. This is crucial for maintaining consistent state management and reducing errors during repeated provisioning or configuration updates.
- To improve the performance of your playbooks, explore strategies like asynchronous actions and polling to manage long-running tasks and batch operations to reduce overhead when interacting with AWS APIs.
- Consider using an infrastructure management and collaboration tool such as Spacelift to manage the complexities and compliance challenges of using Ansible at scale. Check out the Getting Started guide.
If you are looking for generic Ansible best practices, check out the 44 Ansible Best Practices to Follow [Tips & Tricks] blog post.
Spacelift’s vibrant ecosystem and excellent GitOps flow can greatly assist you in managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can easily create custom workflows based on pull requests and apply any necessary compliance checks for your organization.
With Spacelift, you get:
- Better playbook automation – Manage the execution of Ansible playbooks from one central location.
- Inventory observability – View all Ansible-managed hosts and related playbooks, with clear visual indicators showing the success or failure of recent runs.
- Playbook run insights – Audit Ansible playbook run results with detailed insights to pinpoint problems and simplify troubleshooting.
- Policies – Control what kind of resources engineers can create, what parameters they can have, how many approvals you need for a run, what kind of task you execute, what happens when a pull request is open, and where to send your notifications
- Stack dependencies – Build multi-infrastructure automation workflows with dependencies, having the ability to build a workflow that, for example, generates your EC2 instances using Terraform and combines it with Ansible to configure them
- Self-service infrastructure via Blueprints, or Spacelift’s Kubernetes operator – Enable your developers to do what matters – developing application code while not sacrificing control
- Creature comforts such as contexts (reusable containers for your environment variables, files, and hooks), and the ability to run arbitrary code
- Drift detection and optional remediation
If you want to learn more about using Spacelift with Ansible, check our documentation, read our Ansible guide, or book a demo with one of our engineers.
Would you like to see this in action – or just want a tl;dr? Check out this video I put together showing you Spacelift’s new Ansible functionality:
In this blog post, we delved into using Ansible in AWS environments. We explored the benefits of leveraging Ansible for configuration management and automation of cloud resources and went through detailed examples of using Ansible to create and manage resources on AWS and automate Ansible playbook deployment. Finally, we discussed a few best practices, such as using dedicated AWS related modules, to take your Ansible AWS game to the next level.
Thanks for reading, and I hope you enjoyed this article as much as I did.
Manage Ansible Better with Spacelift
Managing large-scale playbook execution is hard. Spacelift enables you to automate Ansible playbook execution with visibility and control over resources, and seamlessly link provisioning and configuration workflows.
![How to Manage Kubernetes with Ansible [Tutorial]](/_next/image?url=https%3A%2F%2Fspaceliftio.wpcomstaging.com%2Fwp-content%2Fuploads%2F2024%2F04%2F366.kubernetes-with-ansible.png&w=3840&q=75)
