This article delves into Ansible modules, one of the core building blocks of Ansible. In this blog post, we will examine the purpose and usage of modules, along with information on how to build them and best practices.
If you are interested in other Ansible concepts, these Ansible tutorials posted on Spacelift’s blog might be helpful for you.
Modules represent distinct units of code, each one with specific functionality. Basically, they are standalone scripts written for a particular job and are used in tasks as their main functional layer.
We build Ansible modules to abstract complexity and provide end-users with an easier way to execute their automation tasks without needing all the details. This way, some of the cognitive load of more complex tasks is abstracted away from Ansible users by leveraging the appropriate modules.
Here’s an example of a task using the apt package manager module to install a specific version of Nginx.
- name: "Install Nginx to version {{ nginx_version }} with apt module"
ansible.builtin.apt:
name: "nginx={{ nginx_version }}"
state: presentModules can be executed as well directly from the command line. Here’s an example of running the ping module against all the database hosts from the command line.
ansible databases -m pingA well-designed module provides a predictable and well-defined interface that accepts arguments that make sense and are consistent with other modules. Modules take some arguments as input and return values in JSON format after execution.
Ansible modules should follow idempotency principles, which means that consecutive runs of the same module should have the same effect if nothing else changes. Well-designed modules detect if the current and desired state match and avoid making changes if that’s the case.
We can utilize handlers to control the flow execution of modules and tasks in a playbook. Modules can trigger additional downstream modules and tasks by notifying specific handlers.
As mentioned, modules return data structures in JSON data. We can store these return values in variables and use them in other tasks or display them to the console. Look at the common return values for all modules to get an idea.
For custom modules, the return values should be documented along with other useful information for the module. The command-line tool ansible-doc displays this information.
Here’s an example output of running the ansible-doc command.
ansible-doc apt
In the latest versions of Ansible, most modules are part of collections, a distribution format that includes roles, modules, plugins, and playbooks. Many of the core modules we use extensively are part of the Ansible.Builtin collection. To find other available modules have a look at the Collection docs.
In this part, we explore some of the most used and helpful modules, and for each, we provide a working example. The modules in this list are picked based on their popularity within the Ansible community and functionality to perform everyday automation tasks.
Package Manager Modules yum & apt
The apt module is part of ansible-core and manages apt packages for Debian/Ubuntu Linux distributions. Here’s an example that updates the repository cache and updates the Nginx package to the latest version:
- name: Update the repository cache and update package "nginx" to latest version
ansible.builtin.apt:
name: nginx
state: latest
update_cache: yesThe Ansible yum module is also part of ansible-core and manages packages with yum for RHEL/Centos/Fedora Linux distributions. Here’s the same example as above with the yum module:
- name: Update the repository cache and update package "nginx" to latest version
ansible.builtin.yum:
name: nginx
state: latest
update_cache: yesService Module
The service module controls services on remote hosts and can leverage different init systems depending on their availability in a system. This module provides a nice abstraction layer for underlying service manager modules. Here’s an example of restarting the docker service:
- name: Restart docker service
ansible.builtin.service:
name: docker
state: restartedFile Module
The file module handles operations to files, symlinks, and directories. Here’s an example of using this module to create a directory with specific permissions:
- name: Create the directory "/etc/test" if it doesnt exist and set permissions
ansible.builtin.file:
path: /etc/test
state: directory
mode: '0750'Copy Module
The Ansible copy module copies files to the remote machine and handles file transfers or moves within a remote system. Here’s an example of copying a file to the remote machine with permissions set:
- name: Copy file with owner and permissions
ansible.builtin.copy:
src: /example_directory/test
dest: /target_directory/test
owner: joe
group: admins
mode: '0755'Template Module
The template module assists us to template files out to target hosts by leveraging the Jinja2 templating language. Here’s an example of using a template file and some set Ansible variables to generate an Nginx configuration file:
- name: Copy and template the Nginx configuration file to the host
ansible.builtin.template:
src: templates/nginx.conf.j2
dest: /etc/nginx/sites-available/defaultRead more about how to create an Ansible template.
Lineinfile & Blockinfile Modules
The lineinfile module adds, replaces, or ensures that a particular line exists in a file. It’s pretty common to use this module when we need to update a single line in configuration files.
- name: Add a line to a file if it doesnt exist
ansible.builtin.lineinfile:
path: /tmp/example_file
line: "This line must exist in the file"
state: presentThe blockinfile module inserts, updates, or removes a block of lines from a file. It has the same functionality as the previous module but is used when you want to manipulate multi-line text blocks.
- name: Add a block of config options at the end of the file if it doesn’t exist
ansible.builtin.blockinfile:
path: /etc/example_dictory/example.conf
block: |
feature1_enabled: true
feature2_enabled: false
feature2_enabled: true
insertafter: EOFCron Module
The cron module manages crontab entries and environment variables entries on remote hosts.
- name: Run daily DB backup script at 00:00
ansible.builtin.cron:
name: "Run daily DB backup script at 00:00"
minute: "0"
hour: "0"
job: "/usr/local/bin/db_backup_script.sh > /var/log/db_backup_script.sh.log 2>&1"Wait_for Module
The wait_for module provides a way to stop the execution of plays and wait for conditions, amount of time to pass, ports to become open, processes to finish, files to be available, strings to exist in files, etc.
- name: Wait until a string is in the file before continuing
ansible.builtin.wait_for:
path: /tmp/example_file
search_regex: "String exists, continue"Command & Shell Modules
The command and shell modules execute commands on remote nodes. Their main difference is that the command module bypasses the local shell, and consequently, variables like $HOSTNAME or $HOME aren’t available, and operations like “<”, “&” don’t work. If you need these features, you have to use the shell module.
On the other hand, the remote local environment won’t affect the command module, so its outcome is considered more predictable and secure.
Usually, it’s always preferred to use specialized Ansible modules to perform tasks instead of command and shell. There are cases, though, where you won’t be able to get the functionality that you need from specialized modules, and you will have to use one of these two. Use command and shell with care, and always try to check if there is a specialized module that can serve you better before relying on them.
- name: Execute a script in remote shell and capture the output to file
ansible.builtin.shell: script.sh >> script_output.logRead more about the Ansible shell module.
For advanced users, there is always the option to develop custom modules if they have needs that can’t be satisfied by existing modules. Since modules always should return JSON data, they can be written in any programming language.
Before jumping into module development, ensure that a similar module doesn’t exist to avoid unnecessary work. Αdditionally, you might be able to combine different modules to achieve the functionality you need. In this case, you might be able to replicate the behavior you want by creating a role that leverages other modules. Another option is to use plugins to enhance Ansible’s basic functionality with logic and new features accessible to all modules.
Next, we will go through an example of creating a custom module that takes as input a string that represents an epoch timestamp and converts it to its human-readable equivalent of type datetime in Python. You can find the code for this tutorial on this repository.
First, let’s create a library directory on the top of our repository to place our custom module. Playbooks with a ./library directory relative to their YAML file can add custom ansible modules that can be recognized in the ansible module path. This way, we can group custom modules and their related playbooks.
We create our custom Python module epoch_converter.py inside the library directory. This simple module takes as input the argument epoch_timestamp and converts it to datetime type. We use another argument, state_changed, to simulate a change in the target system by this module.
library/epoch_converter.py
#!/usr/bin/python
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import datetime
DOCUMENTATION = r'''
---
module: epoch_converter
short_description: This module converts an epoch timestamp to human-readable date.
# If this is part of a collection, you need to use semantic versioning,
# i.e. the version is of the form "2.5.0" and not "2.4".
version_added: "1.0.0"
description: This module takes a string that represents a Unix epoch timestamp and displays its human-readable date equivalent.
options:
epoch_timestamp:
description: This is the string that represents a Unix epoch timestamp.
required: true
type: str
state_changed:
description: This string simulates a modification of the target's state.
required: false
type: bool
author:
- Ioannis Moustakis (@Imoustak)
'''
EXAMPLES = r'''
# Convert an epoch timestamp
- name: Convert an epoch timestamp
epoch_converter:
epoch_timestamp: 1657382362
'''
RETURN = r'''
# These are examples of possible return values, and in general should use other names for return values.
human_readable_date:
description: The human-readable equivalent of the epoch timestamp input.
type: str
returned: always
sample: '2022-07-09T17:59:22'
original_timestamp:
description: The original epoch timestamp input.
type: str
returned: always
sample: '16573823622'
'''
from ansible.module_utils.basic import AnsibleModule
def run_module():
# define available arguments/parameters a user can pass to the module
module_args = dict(
epoch_timestamp=dict(type='str', required=True),
state_changed=dict(type='bool', required=False)
)
# seed the result dict in the object
# we primarily care about changed and state
# changed is if this module effectively modified the target
# state will include any data that you want your module to pass back
# for consumption, for example, in a subsequent task
result = dict(
changed=False,
human_readable_date='',
original_timestamp=''
)
# the AnsibleModule object will be our abstraction working with Ansible
# this includes instantiation, a couple of common attr would be the
# args/params passed to the execution, as well as if the module
# supports check mode
module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=True
)
# if the user is working with this module in only check mode we do not
# want to make any changes to the environment, just return the current
# state with no modifications
if module.check_mode:
module.exit_json(**result)
# manipulate or modify the state as needed (this is going to be the
# part where your module will do what it needs to do)
result['original_timestamp'] = module.params['epoch_timestamp']
result['human_readable_date'] = datetime.datetime.fromtimestamp(int(module.params['epoch_timestamp']))
# use whatever logic you need to determine whether or not this module
# made any modifications to your target
if module.params['state_changed']:
result['changed'] = True
# during the execution of the module, if there is an exception or a
# conditional state that effectively causes a failure, run
# AnsibleModule.fail_json() to pass in the message and the result
if module.params['epoch_timestamp'] == 'fail':
module.fail_json(msg='You requested this to fail', **result)
# in the event of a successful module execution, you will want to
# simple AnsibleModule.exit_json(), passing the key/value results
module.exit_json(**result)
def main():
run_module()
if __name__ == '__main__':
main()
To test our module, let’s create a test_custom_module.yml playbook in the same directory as our library directory.
test_custom_module.yml
- name: Test my new module
hosts: localhost
tasks:
- name: Run the new module
epoch_converter:
epoch_timestamp: '1657382362'
state_changed: yes
register: show_output
- name: Show Output
debug:
msg: '{{ show_output }}'Last stop, let’s execute the playbook to test our custom module. Since we opted to set the state_changed argument, we expect the task state to appear as changed and displayed in yellow.
If you wish to contribute to an existing Ansible collection or create and publish a new one with your custom modules, look at Distributing collections and Ansible Community Guide, where you can find information on how to configure and distribute Ansible content.
Use specialized modules over shell or command: Although It might be tempting to use the shell or command module often, it’s considered a best practice to leverage more specific modules for each job. Specialized modules are typically recommended because they implement the concept of desired state and idempotency, have been tested, and fulfill basic standards, like error handling.
Specify arguments when it makes sense: Some module arguments have default values that can be omitted. To be more transparent and explicit, we can opt to specify some of these arguments like the state in our playbook definitions.
Prefer multi-tasks in a module over loops: The most efficient way of defining a list of similar tasks, like installing packages, is to use multiple tasks in a single module.
- name: Install Docker dependencies
ansible.builtin.apt:
name:
- curl
- ca-certificates
- gnupg2
- lsb-release
state: latestThe above method should be preferred over the Ansible loop or defining multiple separate tasks using the same module.
Custom modules should be simple and tackle a specific job: If you decide to build your own module, focus on solving a particular problem. Each module should have a concise functionality, be as simple as possible, and perform one thing well. If what you try to achieve goes beyond the scope of a single module, consider developing a new collection.
Custom modules should have predictable parameters: Try to enable others to use your module by defining a transparent and predictable user interface. The arguments should be well-scoped and understandable, and their structures should be as simple as possible. Follow the typical convention of parameter names in lowercase and use underscores as the word separator.
Document and test your custom modules: Every custom module should include examples, explicitly document dependencies, and describe return responses. New modules should be tested thoroughly before releasing. You can create roles and playbooks to test your custom modules and different test cases.
Spacelift’s vibrant ecosystem and excellent GitOps flow can greatly assist you in managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can then easily create custom workflows based on pull requests and apply any necessary compliance checks for your organization.
Another great advantage of using Spacelift is that you can manage different infrastructure tools like Ansible, Terraform, Pulumi, AWS CloudFormation, and even Kubernetes from the same place and combine their Stacks with building workflows across tools.
Our latest Ansible enhancements solve three of the biggest challenges engineers face when they are using Ansible:
- Having a centralized place in which you can run your playbooks
- Combining IaC with configuration management to create a single workflow
- Getting insights into what ran and where
Provisioning, configuring, governing, and even orchestrating your containers can be performed with a single workflow, separating the elements into smaller chunks to identify issues more easily.
Would you like to see this in action – or just want a tl;dr? Check out this video I put together showing you Spacelift’s new Ansible functionality:
If you want to learn more about using Spacelift with Ansible, check our documentation, read our Ansible guide, or book a demo with one of our engineers.
We deep-dived into Ansible modules and examined their use and functionality in detail. We discussed best practices and showed practical examples of leveraging the most commonly-used modules. Lastly, we went through a complete example of developing a custom module.
Thank you for reading, and I hope you enjoyed this article as much as I did.
Manage Ansible Better with Spacelift
Managing large-scale playbook execution is hard. Spacelift enables you to automate Ansible playbook execution with visibility and control over resources, and seamlessly link provisioning and configuration workflows.
