Using Ansible Shell Module to Execute Remote Commands

The Ansible shell module runs a command through a shell on the remote node, which is what makes pipes, redirection, and wildcards work. 

This blog post will dive deep into the Ansible shell module and explore different ways to execute remote commands on nodes as part of our automation efforts. We will review different options and modules for running remote commands and discuss their differences and when to use each.

If you are new to Ansible or interested in other Ansible concepts, these Ansible tutorials on Spacelift’s blog might be handy. 

We will cover:

1. What is the Ansible shell module?
2. Using Ansible shell module examples
3. Alternative options to run remote commands with Ansible
4. When to use the shell module vs. command module

The Ansible shell module is used to execute shell commands in the remote target machines. The shell module takes the command name followed by a list of space-delimited arguments. 

Either a free form command or the cmd parameter is required. The module supports check mode only partially, does not support diff mode, and targets posix platforms.

The shell module does not execute directly on the target but in a shell environment (/bin/sh) on the target. This makes it possible to use shell-specific features and functions such as pipe |, redirection <, >, >>, and so forth.

Because /bin/sh is the default, non-POSIX features such as [[ ]] or process substitution need executable: /bin/bash set explicitly.

If you are targeting Windows nodes, use the ansible.windows.win_shell module.

Ansible shell module parameters

Parameter	Type	Description
chdir	path	Change into this directory before running the command.
cmd	string	The command to run followed by optional arguments.
creates	path	A filename. When it already exists, this step will not be run.
executable	path	Change the shell used to execute the command. Expects an absolute path to the executable.
free_form	string	The shell module takes a free form command to run as a string. There is no actual parameter named free_form.
removes	path	A filename. When it does not exist, this step will not be run.
stdin	string	Set the stdin of the command directly to the specified value.
stdin_add_newline	boolean	Whether to append a newline to stdin data. Defaults to true. Added in Ansible 2.8.

removes is the inverse of creates, and neither parameter appears in the examples below as often as it should. Both are also the only way to get meaningful check mode behavior out of this module.

Let’s take a look at some examples of using the shell module in action.

Example 1: Ansible shell module to execute a single command

- name: Execute shell command
  ansible.builtin.shell: tail -n 10 /var/log/syslog > tail_syslog.txt

In this example, we use the shell module to take the last 10 lines of /var/log/syslog and redirect the output to tail_syslog.txt. The shell module is required here, because > is a redirection operator handled by the shell, not by the command itself.

Example 2: Run a command using the shell module if a file doesn’t exist

- name: This command will only run when file_to_check.txt doesn't exist
  ansible.builtin.shell: tail -n 10 /var/log/syslog > tail_syslog.txt
  args:
    creates: file_to_check.txt

The above example would run the command only if the file file_to_check.txt doesn’t exist. This is achieved with the creates parameter and is useful in cases where we need to check if commands that produce a specific artifact have already been executed.

Use removes instead of creates to invert the check, so the task runs only while the file still exists.

Example 3: Show disk usage

- name: Check disk usage and grep for /dev/sda1
  ansible.builtin.shell: df -h | grep /dev/sda1
  register: disk_usage

- name: Display output
  ansible.builtin.debug:
    var: disk_usage.stdout_lines

In the above example, we use the df -h command to show disk usage and pipe it to filter only results for /dev/sda1. This is an example of a command that can’t be run correctly with the command module.

The command output is registered in the disk_usage variable and displayed in the next task.

Example 4: Execute command in a specific directory

- name: Compile software in a specific directory
  ansible.builtin.shell: make install
  args:
    chdir: /path/to/source/code

That’s another example of using the args keyword with chdir to execute a shell command in a specific directory.

The make install command runs in the /path/to/source/code directory in this case.

Example 5: Execute multiple commands

- name: Update system packages and clean up
  ansible.builtin.shell: |
    apt-get update &&
    apt-get upgrade -y &&
    apt-get clean
  become: true
  register: apt_result
  changed_when: "'0 upgraded' not in apt_result.stdout"

Chain multiple commands together with the && operator, as shown above. The shell executes them in order and stops at the first failure. Two things this example needs that a naive version omits: become: true, because package management requires root, and changed_when, because the shell module otherwise reports changed on every single run. 

For package management specifically, ansible.builtin.apt is the idempotent choice and this pattern should be a fallback, not a default.

Example 6: Check if a process is running

- name: Check if a process is running
  ansible.builtin.shell: ps aux | grep 'nginx' | grep -v grep
  register: process_status
  changed_when: false
  failed_when: false

- name: Display process status
  ansible.builtin.debug:
    msg: "nginx is {{ 'running' if process_status.rc == 0 else 'not running' }}"

In this example, we use ps aux to list all the processes and then the pipe operator | to find the nginx process we’re interested in.

grep returns 1 when it finds no match, so failed_when: false is required or the task fails on exactly the condition you are checking for. changed_when: false stops a read-only check from reporting a change.

We also use a second pipe and grep to exclude our own grep command from the results.

Example 7: Select the shell used to execute the command

- name: Change shell to bash
  ansible.builtin.shell: cat /var/log/*log > logs_snaphot.txt
  args:
    executable: /bin/bash

In this example, we are using the executable parameter to specify the shell used to execute the command. This can be useful in cases where the default /bin/sh doesn’t support a feature we want to leverage.

Example 8: Use a Templated Variable in the command

- name: Run the command using a templated variable to avoid injection
  ansible.builtin.shell: cat {{ logs_snapshot_file|quote }}
  register: logs

In the above example, we use a templated variable in a command. 

When using Ansible variables in commands, make sure to use {{ var | quote }} instead of {{ var }} to add quotes around the variable value, which helps to avoid injection and ensure the contents of the variable are treated as a single argument and not interpreted by the shell.

The quote filter matters most when the variable comes from inventory, a survey, or an API response. Without it, a value containing a semicolon executes as a second command.

We generally prefer using specialized Ansible modules over raw shell or command scripts. Task-specific Ansible modules are designed to be idempotent and to abstract away the underlying complexities of tasks, which makes them preferable to directly executing commands via the shell or command modules, for example.

Even more, specialized modules handle errors gracefully.

If something goes wrong, they often provide helpful error messages to aid in troubleshooting and are safer to use than arbitrary shell commands, which can inadvertently open up security risks. Ansible modules can report whether they made a change on the remote system, which helps to understand better changes performed to the targeted systems.

On some occasions, you might not be able to leverage any task-specific module to achieve your desired outcome. We can use Ansible to execute commands directly on remote hosts in these cases. Ansible provides several ways to execute commands on remote nodes.

We’ve already seen the shell module, and next, we will look at the command, expect, script, and raw modules for this purpose.

Ansible command module

The command module in Ansible executes commands on all selected hosts. It’s one of the most straightforward modules; it takes the command name followed by a list of space-delimited arguments. If you are targeting Windows nodes, use the ansible.windows.win_command instead.

Commands are not processed through a shell with the command module, so shell-specific features such as pipes, redirection operators (<, >, >>, |), semicolons, and ampersands will not be interpreted.

Environment variables behave differently than most guides claim: since ansible-core 2.16, argument variables are resolved through Python rather than the shell, controlled by the expand_argument_vars parameter, which defaults to true. 

So $HOME is expanded before the command runs. An unmatched variable is left unchanged as literal text, whereas a shell would remove it. Set expand_argument_vars: false to treat the value as a literal argument.

This behavior makes the command module safer and more predictable than the shell module that we will discuss later. When using the command module, you can be sure the command will execute exactly as you’ve written it, without any unexpected side effects from shell processing.

Here’s a basic example showcasing how to leverage the command module in a task:

 - name: Display list of files in /var/log
   ansible.builtin.command: ls /var/log
   register: log_files
 - name: Output list of log files
   ansible.builtin.debug:
     var: log_files.stdout_lines

In the above example, ls /var/log is the command being executed. The command output is registered in the log_files variable, which is then displayed using the debug module in the next task.

Here’s an example output of the above two tasks:

Here’s another example of using the command module to check whether a package is installed in a target system:

- name: Check if NGINX is installed
  ansible.builtin.command: which nginx
  register: nginx_installed
  changed_when: false
  failed_when: false

- name: Display a message if NGINX is not installed
  ansible.builtin.debug:
    msg: "NGINX is not installed on this system."
  when: nginx_installed.rc != 0

Here, we check whether NGINX is installed by running the which nginx command and checking the return code(rc).

The changed_when: false line ensures that Ansible doesn’t report a change every time the Ansible playbook is executed, and failed_when: false makes the command succeed and not block the playbook from continuing even if the package isn’t found. 

The command module also accepts argv, which passes the command as a list instead of a string. Use it to avoid quoting values that would otherwise be split incorrectly, such as a username containing a space:

- name: Create a database with a username containing spaces
  ansible.builtin.command:
    argv:
      - /usr/bin/make_database.sh
      - Username with whitespace
      - dbname with whitespace
    creates: /path/to/database

One more difference worth knowing: the command module’s executable parameter was removed in version 2.4. If you need to set the shell, use the shell module.

Here’s an example output of the above two tasks:

Finally, here’s an example using loop to run multiple commands in one task. Ansible has recommended loop over with_items for most use cases since 2.5, though with_items is not deprecated.

- name: Run multiple commands
  ansible.builtin.command: "{{ item }}"
  loop:
    - ls /var/log
    - touch /tmp/tmp.txt
    - ps aux

Ansible expect module

The expect module requires pexpect version 3.3 or higher on the host that executes it, plus Python 3.8 or higher. A missing pexpect is the most common reason these tasks fail on a fresh host.

The expect module in Ansible executes commands and responds to prompts. It’s often used to automate interactions with applications that require responses to prompts.

When using the expect module, you must specify the command that will be run and the responses. The responses are a mapping of expected string/regex and string to respond with.

Note that commands aren’t processed through the shell. 

Here’s an example usage of the expect module that responds to user input during the installation process of a package.

- name: Install software
  ansible.builtin.expect:
    command: /path/to/software/install.sh
    timeout: 120
    responses:
      Continue\?: "yes"
      Please enter the installation directory: "/path/to/installation/directory"
      Enable automatic updates\?: "no"
  no_log: true

Each key in responses is a Python regex, which is why ? is escaped above. Case-insensitive matching uses a (?i) prefix. timeout defaults to 30 seconds and accepts null to disable it. Set no_log: true on any task where a response contains a password, or the value lands in your logs. Pass a list instead of a string to give different answers to successive matches of the same prompt.

Consider the security implications of automating prompt responses and handling sensitive data when using the expect module.

Since the expect module has been designed for simple use cases, consider using the shell or scripts module for more complex and advanced use cases.

Ansible script module

Like the raw module, script does not require Python on the remote system. It also accepts removes and decrypt in addition to chdir, creates, and executable.

The script module executes a local script on remote nodes after transferring it. This module takes the script name followed by a list of space-delimited arguments. The given script will be processed through the shell environment on the remote node.

Note that if the path to the local script contains spaces, it needs to be quoted.

Here’s a basic usage of the script module:

- name: Run a script on a remote node
  ansible.builtin.script: /path/to/local/script.sh --flag some_value

Here’s a more advanced example using the args keyword to control the script environment.

- name: Run a script with custom environment variables
  ansible.builtin.script: /path/to/local/script.sh
  args:
    executable: /bin/bash
    chdir: /tmp/
    creates: /tmp/example.txt

In the above example, we run a script while setting the shell to use, the directory to change into before running the script, and checking if a file exists before running the script. 

As covered above, use or write an Ansible module rather than shipping long scripts. Consider converting your script to an Ansible module to make your playbooks more readable and understandable.

Ansible raw module

The raw module executes raw commands on remote hosts, much like how you might execute commands over SSH. It executes low-down and dirty SSH commands, not going through the module subsystem.

This module does not require Python on the remote system, making it useful in scenarios where Python is not installed or when you’re dealing with devices that don’t support Python.

This module is also supported for Windows targets.

Here’s an example of using the raw module to install Python with yum:

- name: Bootstrap a host without Python installed
  ansible.builtin.raw: dnf install -y python3 python3-libdnf

If you are running raw from a playbook to bootstrap Python, set gather_facts: false on the play. Fact gathering requires Python, so it fails before your bootstrap task ever runs.

The environment keyword does not work with raw under normal conditions, because it needs a shell. It applies only when executable is set or when you use privilege escalation with become. There is also no change handler support, and check mode is not supported.

It’s worth noting that the raw module is less safe, idempotent, and predictable than most other Ansible modules, and it doesn’t support advanced Ansible features such as variable substitutions, loops, conditionals, etc.

Its usage should be avoided unless there is no other way. To execute a command securely and predictably, use the command or shell module instead.

Here is the short version.

Unlike the command module, shell module does not execute directly on the target but in a shell environment on the target. This makes it possible to use shell-specific features and functions such as pipe |, redirection <, >, >>, and others. If your commands contain any such shell-specific features, such as piping commands, variable substitution, redirecting output to files, you have to use the shell module.

Another use case for the shell module would be if your command involves a script that must be executed in the shell context. Because the shell module depends on shell features, it makes your playbooks less portable across target operating systems.

On the other hand, the command module is preferred in most cases as the most straightforward way to run a command on a remote host. Using the command module to execute a command is considered more secure and produces more predictable results.

Best practices when writing playbooks will follow the trend of using the command module unless using shell features is explicitly required.

Check out more Ansible best practices.

Running Ansible at scale usually means playbooks scattered across machines, no single record of what ran where, and provisioning and configuration living in two separate worlds. Spacelift sits on top of Ansible and closes that gap. 

You get one place to run playbooks, GitOps workflows triggered on pull requests, and policy checks that apply to every run.

The same control plane manages Terraform, OpenTofu, Terragrunt, Pulumi, CloudFormation, and Kubernetes, so you can combine provisioning and configuration into a single workflow instead of stitching tools together by hand.

Spacelift’s Ansible functionality solves three of the biggest challenges engineers face when using Ansible:

• Having a centralized place in which you can run your playbooks
• Combining infrastructure as code with configuration management to create a single workflow
• Getting insights into what ran and where

Provisioning, configuring, governing, and even orchestrating your containers can be performed with a single workflow, separating the elements into smaller chunks to identify issues more easily.

Would you like to see this in action? Check out this video showing Spacelift’s Ansible functionality:

In this blog post, we explored how to leverage the Ansible shell module and other different options for executing remote commands with Ansible. We have gone through detailed examples and explained the intricacies and particularities of each different option.

Lastly, we saw examples of each module in action while discussing the different use cases that will make you select one over the other.

Thank you for reading, and I hope you enjoyed this as much as I did!