One of the most important aspects of infrastructure management is the ability to retrieve files from remote systems, whether it’s collecting logs for troubleshooting, backing up configuration files, or gathering reports from distributed services. You could SSH into each server individually to grab what you need, but that approach doesn’t scale well when you’re managing dozens or hundreds of hosts.
In Ansible, the fetch module is the recommended way to pull files from remote hosts back to your control node. It handles the complexity of connecting to multiple servers, organizing the retrieved files, and ensuring data integrity during the transfer process.
In this article, we will examine how it works, common use cases for it, and some alternatives.
What we’ll cover:
The Ansible fetch module copies files from remote hosts to the local machine where Ansible is running. Use it when you need to collect logs, configuration files, or other artifacts from target nodes.
It’s essentially the reverse operation of the copy module: Whereas the copy module moves files from your local machine to remote hosts, the fetch module pulls files from remote hosts back to your local machine.
During the fetching process, the Ansible fetch module allows you to:
- Retrieve configuration files from remote servers for backup or analysis
- Pull logs from multiple servers to a centralized location
- Download generated reports or output files from remote processes
- Create a directory structure that mirrors your inventory organization
- Validate file integrity using checksums
- Handle binary files safely with base64 encoding when needed
The fetch module is particularly useful when you need to collect files from multiple hosts in your infrastructure. By default, it organizes retrieved files in a directory structure based on hostname, making it easy to identify which file came from which server.
When you execute the fetch module, Ansible connects to the remote host and locates the specified file. It then transfers the file back to the control node, creating a directory structure under your specified destination path. The default behavior creates subdirectories named after each host, preventing file conflicts when fetching the same filename from multiple servers.
For example, if you fetch /etc/nginx/nginx.conf from servers web1 and web2 to the destination /backup/, the module will create:
/backup/web1/etc/nginx/nginx.conf
/backup/web2/etc/nginx/nginx.confThis automatic organization ensures that files from different hosts don’t overwrite each other and maintains a clear audit trail of where each file originated.
Key parameters
The fetch module provides several parameters to control its behavior:
src(required): The file path on the remote host that you want to fetchdest(required): The directory on the Ansible control node where files will be savedflat: When set toyes, stores the file directly indestwithout creating host-based subdirectoriesfail_on_missing: Controls whether the task fails if the source file doesn’t exist (default:yes)validate_checksum: Verifies file integrity after transfer (default:yes)
The fetch module is typically used for gathering logs, configuration files, or artifacts generated during playbook runs. Common use cases include:
Collecting configuration files for backup and version control
One of the most frequent use cases is pulling configuration files from production servers. Whether you’re managing a fleet of web servers, databases, or application instances, having local copies of config files is essential for disaster recovery and change tracking, or when you want to deploy a hotfix without editing files on your target host.
Log aggregation
System administrators often need to collect logs from multiple servers for centralized analysis. Instead of SSHing into each machine individually, fetch can pull log files from your entire infrastructure in a single playbook run.
This is particularly useful when investigating incidents that might span multiple servers or when you need to correlate events across your infrastructure.
A good example is tracking a recurring error between your web server and a self-hosted message queue, such as RabbitMQ. With Ansible fetch, you can pull logs from both servers in one playbook run and then correlate the timing locally. This allows you to see when requests hit the web server versus when errors appear in the message queue.
With fetch, you avoid the usual process of SSHing into each server separately to piece together what happened.
Gathering generated reports and data exports
Applications often generate reports, database dumps, or data exports that need to be collected and processed centrally. The fetch module excels at this batch collection process.
Certificate and security file management
When managing SSL certificates, SSH keys, or other security artifacts across your infrastructure, fetch helps you maintain an inventory of what’s deployed where. This is crucial for planning certificate renewals and security audits.
Some organizations still obtain certs from specific vendors, and they might not necessarily provide tooling to rotate them automatically. Combining the copy module with the fetch module can help rotate and create a backup of the certificates obtained from a vendor.
Using the Ansible fetch module is straightforward: You specify which file you want to retrieve and where you want to store it locally. The module handles the heavy lifting of connecting to remote hosts, transferring files, and organizing them in a logical directory structure.
Here are some practical examples:
1. Fetching a single log file from a remote host
Say you’re troubleshooting an issue with your web application and need to examine the error log from your production server. Instead of SSHing in and browsing through the log file on the remote machine, you can pull it locally for easier analysis.
---
- name: Fetch application error log
hosts: webserver
tasks:
- name: Retrieve error log from production server
fetch:
src: /var/log/myapp/error.log
dest: /tmp/logs/
flat: noThis playbook above connects to your web server host and fetches the error log file. Since flat: no is specified (which is actually the default), the file will be saved as /tmp/logs/webserver/var/log/myapp/error.log on your control node.
The hostname directory structure ensures you know exactly which server the log came from. This is especially useful when you’re managing multiple web servers and need to fetch logs from all of them.
Once the file is local, you can use log analysis tools such as the Elastic Stack (ELK), Graylog, and Fluentd to do more advanced searches, pattern matching, and visualization. This is also useful for obtaining live data from pre-production environments without requiring direct access to production systems for your analysis tools.
2. Fetching files into a flat directory
In the previous example, files were stored in a hierarchical directory structure based on the hostname, which is great for organization and preventing conflicts. However, sometimes you just want all files dumped into a single directory without the extra folder hierarchy, especially when fetching files with unique names or when targeting only a single host.
---
- name: Fetch configuration file to flat directory
hosts: database_server
tasks:
- name: Get database config for quick review
fetch:
src: /etc/mysql/my.cnf
dest: /tmp/configs/my.cnf
flat: yesWith flat: yes, the file is saved directly as /tmp/configs/my.cnf on your control node, skipping the hostname directory structure entirely. This approach is particularly useful when working with a single host or when you need to quickly retrieve a file for immediate inspection without navigating through multiple directories.
The flat option is also useful for integrating with other tools that expect files in a specific location, or when trying to replicate an issue with a production configuration.
Note: If all have files with the same name, the last one fetched will overwrite the previous ones.
3. Conditional fetching based on task output
One of the most powerful uses of the fetch module is combining it with conditional logic based on task results.
You are probably familiar with this process: You’re restarting a web server as part of a deployment, but if the restart fails, you want to automatically grab the error logs without having to SSH in and hunt around for what went wrong.
With the fetch module, this would look like:
---
- name: Restart web server and fetch logs on failure
hosts: webservers
tasks:
- name: Restart nginx service
systemd:
name: nginx
state: restarted
register: restart_result
ignore_errors: yes
- name: Fetch error logs if restart failed
fetch:
src: /var/log/nginx/error.log
dest: /tmp/failed_restarts/
flat: no
when: restart_result.failed == trueThis playbook attempts to restart nginx and registers the result. If the restart fails, it automatically fetches the error log to your local machine. No more logging into the server, grepping through logs, and crossing your fingers on the next restart attempt; you immediately have the error details locally to diagnose what went wrong before trying again.
4. Fetching multiple files using a loop
Sometimes, you need to grab several related files at once. Perhaps you’re collecting all configuration files from a service or pulling multiple log files from an instance you don’t necessarily control, such as a customer’s VM.
Rather than writing separate fetch tasks for each file, you can use a loop to handle them all in one go.
---
- name: Fetch multiple configuration files
hosts: webservers
tasks:
- name: Collect all nginx config files
fetch:
src: "{{ item }}"
dest: /tmp/nginx_configs/
flat: no
loop:
- /etc/nginx/nginx.conf
- /etc/nginx/sites-available/default
- /etc/nginx/conf.d/ssl.conf
- /var/log/nginx/access.logThis approach is particularly useful when you’re conducting a deep dive into a service’s configuration or when you need to collect all related files for troubleshooting, rather than running multiple playbooks or tasks.
It is also important to remember that this is best suited for tasks with predictable paths; if the paths are more dynamic, you might find yourself doing more templating than normal.
The fetch module is great for pulling files from remote hosts, but it’s not the only tool for file operations. Understanding when to use fetch versus other Ansible modules can save you time and help you choose the right approach for each scenario.
The following are some key differences, along with instances where you might want to consider alternatives instead.
What is the difference between fetch and copy in Ansible?
The fetch and copy modules are essentially opposites in terms of data flow direction. While fetch pulls files from remote hosts to your Ansible control node, copy pushes files from your control node out to remote hosts.
A quick comparison to help you remember is:
| Copy module | Fetch module: | |
| Direction | Local → Remote | Remote → Local |
| Use case: | Deploying configurations, pushing updates, distributing files | Collecting logs, backing up configs, gathering reports |
| Example | Rolling out a new nginx.conf to all web servers | Pulling error logs from all servers after an incident |
As a refresher, the copy module typically looks like this in a playbook:
- name: Deploy nginx configuration
copy:
src: /local/configs/nginx.conf
dest: /etc/nginx/nginx.conf
owner: nginx
group: nginx
mode: '0644'Note: The owner, group, and mode parameters let you set file permissions during the copy operation.
What is the difference between fetch and synchronize in Ansible?
The synchronize module wraps the powerful rsync tool and is primarily used for synchronizing large sets of files and directories between hosts. It’s extremely handy when you need to copy files directly between target machines within or even across different regions or availability zones, rather than routing everything through your control node.
Although synchronize supports both “push” (default, where files are pushed from the control machine to the target) and “pull” (where files are pulled from the target to the control machine) modes, its real strength lies in remote-to-remote synchronization through delegation.
When writing a playbook, you will typically use it like this:
- name: Sync data between servers
synchronize:
src: /data/uploads/
dest: /backup/uploads/
delegate_to: backup-serverIn the example above, delegate_to: backup-server enables remote-to-remote copying, i.e, the files go directly between the target hosts without passing through your control node.
See also: Ansible Delegate_to: Run Tasks on Delegated Host
What is the difference between fetch and slurp in Ansible?
The official one-line description of the slurp module is “Slurps a file from remote nodes”. Unlike fetch, slurp reads the content of a file as opposed to copying it to the control node. The output is base64 encoded to ensure binary data and special characters are handled safely, and is useful for using the content of a file as input for another task.
The key differentiator is that slurp reads file content while fetch copies the actual file. In practice, you would use it like so:
- name: Read config file content
slurp:
src: /etc/app/version.txt
register: version_file
- name: Use file content in decision
debug:
msg: "Version is {{ version_file.content | b64decode | trim }}"Notice the use of version_file.content | b64decode | trim decodes the base64 content and removes whitespace, making the file’s content usable in subsequent tasks.
Spacelift’s vibrant ecosystem and excellent GitOps flow are helpful for managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can easily create custom workflows based on pull requests and apply any necessary compliance checks for your organization.
Another advantage of using Spacelift is that you can manage infrastructure tools like Ansible, Terraform, OpenTofu, Pulumi, AWS CloudFormation, and even Kubernetes from the same place, combining their stacks with building workflows across tools.
You can bring your own Docker image and use it as a runner to speed up deployments that leverage third-party tools. Spacelift’s official runner image can be found here.
Our latest Ansible enhancements solve three of the biggest challenges engineers face when they are using Ansible:
- Having a centralized place in which you can run your playbooks
- Combining IaC with configuration management to create a single workflow
- Getting insights into what ran and where
Provisioning, configuring, governing, and even orchestrating your containers can be performed with a single workflow, separating the elements into smaller chunks to identify issues more easily.
If you want to learn more about using Spacelift with Ansible, check our documentation, read our Ansible guide, or book a demo with one of our engineers.
The fetch module is one of those core features you don’t use often but can reach for when it counts. In this article, we covered some of the use cases for leveraging it and reviewed how it stacks up against alternative modules.
Instead of alternating between servers to search for information, you centralize the data where you can work with it effectively. Whether you’re building incident response playbooks that automatically collect diagnostic files, creating backup workflows that preserve critical configurations, or setting up monitoring systems that aggregate logs from across your fleet, fetch can make your infrastructure more accessible and manageable.
Manage Ansible better with Spacelift
Managing large-scale playbook execution is hard. Spacelift enables you to automate Ansible playbook execution with visibility and control over resources, and seamlessly link provisioning and configuration workflows.
