Ansible is an open-source automation tool that is used throughout DevOps practices to manage IT infrastructure. The platform is primarily known to simplify any repetitive tasks, configuration management, application deployment, orchestration, cloud provisioning, security and compliance tasks. Ansible is a configuration management tool and not a IaC (Infrastructure as Code) tool. The automation tasks are written in Ansible’s declarative language (YAML) throughout its playbooks.
One of the key features of Ansible is the use of its various modules. Ansible modules are units of code that can execute system commands or control system resources. They can be executed directly on remote hosts or through playbooks. Users can also write their own custom modules.
In this article, we will be focusing on the copy module.
We will cover:
The Ansible Copy module allows you to copy files and recursive directories from a local machine (Ansible control node) to a remote machine. It’s mainly used to move files from one location to another, but it can also handle more complex situations like making sure files have not changed before moving, creating backups, and checking that files are correct after copying.
During the copying process, the Ansible copy module allows you to:
- Back up the original file before it is copied over
- Check if the file/directory already exists to prevent replacement (idempotency)
- Assign the file/directory access to a specific user/group with restricted or open permissions
- Move files within a remote machine
- Perform in-line content changes on existing files in the remote host
- Apply conditional logic
What is the difference between synchronize and copy in Ansible?
Other Ansible modules, such as the synchronize module, also perform file/directory transfers. The synchronize module uses the rsync protocol to transfer and sync multiple files or directories between the controller and remote machines. It is better for moving large sets of files and directories, while the copy module is more suitable for smaller sets of data.
What is the difference between copy and fetch module in Ansible?
Ansible fetch is another module that is similar to the copy module. This module is used to copy files from remote machines to your Ansible control node (remote to local). For example, if you have a config file on a remote server and you want to grab that file and move it over to your Ansible control node, you would use the fetch module. Refer to the Ansible documentation for more details.
Let’s look at some use-case examples.
Example 1: Copy files and directories from local to remote
The simplest use-case of the copy module is to perform a copy over from the Ansible control node to the remote machine. You will need to state the source location as a file location on the ansible control node, and the destination will be the specific location with the file name on your remote machine.
By default, the copied over files will receive the permissions of 644 (-rw-r — r — )
, and the copied over directories will receive the permissions of 755 (drwxr-xr-x)
. This is based on your unmask value on your Linux system, which by default is 022
. If it was modified at some point, the default values for your files and directories might be different.
Here is a simple example of using the Ansible copy module for copying over a text file to the remote host:
---
- name: Ansible Copy File
hosts: test_group
tasks:
- name: Copy 'test-file-1' from Local to Remote location
copy:
src: ~/ansible/files/test-directory-1/test-file-1.txt
dest: /tmp/test-file-1.txt
The first example below will copy over a directory along with all of its files and sub-directories to a non-existing directory on the remote machine.
The second example will copy over just the contents in the source directory to an existing directory in the remote location.
Note become: true
, this is due to copying to a location that requires elevated sudo rights (/opt/)
. Copying over to /tmp
would not require sudo rights (become: true
).
- name: Ansible Copy Directory
hosts: test_group
become: true
tasks:
#example-1: Directory does not exist, creates folder
- name: Copy config directory from local to remote
copy:
src: ~/ansible/files/myapp/opt-tomcat-conf/
dest: /opt/tomcat/opt-tomcat-conf/
#example-2: Directory exists, copies over contents only
- name: Copy config directory contents to /opt/tomcat/conf
copy:
src: ~/ansible/files/myapp/opt-tomcat-conf/
dest: /opt/tomcat/conf/
Example 2: Copy files and directories from local to remote with permissions
It is highly recommended and one of Ansible’s best practices to add Permissions using the copy function. This will ensure the same version of the file or directory with the exact same permissions is assigned across all your remote hosts. It is important to maintain uniformity in scripts, configuration files, and binary files across different environments.
However, it’s just as important to make sure you don’t assign the wrong permissions to sensitive files as it can expose them to unauthorized access.
For the most part, you will always use this format of the Ansible copy module:
- name: Ansible Copy File with Permissions
hosts: test_group
become: true
tasks:
- name: Copy Tomcat context.xml from local to remote with permissions
copy:
src: ~/ansible/files/myapp/opt-tomcat-webapps-manager-meta-inf/context.xml
dest: /opt/tomcat/webapps/manager/META-INF/context.xml
owner: tomcat
group: tomcat
mode: '0644'
- name: Copy myapp /conf files
copy:
src: ~/ansible/files/myapp/opt-tomcat-conf/
dest: /opt/tomcat/conf/
owner: tomcat
group: tomcat
mode: '0644'
Example 3: Move files within the remote machine
The Ansible copy module also allows you to move a file within a remote location from one directory to another. You just need to apply the parameter remote_src: yes
.
It’s important to note that this will only work for files and not directories. This step will also not work if become
is not set to true, and it requires elevated permissions since you are moving over files from a source location that is not your Ansible control node.
- name: Ansible Copy Directory with Permissions
hosts: test_group
tasks:
- name: Move 'Single File-1' within remote location
copy:
src: /tmp/test-file-2.txt
dest: /tmp/copy-temp-directory/test-file-2.txt
owner: root
group: root
mode: '0755'
remote_src: yes
become: true
Example 4: Copy inline content into a specific file
In certain situations, you don’t want to manage the files in your Ansible control node and would rather paste that specific content over to the file in the remote machine. Ansible’s copy inline content feature does just that.
Using the content
parameter of the copy module, you can define the contents of a file inline within an Ansible playbook. This can be very useful for creating or modifying files directly on the remote systems without the need to manage a separate file on the control node. This method also allows you to insert variables directly into the content you are copying over to the file in the remote machine.
In the following example, I am using the copy content parameter to add tomcat-users
to the tomcat-users.xml
file in my remote machine using the specified variables.
Note: In a real-world situation, you would not expose the secret in plain text on the file but would preferably use a secrets manager/vault. This is just for demonstration purposes.
- name: Ansible Copy using inline content
hosts: test_group
vars:
tomcat_conf_dir: /opt/tomcat/conf
tomcat_users:
- username: admin
password: password
roles: "manager-gui,admin-gui"
tasks:
- name: Setup tomcat-users.xml
copy:
dest: "{{ tomcat_conf_dir }}/tomcat-users.xml"
content: |
<?xml version='1.0' encoding='UTF-8'?>
<tomcat-users>
{% for user in tomcat_users %}
<user username="{{ user.username }}" password="{{ user.password }}" roles="{{ user.roles }}"/>
{% endfor %}
</tomcat-users>
- name: Copy cleanup script
copy:
dest: /usr/local/bin/cleanup.sh
content: |
#!/bin/bash
echo "Cleaning up temporary files..."
rm -rf /tmp/*
mode: '0755'
Example 5: Copy with conditional statements
Like anything in code, logic and conditions are the key to a successful deployment. We will always encounter situations where we need the playbook run to fit our needs. We can accomplish that through Ansible’s conditional statements, which allow us to control whether specific tasks should get executed based on the conditions.
For example, you might be concerned about specific files being deployed to the wrong environment. In that case, you can place conditions for specific environments (Dev, QA, UAT, Prod) for that copy job to be triggered. This is one of many methods to deliver files to specific environments.
The same concept applies to different OS versions, security conditions to be met for sensitive files, package dependencies, etc. Conditional statements are a powerful way to ensure that file transfers are efficient, secure, and relevant. You can maintain finer control over your infrastructure, reduce unnecessary data transfers, and ensure each machine receives exactly what it needs based on environment or state. This brings more flexibility to handle complex environments with diverse requirements.
Below is an example of conditional copying using Ansible.
- name: Ansible Copy using Conditional Statements
hosts: test_group
tasks:
#The env variables can be passed in:
#ex: ansible-playbook ~/playbook.yml -e "env=prod"
- name: Copy Apache config for production
copy:
src: prod_httpd.conf
dest: /etc/httpd/conf/httpd.conf
when: env == 'prod'
- name: Copy Apache config for development
copy:
src: dev_httpd.conf
dest: /etc/httpd/conf/httpd.conf
when: env == 'dev'
#Using 'Register' and 'When' conditions for a Tomcat Deployment
- name: Copy WAR file to Tomcat webapps
copy:
src: ~/ansible/files/app.war
dest: /opt/tomcat/webapps/app.war
register: war_copy
- name: Restart Tomcat service if new WAR file deployed
service:
name: tomcat
state: restarted
when: war_copy is changed
- name: Wait for Tomcat to restart
wait_for:
port: 8080
delay: 10
timeout: 120
- name: Verify deployment
uri:
url: http://localhost:8080/app/
return_content: yes
register: http_response
- name: Check if application deployed successfully
debug:
msg: "Application deployed successfully."
when: '"Application Title" in http_response.content'
#Copy over a script that depends on the OS of the remote machine:
- name: Copy CentOS init script
copy:
src: ~/ansible/files/centos_init_script.sh
dest: /etc/init.d/myservice
when: ansible_os_family == "RedHat"
- name: Copy Ubuntu init script
copy:
src: ~/ansible/files/ubuntu_init_script.sh
dest: /etc/init.d/myservice
when: ansible_os_family == "Debian"
Example 6: Copy files with sudo permissions
I briefly went over this during the copying files over with the permissions section, but here, I will dive a little deeper and give more examples of when we should be using this.
For the most part, all Linux systems are tightly locked down. To make any changes to the root directory, setting permissions on files/directories, modifying system files, placing scripts under root, etc., requires you to have elevated permissions. If you don’t, I highly recommend re-evaluating your Linux environment’s security structure.
There are two ways to go about using sudo permissions in your playbooks.
- Assign
become:true
on the entire playbook – it is not recommended since you don’t always need elevated rights to perform certain Ansible tasks. - Place the
become: true
on the specific task that requires elevated rights. This keeps things more secure and utilizes sudo rights when needed instead of blindly giving the entire playbook sudo rights.
The following example covers using sudo permissions on copying over a config file to the root directory:
- name: Copy Apache SSL configuration with sudo
copy:
src: ~/ansible/files/ssl.conf
dest: /etc/apache2/sites-available/ssl.conf
owner: root
group: root
mode: '0644'
become: true
The Ansible Copy module has a few other parameters that should be used throughout your playbook to help you standardize your deployment and prevent any errors.
1) Checksum
With Checksum, you can ensure idempotency to prevent redundant copy tasks from kicking off. This is useful if you want to prevent unnecessary network traffic and save more time during playbook run. You would specify the source path of the file on the Ansible control node, and Ansible will verify the local files checksum against the Checksum of the file at the destination path in the remote machine. Therefore, if the checksums match, Ansible will not copy the file.
2) Backup
Backup is another parameter that is helpful if you want to prevent overwriting the existing file on the remote machine and capturing a backup of the file before the copy task is run. This is essential for preserving the previous state of a file and allowing an easy rollback if new configs cause any errors.
3) Validate
The Validate parameter runs after the copy task is completed. This checks the integrity of the file you just copied over and makes sure it’s functional.
Below is an example of using all these options.
- name: Ansible Checksum, Backup, Validate
hosts: test_group
tasks:
#Checksum
- name: Deploy WAR file to Tomcat
copy:
src: ~/ansible/files/app.war
dest: /opt/tomcat/webapps/app.war
checksum: "{{ lookup('file', '~/ansible/files/app.war') | hash('sha256') }}"
become: true
#Backup
- name: Update Tomcat server configuration
copy:
src: ~/ansible/files/server.xml
dest: /opt/tomcat/conf/server.xml
backup: yes
become: true
#Validate
- name: Copy Tomcat control script
copy:
src: ~/ansible/files/tomcat-control.sh
dest: /usr/local/bin/tomcat-control.sh
mode: '0755'
validate: 'bash -n %s'
become: true
The Ansible Copy module provides many functionalities but does have some limitations. When we are running a copy task, it’s usually to copy a static file from one location to another.
What do we do when we want to customize the file by injecting variables into it and copying it over? We can use the in-line content parameter to perform the variable injects. However, that also comes with limitations since the Ansible Copy content parameter is recommended for small amounts of data. Adding large amounts of text will make our playbooks overcrowded.
This is when we can utilize the Ansible Template module, designed for transferring files that need to be dynamically modified or populated with variables before being copied over on the remote machine.
The Ansible Template module uses Jinja2 templating language. You can copy your script or configuration file over to the .j2 template, and for variable substitution, replace all the areas of the template that need to be dynamic with variables.
Now, you can pass in variables to any script or configuration file you want to copy to the remote machine. This is very useful when you have environment variables you want to pass into your script.
Jinja templating can also be used for other scenarios outside of variable substitution, such as complex logic, loops, filters, and conditional statements.
In the following example, I will go over a small Python script that needs to be environment-specific. I saved the script as a Jinja2 template and copied it over to the remote machine with the necessary variables. For more details, you can refer to the Ansible documentation.
Python script (this is saved as config.properties.py.j2 (jinja2 template)):
from azure.identity import ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient
def get_secret_from_key_vault(secret_name):
key_vault_url = "{{ key_vault_url }}" #only variable i will pass in.
credential = ManagedIdentityCredential()
secret_client = SecretClient(vault_url=key_vault_url, credential=credential)
retrieved_secret = secret_client.get_secret(secret_name)
return retrieved_secret.value
def update_properties_file(secret_name1_value, secret_name2_value):
config_file_path = "/opt/tomcat/.myapp/config.properties"
with open(config_file_path, 'r') as file:
filedata = file.read()
filedata = filedata.replace('DB_ADMIN_PASSWORD', secret_name1_value)
filedata = filedata.replace('DB_READONLY_PASSWORD', secret_name2_value)
with open(config_file_path, 'w') as file:
file.write(filedata)
if __name__ == "__main__":
secret_name1 = "myapp-db-adm-password"
secret_name2 = "myapp-db-adm-ro-password"
password1 = get_secret_from_key_vault(secret_name1)
password2 = get_secret_from_key_vault(secret_name2)
update_properties_file(password1, password2)
Ansible template module:
- name: Ansible Template
hosts: test_group
tasks:
#The override variables are used to declare what keyvault to use upon what environment is specified
- name: Override variables for DEV
set_fact:
key_vault_url: "https://kv-myapp-dev.vault.azure.net/"
when: env == "dev"
- name: Override variables for PROD
set_fact:
key_vault_url: "https://kv-myapp-prod.vault.azure.net/"
when: env == "prod"
#Notice at the destination the file is .py? Jinja2 template is copied over to whatever
#extension you want to make it with the variables being populated.
- name: Copy config.properties.py script
template:
src: ~/ansible/files/myapp/config.properties.py.j2
dest: /opt/tomcat/bin/config.properties.py
owner: tomcat
group: tomcat
mode: '0755'
Spacelift’s vibrant ecosystem and excellent GitOps flow can greatly assist you in managing and orchestrating Ansible. By introducing Spacelift on top of Ansible, you can then easily create custom workflows based on pull requests and apply any necessary compliance checks for your organization.
Another great advantage of using Spacelift is that you can manage different infrastructure tools like Ansible, Terraform, Pulumi, AWS CloudFormation, and even Kubernetes from the same place and combine their Stacks with building workflows across tools.
Our latest Ansible enhancements solve three of the biggest challenges engineers face when they are using Ansible:
- Having a centralized place in which you can run your playbooks
- Combining IaC with configuration management to create a single workflow
- Getting insights into what ran and where
Provisioning, configuring, governing, and even orchestrating your containers can be performed with a single workflow, separating the elements into smaller chunks to identify issues more easily.
Would you like to see this in action – or just want a tl;dr? Check out this video I put together showing you Spacelift’s new Ansible functionality:
If you want to learn more about using Spacelift with Ansible, check our documentation, read our Ansible guide, or book a demo with one of our engineers.
Ansible’s copy module is both flexible and powerful, making it a key tool in IT automation. It’s great for simple tasks like moving files from one location to another, but it can also handle more complex needs like making sure files have not changed before moving them using checksum, creating backups, and checking that files are correct and safe. This module is useful for different situations, whether it’s just copying over files as they are or ensuring that everything is secure and in the right place.
Manage Ansible Better with Spacelift
Managing large-scale playbook execution is hard. Spacelift enables you to automate Ansible playbook execution with visibility and control over resources, and seamlessly link provisioning and configuration workflows.