In this article, we’ll share the basics of what Compose is and how to use it. We’ll also provide some examples of using Compose to deploy popular applications. Let’s get started!
We will cover:
Docker Compose is a tool that makes it easier to create and run multi-container applications. It automates the process of managing several Docker containers simultaneously, such as a website frontend, API, and database service.
Docker Compose allows you to define your application’s containers as code inside a YAML file you can commit to your source repository. Once you’ve created your file (normally named docker-compose.yml), you can start all your containers (called “services”) with a single Compose command.
Compared with manually starting and linking containers, Compose is quicker, easier, and more repeatable. Your containers will run with the same configuration every time—there’s no risk of forgetting to include an important docker run flag.
Compose automatically creates a Docker network for your project, ensuring your containers can communicate with each other. It also manages your Docker storage volumes, automatically reattaching them after a service is restarted or replaced.
Why use Docker Compose?
Most real-world applications have several services with dependency relationships—for example, your app may run in one container, but depend on a database server that’s deployed adjacently in another container. Moreover, services usually need to be configured with storage volumes, environment variables, port bindings, and other settings before they can be deployed.
Compose lets you encapsulate these requirements as a “stack” of containers that’s specific to your app. Using Compose to bring up the stack starts every container using the config values you’ve set in your file. This improves developer ergonomics, supports reuse of the stack in multiple environments, and helps prevent accidental misconfiguration.
What is the difference between Docker and Docker Compose?
Docker is a containerization engine that provides a CLI for building, running, and managing individual containers on your host.
Compose is a tool that expands Docker with support for multi-container management. It supports “stacks” of containers that are declaratively defined in project-level config files.
You can use Docker without Compose; however, adopting Compose when you’re developing a containerized system allows you to deploy your app in any environment with a single command. Whereas the Docker CLI only interacts with one container at a time, Compose integrates with your project and is aware of the relationships between your containers.
Below are some of the benefits of using Docker Compose:
- Fast and easy configuration with YAML scripts
- Single host deployment
- Increased productivity
- Security with isolated containers
Let’s see how to get started using Compose in your own application. We’ll create a simple Node.js app that requires a connection to a Redis server running in another container.
1. Check if Docker Compose is installed
Historically, Docker Compose was distributed as a standalone binary called docker-compose, separately to Docker Engine. Since the launch of Compose v2, the command is now built into the docker CLI as docker compose. Compose v1 is no longer supported.
You should already have Docker Compose v2 available if you’re using a modern version of Docker Desktop or Docker Engine. You can check by running the docker compose version command:
$ docker compose version
Docker Compose version v2.18.1
2. Create Your Application
Begin this tutorial by copying the following code and saving it to app.js inside your working directory:
const express = require("express");
const {createClient: createRedisClient} = require("redis");
(async function () {
const app = express();
const redisClient = createRedisClient({
url: `redis://redis:6379`
});
await redisClient.connect();
app.get("/", async (request, response) => {
const counterValue = await redisClient.get("counter");
const newCounterValue = ((parseInt(counterValue) || 0) + 1);
await redisClient.set("counter", newCounterValue);
response.send(`Page loads: ${newCounterValue}`);
});
app.listen(80);
})();The code uses the Express web server package to create a simple hit tracking application. Each time you visit the app, it logs your hit in Redis, then displays the total number of page loads.
Use npm to install the app’s dependencies:
$ npm install express redis
Next, copy the following Dockerfile content to the Dockerfile in your working directory:
FROM node:18-alpine
EXPOSE 80
WORKDIR /app
COPY package.json .
COPY package-lock.json .
RUN npm install
COPY app.js .
ENTRYPOINT ["node", "app.js"]Compose will build this Dockerfile later to create the Docker image for your application.
3. Create a Docker Compose file
Now, you’re ready to add Compose to your project. This app is a great candidate for Compose because you need two containers to successfully run the app:
- Container 1 – The Node.js server app you’ve created.
- Container 2 – A Redis instance for your Node.js app to connect to.
Creating a docker-compose.yml file is the first step in using Compose. Copy the following content and save it to your own docker-compose.yml—don’t worry, we’ll explain it below:
services:
app:
image: app:latest
build:
context: .
ports:
- ${APP_PORT:-80}:80
redis:
image: redis:6Let’s dive into what’s going on here.
- The top-level
servicesfield is where you define the containers that your app requires. - Two services are specified for this app:
app(your Node.js application) andredis(your Redis server). - Each service has an
imagefield that defines the Docker image the container will run. In the case of the app service, it’s the customapp:latestimage. As this may not exist yet, thebuildfield is set to tell Compose it can build the image using the working directory (.) as the build context. Theredisservice is simpler, as it only needs to reference the official Redis image on Docker Hub. - The
appservice has aportsfield that declares the port bindings to apply to the container, similarly to the-pflag ofdocker run. An interpolated variable is used; this means that the port number given by yourAPP_PORTenvironment variable will be supplied when it’s set, with a fallback to the default port 80.
From this explanation, you can see that the Compose file contains all the configuration needed to launch a functioning deployment of the app.
4. Bring Up Your Containers
Now, you can use Compose to bring up the stack!
Call docker compose up to start all the services in your docker-compose.yml file. In the same way as when calling docker run, you should add the -d argument to detach your terminal and run the services in the background:
$ docker compose up -d
[+] Building 0.5s (11/11) FINISHED
...
[+] Running 3/3
✔ Network node-redis_default Created 0.1s
✔ Container node-redis-redis-1 Started 0.7s
✔ Container node-redis-app-1 Started 0.6s Because your app’s image doesn’t exist yet, Compose will first build it from your Dockerfile. It’ll then run your stack by creating a Docker network and starting your containers.
Visit localhost in your browser to see your app in action.
Try refreshing the page a few times—you’ll see the counter increase as each hit is recorded in Redis.
In the app.js file, we set the Redis client URL to redis:6379. The redis hostname matches the name of the redis service in docker-compose.yml.
Compose uses the names of your services to assign your container hostnames; because the containers are part of the same Docker network, your app container can resolve the redis hostname to your Redis instance.
5. Manage your Docker Compose stack – commands
Now that you’ve started your app, you can use other Docker Compose commands to manage your stack:
docker compose ps
You can see the containers that Compose has created by running the ps command; the output matches that produced by docker ps:
$ docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
node-redis-app-1 app:latest "node app.js" app 12 minutes ago Up 12 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp
node-redis-redis-1 redis:6 "docker-entrypoint.s…" redis 12 minutes ago Up 12 minutes 6379/tcpdocker compose stop
This command will stop all the Docker containers created by the stack. Use docker compose start to restart them again afterwards.
docker compose restart
The restart command forces an immediate restart of your stack’s containers.
docker compose down
Use this command to remove the objects created by docker compose up. It will destroy your stack’s containers and networks.
Volumes are not deleted unless you set the -v or --volumes flag. This prevents accidental loss of persistent data.
docker compose logs
View the output from your stack’s containers with the logs command. This collates the standard output and error streams from all the containers in the stack. Each log line is tagged with the name of the container that created it.
docker compose build
You can force a rebuild of your images with the build command. This will rebuild the images for the services in your docker-compose.yml file that include the build field in their configuration.
Afterwards, you can repeat the docker compose up command to restart your stack with the rebuilt images.
docker compose push
After building your images, use push to push them all to their remote registry URLs. Similarly, docker compose pull will retrieve the images needed by your stack, without starting any containers.
6. Use Compose Profiles
Sometimes, a service in your stack might be optional. For example, you could expand the demo application to support the use of alternative database engines instead of Redis. When a different engine is used, you wouldn’t need the Redis container.
You can accommodate these requirements using Compose’s profiles feature. Assigning services to profiles allows you to manually activate them when you run Compose commands:
services:
app:
image: app:latest
build:
context: .
ports:
- ${APP_PORT:-80}:80
redis:
image: redis:6
profiles:
- with-redisThis docker-compose.yml file assigns the redis service to a profile called with-redis. Now the Redis container will only be considered when you include the --profile with-redis flag with your docker compose commands:
# Does not start Redis
$ docker compose up -d
# Will start Redis
$ docker compose --profile with-redis up -d
7. Understand Docker Compose projects
Projects are an important concept in Docker Compose v2. Your “project” is your docker-compose.yml file and the resources it creates.
Compose uses your working directory’s docker-compose.yml file by default. It assumes your project’s name is equal to your working directory’s name. This name prefixes Docker objects that Compose creates, such as your containers and networks. You can override the project name by setting Compose’s --project-name flag or by including a top-level name field in your docker-compose.yml file:
name: "demo-app"
services:
...You can run Docker Compose commands from outstide your project’s working directory by setting the --profile-directory flag:
$ docker compose --profile-directory=/path/to/directory psThe flag accepts a path to a docker-compose.yml file, or a directory that contains one.
8. Set Docker Compose environment variables
One of Docker Compose’s advantages is the ease with which you can set environment variables for your services.
Instead of manually repeating docker run -e flags, you can define variables in your docker-compose.yml file, set default values, and facilitate simple overrides:
services:
app:
image: app:latest
build:
context: .
environment:
- DEV_MODE
- REDIS_ENABLED=1
- REDIS_HOST_URL=${REDIS_HOST:-redis}
ports:
- ${APP_PORT:-80}:80This example demonstrates a few different ways to set a variable:
DEV_MODE– Not supplying a value means Compose will take it from the environment variable set in your shell.REDIS_ENABLED=1– Setting a specific value will ensure it’s used (unless it’s overridden later on).REDIS_HOST_URL=${REDIS_HOST:-redis}– This interpolated example assignsREDIS_HOST_URLto the value of yourREDIS_HOSTshell variable, falling back to a default value ofredis.${APP_PORT:-80}– Environment variables set in your shell can be interpolated into arbitrary fields in yourdocker-compose.ymlfile, permitting easy customization of your stack’s configuration.
Furthermore, you can override these values by creating an environment file—either .env, which is automatically loaded, or another file which you pass to Compose’s --env-file flag:
$ cat config.env
DEV_MODE=1
APP_PORT=8000
$ docker compose --env-file=config.env up -d9. Control service startup order
Many applications require their components to wait for dependencies to be ready—in our demo app above, the Node application will crash if it starts before the Redis container is live, for example.
You can control the order in which services start by setting the depends_on field in your docker-compose.yml file:
services:
app:
image: app:latest
build:
context: .
depends_on:
- redis
ports:
- ${APP_PORT:-80}:80
redis:
image: redis:6Now Compose will delay starting the app service until the redis container is running. For greater safety, you can wait until the container is passing its healthcheck by using the long form of depends_on instead:
services:
app:
image: app:latest
build:
context: .
depends_on:
redis:
condition: service_healthy
ports:
- ${APP_PORT:-80}:80
redis:
image: redis:6
Do you want to see Compose in action, deploying some real-world applications? Here are some examples!
WordPress (Apache/PHP and MySQL) with Docker Compose
WordPress is the most popular website content management system (CMS). It’s a PHP application that requires a MySQL or MariaDB database connection. Consequently, there are two containers to deploy with Docker:
- WordPress application container – Serves WordPress using PHP and the Apache web server.
- MySQL database container – Runs the database server that the WordPress container will connect to.
The following docker-compose.yml file can be used to create these containers and bring up a functioning WordPress site:
services:
wordpress:
image: wordpress:${WORDPRESS_TAG:-6.2}
depends_on:
- mysql
ports:
- ${WORDPRESS_PORT:-80}:80
environment:
- WORDPRESS_DB_HOST=mysql
- WORDPRESS_DB_USER=wordpress
- WORDPRESS_DB_PASSWORD=${DATABASE_USER_PASSWORD}
- WORDPRESS_DB_NAME=wordpress
volumes:
- wordpress:/var/www/html
restart: unless-stopped
mysql:
image: mysql:8.0
environment:
- MYSQL_DATABASE=wordpress
- MYSQL_USER=wordpress
- MYSQL_PASSWORD=${DATABASE_USER_PASSWORD}
- MYSQL_RANDOM_ROOT_PASSWORD="1"
volumes:
- mysql:/var/lib/mysql
restart: unless-stopped
volumes:
wordpress:
mysql:This Compose file contains everything required to configure a WordPress deployment with a connection to a MySQL database.
Environment variables are set to configure the MySQL instance and supply credentials to the WordPress container.
Docker volumes are also defined to store the persistent data created by the containers, independently of their container lifecycles.
Now you can bring up MySQL with a simple command—the only environment variable you need is the wordpress database user’s password:
$ DATABASE_USER_PASSWORD=abc123 docker compose up -dVisit localhost in your browser to access your WordPress site’s installation page:
Prometheus and Grafana with Docker Compose
Prometheus is a popular time-series database used to collect metrics from applications. It’s often paired with Grafana, an observability platform that allows data from Prometheus and other sources to be visualized on graphical dashboards.
Let’s use Docker Compose to deploy and connect these applications.
First, create a Prometheus config file—this configures the application to scrape its own metrics, which supplies data for our demonstration purposes:
scrape_configs:
- job_name: prometheus
honor_timestamps: true
scrape_interval: 10s
scrape_timeout: 5s
metrics_path: /metrics
scheme: http
static_configs:
- targets:
- localhost:9090Save the file to prometheus/prometheus.yml in your working directory.
Next, create a Grafana file that will configure the application with a data source connection to your Prometheus instance:
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
url: http://prometheus:9090
access: proxy
isDefault: true
editable: true
This file should be saved to grafana/grafana.yml in your working directory.
Finally, copy the following Compose file and save it to docker-compose.yml:
services:
prometheus:
image: prom/prometheus:latest
command:
- "--config.file=/etc/prometheus/prometheus.yml"
ports:
- 9090:9090
volumes:
- ./prometheus:/etc/prometheus
- prometheus:/prometheus
restart: unless-stopped
grafana:
image: grafana/grafana:latest
ports:
- ${GRAFANA_PORT:-3000}:3000
environment:
- GF_SECURITY_ADMIN_USER=${GRAFANA_USER:-admin}
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD:-grafana}
volumes:
- ./grafana:/etc/grafana/provisioning/datasources
restart: unless-stopped
volumes:
prometheus:Use docker compose up to start the services and optionally set custom user credentials for your Grafana account:
$ GRAFANA_USER=demo GRAFANA_PASSWORD=foobar docker compose up -dNow visit localhost:3000 in your browser to login to Grafana:
In this article, you’ve learned how Docker Compose allows you to work with stacks of multiple Docker containers. We’ve shown how to create a Compose file and looked at some examples for WordPress and Prometheus/Grafana.
Now you can use Compose to interact with your application’s containers, while avoiding error-prone docker run CLI commands. A single docker compose up will start all the containers in your stack, guaranteeing consistent configuration in any environment.
You can also take a look at how Spacelift uses Docker containers to run CI jobs. Spacelift offers full flexibility when it comes to customizing your workflow. You have the possibility of bringing your own Docker image and use it as a runner to speed up the deployments that leverage third party tools. Spacelift’s official runner image can be found here.
The Most Flexible CI/CD Automation Tool
Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.
