Backup Docker Volumes with Duplicati and Docker Compose

Backing up your Docker persistent volumes is a crucial part of disaster recovery for container filesystem resources that exist on your Docker hosts. This guide looks into installing Duplicati with Docker Compose. We will focus on backing up container data and volume data.

What is Duplicati?

Duplicati is an open source tool that allows us to do something very much needed and that is to create backup jobs for Docker containers. It also can perform incremental backups of all the Docker volumes used in the environment. It is also able to work with a lot of file transfer protocols, including FTP, SSH, WebDAV, and even cloud services. If you want to use cloud services, these include: Microsoft OneDrive, Amazon Cloud Drive & S3, Google Drive, Box.com, Mega, hubiC, and others.

Duplicati features

It also has the following features:

• Run encrypted incremental backups
• Backups using protocols like ftp ssh
• Create local backups and store local backups to a directory
• Create and send backup of your Docker volumes to the cloud
• Backup encryption
• Scheduling and retention

Duplicati Docker Compose install

Let’s look at the steps you need to do to install Duplicati on your Docker host. Most likely, you already have Docker installed along with Docker Compose if you want to back up your host’s Docker volumes. However, remember that you will need to have Docker and Docker Compose installed as a prerequisite.

Preparing the Duplicati Docker Environment

Run through the following commands below to install Docker and Docker Compose in Ubuntu Server.

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install the latest version of Docker and Docker Compose
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Steps for Duplicati Docker Installation

1. Docker Compose File Creation: Initiate by creating a docker-compose.yml file. This file is the heart of your Duplicati Docker setup, defining how your Docker container will operate.
2. Defining the Duplicati Docker Service: Use the Docker image tag lscr.io linuxserver duplicati:latest for the most updated version. This tag ensures you’re using a multi-arch image, suitable for different systems​. The available image tags also include the “development” image tag that is the canary releases of Duplicati.
3. Configuring Environment Variables: Use the appropriate environment variable for Duplicati, such as PUID and PGID, which defines the permissions within the Docker environment.
4. Mapping Volumes and Ports: Properly map Docker volumes for storing Duplicati’s configuration and backup files. Additionally, configure the ports to access Duplicati’s user interface.
5. Launching Duplicati Docker Container: Execute the docker-compose up command to start your Duplicati service within the Docker container.

As an example, you can use the below Docker Compose code. Note below, under the volumes configuration. This configuration line is where you will setup your volumes for Duplicati, and then the volumes for where you want to store backups and the paths to Docker volumes that you want to back up.

---
version: "2.1"
services:
  duplicati:
    image: lscr.io/linuxserver/duplicati:latest
    container_name: duplicati
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - CLI_ARGS= #optional
    volumes:
      - /path/to/appdata/config:/config
      - /path/to/backups:/backups
      - /path/to/source:/source
    ports:
      - 8200:8200
    restart: unless-stopped

After you create the docker-compose file, you can bring up the containers using Docker compose:

docker-compose up -d

Docker CLI installation

You can also use the Docker CLI to install Duplicati instead of Docker Compose. To use the Docker CLI, use the following:

docker run -d \
  --name=duplicati \
  -e PUID=1000 \
  -e PGID=1000 \
  -e TZ=Etc/UTC \
  -e CLI_ARGS= `#optional` \
  -p 8200:8200 \
  -v /path/to/appdata/config:/config \
  -v /path/to/backups:/backups \
  -v /path/to/source:/source \
  --restart unless-stopped \
  lscr.io/linuxserver/duplicati:latest

Browse to Duplicati web interface

Now that we have the container spun, we can browse out to the port configured for the web interface. In this case, we accepted the default port 8200.

You will be asked if you are using a multi-user system or a single user.

Now, we can view the interface. It is very intuitive and simple.

Let’s see how to add a backup.

Create backup jobs in Duplicati

The process to create a backup job is easy. This is done using the Add Backup wizard in Duplicati.

When you click the Add backup option, it will launch the Add a new backup wizard. Here we will select Configure a new backup.

This is a 5-step process. The first screen has you name the backup job. It defaults to AES-256 encryption built in. You will need to generate or create your own password for the encryption process.

Next, we will select the destination. If you click the drop-down on the Storage type, we get an idea of just how many options are available with Duplicati.

I am choosing the Local folder or drive for the test. The destination I am choosing should look familiar from the Docker Compose file.

There are many advanced options to use as well if you choose.

On step 3, we choose the Source data. Scroll down in the folder browser to source. This is the source folder we configured in our Docker Compose file.

On step 4 choose your Schedule. Below are the defaults when you get to this step. By default, it runs every day.

Finally, on the options screen in step 5, we set the remote volume size and the backup retention. Configure your settings here and click Save.

What is the remote volume size?

Understanding Block Size

• What’s a Block? Think of blocks as small pieces of your files. Duplicati breaks your files into these pieces, or “chunks”, for backup.
• Default Chunk Size: By default, each chunk is 100 kilobytes (kb). If a file is smaller than this, or doesn’t split evenly into 100kb chunks, you’ll end up with a smaller final chunk.
• Important: Once you back up with a certain chunk size, you can’t change it later. Duplicati will stop you from doing this to avoid errors.

Choosing Larger Block Sizes

• Fewer, Bigger Chunks: Larger chunk sizes mean you get fewer but bigger pieces. This can be good if your files are big. But, there’s a catch with smaller sizes; they can slow things down.
• Speed and Storage: Bigger chunks mean less clutter in Duplicati’s memory (like a smaller index in a book), especially if you’re not using a super-fast SSD drive.
• Restoration and Storage Impact: Bigger chunks are handy when restoring files, as they can be put back together faster. However, if you change just a small part of a file, Duplicati has to replace the whole chunk, which could use more storage and internet data.
• Deduplication Efficiency: Bigger chunks make it harder for Duplicati to find duplicate pieces, as each chunk contains more data.

Running a Duplicati backup job

Now that we have our Duplicati backup job created, we can run the job.

The backup job begins to run. In addition to the progress bar, it will tell you how many files and the data size left in the backup job which is a nice touch.

It took my backup job around 6:30 minutes to run.

Restore Docker volumes using Duplicati

Let’s look at the Duplicati restore wizard to restore data that we have captured with the Duplicati backup.

The first screen asks you where you want to restore from. The options are:

• Direct restore from backup files
• Restore from configuration
• Your Backup name

Next, choose the data you want to restore and then click Continue.

Finally, select your Restore options. These include:

• Where do you want to restore the files to?
• How do you want to handle existing files
• Permissions options

Wrapping up Duplicati Docker installation

Duplicati is a slick way to back up your Docker volumes and use modern storage locations to store your data. The Duplicati solution is easy to stand up, it has a ton of features, it is open-source, and can target cloud storage services. It really is one of the most fully-featured Docker container volume backup solutions you will find, outside of some type of commercial offering. I think it is a great way to bolster normal backup routines of an entire Docker host virtual machine by having backups of your Docker data by Duplicati stored in a different location.