ogivuk/rsync repository overview

⁠rsync

rsync is an open source utility that provides fast incremental file transfer.

If you are into Dockerizing everything or you just want to have a better view over the rsync process (e.g., to see it in your cluster visualizer), this repository provides you with a Docker image to run the rsync process.

The provided image is open-source and built from scratch with the goal to enable you to run a stateless and an immutable container, according to the best practices.

With this image you can:

• run a simple one-time rsync job for local drives
• run a rsync job for remote drives using ssh (included in the image)
• run scheduled rsync jobs using cron (included in the image)

Supported architectures:

• the image supports multiple architectures: x86-64 and arm32
• the docker manifest is used for multi-platform awareness
• by simply pulling or running ogivuk/rsync:latest, the correct image for your architecture will be retreived

The image is based on alpine image and it includes:

• rsync
• openssh-client for remote sync over ssh
• tzdata for easier setting up of local timezone (for file timestamps)
• cron (included in the alpine) for scheduling regular back-ups
• rsync.sh script that prepares the cron job and starts the cron daemon

⁠Usage

⁠Quick Start: one time run and sync of local folder

docker run --rm \
    --name=rsync \
    --volume /path/to/source/data:/data/src \
    --volume /path/to/destination/data:/data/dst \
    ogivuk/rsync [OPTIONS] /data/src/ /data/dst/

Replace:

• /path/to/source/data with the source folder to be copied or backed-up
• /path/to/destination/data with the destination folder
• [OPTIONS] with desired rsync optional arguments

⁠Use with cron or ssh

⁠Step 1. Prepare the setup (First time only)

• create a folder, for example ~/rsync, that will be later mounted in the container

  • this folder is intended to hold supporting files such as log files, crontab file, and any supporting rsync files (e.g., list of files to rsync)
  • the subfolder logs, for example ~/rsync/logs is suggested for the log files
  • important note: for Docker Swarm, this directory needs to be available on all nodes in Docker swarm, e.g., via network shared storage

  mkdir -p ~/rsync/logs
  

  Copy
  • you can replace ~/rsync with any other desired location

⁠Step 2. Run

• run as a container:

  docker run --rm \
      --name=rsync \
      --env TZ="Europe/Zurich" \
      --env RSYNC_CRONTAB="crontab" \
      --volume ~/rsync:/rsync
      --volume /path/to/source/data:/data/src \
      --volume /path/to/destination/data:/data/dst \
      ogivuk/rsync
  

  Copy

• run as a swarm service:

  docker service create \
      --name=rsync \
      --env TZ="Europe/Zurich" \
      --env RSYNC_CRONTAB="crontab" \
      --mount type=bind,src=~/rsync,dst=/rsync \
      --mount type=bind,src=/path/to/source/data,dst=/data/src \
      --mount type=bind,src=/path/to/destination/data,dst=/data/dst \
      ogivuk/rsync
  

  Copy

Remarks:

• rsync will not be run by default, you need to be specify the rsync command with all its arguments in the crontab, or in a script called in the crontab
• any later changes to the crontab file require the service to be restarted, and that's why consider to define the rsync job in a script that is called in the crontab
• when defining the rsync arguments, including source and destination, do that from the perspective of the container (/data/src, /data/dst)
• more volumes can be mount binded if needed
• the ssh client is included in the image in case your source or destination is a remote host
  • ssh required files (private key, known_hosts, ssh_config) needs to be stored in a folder mounted to the container, for example in ~/rsync/ssh/
  • you can define the ssh connection in a ssh_config file
  • rsync option -e "ssh -F /rsync/ssh/ssh_config" instructs rsync to use the ssh with the ssh_config for the remote sync