Backup container for docker-compose application

This image is used as a sidecar container backup persistant data folders of an application.

Functionality

If the BACKUP_HOST variable is set to "false", no backup is run If BACKUP_HOST is not set to "false", the content of this variable is assumed to be the name of the destination server used for the backup. The backup runs at the time which can be defined in the docker-compose.yml with the environment variable CRON_DATE.

Using the docker-compose.yml which defines the services, the backup server shuts down these other services. The backup server then connects to the destination server and creates the directory to save the backups to if it does not already exist. This connection is made with ssh and to change the defualt keyfile for the user the BACKUP_KEY_FILE variabe needs to be set to the desired key.The user to connect to on the destination side of the backup is defined with the BACKUP_USER variable.

A list of files/directories to exclude is checked from the "exclude" files which can be created at the top level of the directory structure of the service.

The backup is then performed for each service using rsync. The rsync command preserves the user and group of any files copied as well as the permissions and extra attributes. Any hard or soft links are also preserved in addition to modification times. The sync runs recursively from the top level of the service's file structure downwards and deletes any files on the destination server that are no longer on the source server. This ensures that when the sync is done, one has the exact same version of the data for that day on both sides. The destination then needs to be backed up at the server level to form a complete backup strategy.

At this point an updated version of a service's docker-image and be checked for and pulled if the DOCKER_COMPOSE_PULL variable is set. The backup server then checks for any update to its own image before starting up the other services again.

Prerequisites

• file structure: all persistant data need to be stored in the same folder, where the docker-compose.yml is stored

  $ tree -AL 1 /opt/docker/jira.domain.com.├── docker-compose.yml├── .exclusions├── jira└── jira-mysql2 directories, 1 file
  

  Copy
• docker-compose.yml: contains the parameterized backup container as well as the containers that are meant to be backed up

  $ cat /opt/docker/jira.domain.com/docker-compose.ymlversion: '3'services:  backup:    image: xalt/backup    environment:      - BACKUP_HOST=10.0.0.1      - BACKUP_SUFFIX=-live      - CRON_DATE=20 1 * * *      - DOCKER_COMPOSE_PULL=true      - TIME_ZONE=Europe/Berlin    restart: always    volumes:      - /home/dummy/.ssh/id_rsa:/tmp/id_rsa:ro      - /opt/docker:/opt/docker      - .:/path      - /var/run/docker.sock:/var/run/docker.sock:rw      - /var/run/docker:/var/run/docker  jira-mysql:    image: xalt/mysql:5.7    environment:      - MYSQL_ROOT_PASSWORD=dummy      - MYSQL_DATABASE=jira      - MYSQL_USER=jira      - MYSQL_PASSWORD=dummy    networks:      jira:        aliases:          - db    restart: always    volumes:      - ./jira-mysql:/var/lib/mysql  jira:    image: xalt/jira    depends_on:      - jira-mysql    environment:      - VIRTUAL_HOST=jira.domain.com      - VIRTUAL_PORT=8080      - RAM_MIN=768M      - RAM_MAX=4096M      - JAVA_OPTS=-Dfile.encoding=utf-8 -Dsun.jnu.encoding=UTF-8      - HTTPS=true      - PROXY_NAME=jira.domain.com    networks:      jira:      reverse-proxy:    restart: always    volumes:      - ./jira:/var/atlassian/jiranetworks:  jira:  reverse-proxy:    external: true
  

  Copy
• An example of how exclusions could be defined

  $ cat .exclusionsjira/exportjira/log
  

  Copy

Parameter

• BACKUP_HOST
  • default: false
  • mandatory: yes
  • needs to be set for the backup to actually happen
  • specifies the host where the persistant data should be stored (can be either DNS name or IP)
• BACKUP_KEY_FILE
  • default: /tmp/id_rsa
  • mandatory: yes
  • can be modified if the keyfile is supposed to be mounted to a different location

    volumes:  - /home/dummy/.ssh/id_rsa:/tmp/id_rsa:ro
    

    Copy
• BACKUP_PATH
  • default: /backup/docker-applications
  • mandatory: yes
  • is used together with the BACKUP_SUFFIX to specify the path where the persistant data is stored on the backup server
  • the whole path in this example would look like this: /backup/docker-applications/jira.domain.com-live
• BACKUP_USER
  • default: root
  • mandatory: yes
  • user that connects to the backup server via ssh and rsync
  • usually it is the easiest way to use root for this connection since only root can create files and folders with the correct ownership and permission on the destination system
• BACKUP_SUFFIX
  • default: ""
  • mandatory: no
  • is used together with the BACKUP_PATH to specify the path where the persistant data is stored on the backup server
  • the whole path in this example would look like this: /backup/docker-applications/jira.domain.com-live
• CRON_DATE
  • default: "0 0 * * *"
  • mandatory: yes
  • this is the cron expression when this backup should be executed
  • every cron expression⁠ that works for ubuntu:18.04 can be used here
• DOCKER_COMPOSE_PULL
  • default: false
  • mandatory: no
  • if set to "true" this container also pulls the newest version of the docker image before starting the containers
• TIME_ZONE
  • default: false
  • mandatory: no
  • here you can specify the time zone of the backup container which makes it easier to maintain the cron jobs
  • the default time zone inside the container is UTC