Public | Automated Build

Last pushed: 2 days ago
Short Description
A backup service for an Imixs-Workflow instance. It backups a PSQL database and also file resources.
Full Description

imixs/backup

This Docker image provides a backup service to backup a PSQL database. The service can be added into a docker stack with an PSQL instance to backup the database periodically.
All backup files are organized in a backup directory and will be automatically transfered to a backup space if defined.
The service is designed to backup only one database. In case you want to use this service to backup a complete PSQL server, than you should use the command "pg_dumpall" instead of "pg_dump". See the script backup.sh for details.

Features

  • backup PostgreSQL database
  • sftp/scp support to move backups to an external backup space
  • chron job
  • restore feature.

Environment

The imixs/backup image is based on the official postgres image.

imixs/backup provides the following environment variables which need to be set during container startup:

  • SETUP_CRON - the cron timer setting (e.g. "0 3 *")
  • BACKUP_SERVICE_NAME - name of the backup service (defines the target folder on FTP space)
  • BACKUP_POSTGRES_HOST - postgres server
  • BACKUP_POSTGRES_USER - postres database user
  • BACKUP_POSTGRES_PASSWORD - postgres user password
  • BACKUP_POSTGRES_DB - postgres database
  • BACKUP_SPACE_HOST - backup space connected via SFTP/SCP
  • BACKUP_SPACE_USER - backup space user
  • BACKUP_LOCAL_ROLLING - number of backup files to be kept locally
  • BACKUP_SPACE_ROLLING - number of backup files to be kept in the backup space

All backups are located in the following local directory

/root/backups/

In the backup space, the files are located at:

/imixs-cloud/$BACKUP_SERVICE_NAME/

Each backup file has a time stamp prefix indicating the backup time:

2018-01-07_03:00_pgdump.sql

Cron

Based on the cron settings provided in the environment variable "BACKUP_CRON" the backup_init script starts a cron job to schedule the backup.sh script.

Example:

 # Run every day at 03:00
 0 3 * * *   

See details here.

Scripts

All backup scripts are located in the root home directory (/root/).

  • backup_init.sh - initializes the backup service via cron
  • backup.sh - the backup script
  • restore.sh - the restore script
  • backup_get.sh - to get a file form the remote backup space

The scripts can be called manually:

docker exec -it 2f4b2feaa412 /root/backup.sh

Rolling Backup Files

The backup script automatically holds a number of backup files locally. The default number of files to keep is set to 5. You can change this parameter with the environment variable "BACKUP_LOCAL_ROLLING".

The Backup Space

In case the optional environment variable "BACKUP_SPACE_HOST" is provided, the service will push backup files automatically into a backup space via SFTP/SCP.
The backup directory on the backup space is

/imixs-cloud/$BACKUP_SERVICE_NAME/....

The optional environment variable "BACKUP_SERVICE_NAME" can be set to name the backup directory on the backup space. If no service name is set, the docker container ID will be used instead.

Create a SSH Key

To transfers files to the backup space this service uses SFTP/SCP. For this reason a RFC4716 Public Key need to be provided on the backup space.

The backup service expects that a private key file is provided by a docker secret. Docker secrets can be used only in docker swarm. So in this case you are forced to run the backup service in a docker swarm.

To copy a ssh key provided in the file _/root/.ssh/backupspacersa into a docker secret run:

docker secret create backupspace_key /root/.ssh/backupspace_rsa

You can add the key as an environment variable to the stack definition:

version: '3.1'

services:
....
   backup:
    image: imixs/backup:latest
    environment:
     .....
     BACKUP_SPACE_KEY_FILE: "/run/secrets/backupspace_key"
   secrets:
     ...
     - backupspace_key
....
 secrets:
   backupspace_key:
     external: true
....

Running the service

The imixs/backup service is supposed to be run as part of a docker service stack. This means that the service is included in a docker-compose.yml file which already contains PQSL Database Server and a Wildfly Application Server.
In this scenario the wildfly service access the PSQL server via the internal overlay network. In the same way the backup service can access the database. The integration of the backup service into a docker-compose.yml file looks like this:

...
  backup:
    image: imixs/backup
    environment:
      SETUP_CRON: "0 3 * * *"
      BACKUP_POSTGRES_USER: "postgres"
      BACKUP_POSTGRES_PASSWORD: "xxxxxxxxxx"
      BACKUP_POSTGRES_HOST: "db"
      BACKUP_POSTGRES_DB: "my-database"
      BACKUP_LOCAL_ROLLING: "5"
....

If you add a backup space the following optional environment settings are needed:

....
      BACKUP_SERVICE_NAME: "my-app"
      BACKUP_SPACE_HOST: "my-backup.org"
      BACKUP_SPACE_USER: "yyyy"
      BACKUP_SPACE_KEY_FILE: "/run/secrets/backupspace_key"
....

Manual Backup

To start a backup manually from inside the container run:

./backup.sh

You can start a manual backup from outside with the following command

docker exec -it 82526abbabfe /root/backup.sh

(You need to replace the container ID with the id of your backup service.)

Restore

All backup files are stored in the folder /root/backups/ and start with a time stamp in ISO format

You can verify the current available backups from outside with the command:

docker exec -it 82526abbabfe ls -la /root/backups

(You need to replace the container ID with the id of your backup service.)

To restore a backup run the script restore.sh followed by the timestamp

./restore.sh 2018-01-05_03:00

Note: After a restore it is recommended to restart the wildfly container because wildfly uses JPA with a internal cache. To discard this cache a restore or a redeployment is needed.

Also you can trigger a restore from outside with the command:

docker exec -it 82526abbabfe ls -la /root/restore.sh 2018-01-05_03:00

Get a Backup File form the Backup Space

In case you need to pull a backup file from the backup space run the script backup_get.sh :

backup_get.sh /imixs-cloud/SERVICE-ID/BACKUPFILE BACKUPFILE

You need to specify the source file located in your backup space. With SFTP you can print the directory content from the FTP Space:

docker exec -it 82526abbabfe echo ls / | sftp $BACKUP_SPACE_USER@$BACKUP_SPACE_HOST

After the script is completed, the file is written into the directory /root/backups/.
You can run a restore on this file.

Contribute

The source is available on Github. Please report any issues.

To build the image from the Dockerfile run:

docker build --tag=imixs/backup .
Docker Pull Command
Owner
imixs
Source Repository