alwynpan/db-backup repository overview

⁠github.com/tiredofit/docker-db-backup

⁠About

This will build a container for backing up multiple types of DB Servers

Currently backs up CouchDB, InfluxDB, MySQL, Microsoft SQL, MongoDB, Postgres, Redis servers.

• dump to local filesystem or backup to S3 Compatible services, and Azure.
• select database user and password
• backup all databases, single, or multiple databases
• backup all to seperate files or one singular file
• choose to have an MD5 or SHA1 sum after backup for verification
• delete old backups after specific amount of time
• choose compression type (none, gz, bz, xz, zstd)
• connect to any container running on the same system
• Script to perform restores
• Zabbix Monitoring capabilities
• select how often to run a dump
• select when to start the first dump, whether time of day or relative to container start time
• Execute script after backup for monitoring/alerting purposes

⁠Maintainer

• Dave Conroy⁠

⁠Table of Contents

• About⁠
• Maintainer⁠
• Table of Contents⁠
• Prerequisites and Assumptions⁠
• Installation⁠
  • Build from Source⁠
  • Prebuilt Images⁠
    • Multi Architecture⁠
• Configuration⁠
  • Quick Start⁠
  • Persistent Storage⁠
  • Environment Variables⁠
    • Base Images used⁠
    • Container Options⁠
  • Database Specific Options⁠
    • For Influx DB2:⁠
  • Scheduling Options⁠
  • Backup Options⁠
    • Backing Up to S3 Compatible Services⁠
    • Upload to a Azure storage account by blobxfer⁠
• Maintenance⁠
  • Shell Access⁠
  • Manual Backups⁠
  • Restoring Databases⁠
  • Custom Scripts⁠
    • Path Options⁠
    • Pre Backup⁠
    • Post backup⁠
• Support⁠
  • Usage⁠
  • Bugfixes⁠
  • Feature Requests⁠
  • Updates⁠
• License⁠

⁠Prerequisites and Assumptions

• You must have a working connection to one of the supported DB Servers and appropriate credentials

⁠Installation

⁠Build from Source

Clone this repository and build the image with docker build <arguments> (imagename) .

⁠Prebuilt Images

Builds of the image are available on Docker Hub⁠

Builds of the image are also available on the Github Container Registry⁠

docker pull ghcr.io/tiredofit/docker-db-backup:(imagetag)

The following image tags are available along with their tagged release based on what's written in the Changelog⁠:

docker pull docker.io/tiredofdit/db-backup:(imagetag)

⁠Multi Architecture

Images are built primarily for amd64 architecture, and may also include builds for arm/v7, arm64 and others. These variants are all unsupported. Consider sponsoring⁠ my work so that I can work with various hardware. To see if this image supports multiple architecures, type docker manifest (image):(tag)

⁠Configuration

⁠Quick Start

• The quickest way to get started is using docker-compose⁠. See the examples folder for a working docker-compose.yml⁠ that can be modified for development or production use.

• Set various environment variables⁠ to understand the capabilities of this image.

• Map persistent storage⁠ for access to configuration and data files for backup.

• Make networking ports⁠ available for public access if necessary

⁠Persistent Storage

The following directories are used for configuration and can be mapped for persistent storage.

⁠Environment Variables

⁠Base Images used

This image relies on an Alpine Linux⁠ base image that relies on an init system⁠ for added capabilities. Outgoing SMTP capabilities are handlded via msmtp. Individual container performance monitoring is performed by zabbix-agent⁠. Additional tools include: bash,curl,less,logrotate, nano.

Be sure to view the following repositories to understand all the customizable options:

⁠Container Options

⁠Database Specific Options

⁠For Influx DB2:

Your Organization will be mapped to DB_USER and your root token will need to be mapped to DB_PASS. You may use DB_NAME=ALL to backup the entire set of databases. For DB_HOST use syntax of http(s)://db-name

⁠Scheduling Options

• You may need to wrap your DB_DUMP_BEGIN value in quotes for it to properly parse. There have been reports of backups that start with a 0 get converted into a different format which will not allow the timer to start at the correct time.

⁠Backup Options

• When using compression with MongoDB, only GZ compression is possible.

⁠Backing Up to S3 Compatible Services

If BACKUP_LOCATION = S3 then the following options are used.

• When S3_KEY_ID and/or S3_KEY_SECRET is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.

⁠Upload to a Azure storage account by blobxfer

Support to upload backup files with blobxfer⁠ to the Azure fileshare storage.

If BACKUP_LOCATION = blobxfer then the following options are used.

⁠Maintenance

⁠Shell Access

For debugging and maintenance purposes you may want access the containers shell.

bash docker exec -it (whatever your container name is) bash

⁠Manual Backups

Manual Backups can be performed by entering the container and typing backup-now

• Recently there was a request to have the container work with Kubernetes cron scheduling. This can theoretically be accomplished by setting the container MODE=MANUAL and then setting MANUAL_RUN_FOREVER=FALSE - You would also want to disable a few features from the upstream base images specifically CONTAINER_ENABLE_SCHEDULING and CONTAINER_ENABLE_MONITORING. This should allow the container to start, execute a backup by executing and then exit cleanly. An alternative way to running the script is to execute /etc/services.available/10-db-backup/run.

⁠Restoring Databases

Entering in the container and executing restore will execute a menu based script to restore your backups - MariaDB, Postgres, and Mongo supported.

You will be presented with a series of menus allowing you to choose:

• What file to restore
• What type of DB Backup
• What Host to restore to
• What Database Name to restore to
• What Database User to use
• What Database Password to use
• What Database Port to use

The image will try to do autodetection based on the filename for the type, hostname, and database name. The image will also allow you to use environment variables or Docker secrets used to backup the images

The script can also be executed skipping the interactive mode by using the following syntax/

`restore <filename> <db_type> <db_hostname> <db_name> <db_user> <db_pass> <db_port>`

If you only enter some of the arguments you will be prompted to fill them in.

⁠Custom Scripts

⁠Path Options

⁠Pre Backup

If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of .sh in the location defined in SCRIPT_LOCATION_PRE. See the following example to utilize:

$ cat pre-script.sh
##!/bin/bash

# #### Example Pre Script
# #### $1=DB_TYPE (Type