Sign inSign up

viperadnan/dbstash

By viperadnan

Updated 6 months ago

Database backup via rclone. Stream dumps directly to any cloud storage.

Image
0

3.6K

viperadnan/dbstash repository overview

dbstash

Build Status Docker Pulls Docker Image Size GitHub Stars License GitHub Release Go Version

Database backup via rclone. Stream database dumps directly to any cloud storage without intermediate files or local disk pressure. Run as a standalone CLI or as Docker containers.


Features

  • Zero Disk Usage — Stream dumps directly to cloud storage (stream mode)
  • CLI and Docker — Run as a standalone binary with flags or as purpose-built Docker images
  • Any Cloud Storage — Works with S3, GCS, Azure, Dropbox, 40+ backends via rclone
  • Multiplatform — Built for linux/amd64 and linux/arm64
  • Flexible Scheduling — Cron expressions or one-time backups
  • Secrets Support — Docker secrets via _FILE environment variables
  • Compression — Native dump tool compression support
  • Retention Policies — Automatic cleanup by age or file count
  • Notifications — Slack/Discord webhooks on success or failure
  • Hooks — Pre/post-backup shell command execution
  • Encryption at Rest — Transparent encryption via rclone's crypt remote

Quick Start

dbstash requires only three things to run:

  1. Database connectionDB_URI or DB_HOST + DB_NAME
  2. Rclone remoteRCLONE_REMOTE (where to store backups)
  3. Rclone config — A valid rclone config file
# One-time PostgreSQL backup to S3
docker run --rm \
  -e DB_URI="postgresql://user:pass@host:5432/mydb" \
  -e RCLONE_REMOTE="s3:my-bucket/backups" \
  -e BACKUP_SCHEDULE=once \
  -v /path/to/rclone.conf:/root/.config/rclone/rclone.conf:ro \
  ghcr.io/viperadnan-git/dbstash:pg-17

# Dump all MongoDB databases
docker run --rm \
  -e DB_URI="mongodb+srv://user:[email protected]" \
  -e BACKUP_ALL_DATABASES=true \
  -e RCLONE_REMOTE="s3:my-bucket/backups" \
  -e BACKUP_SCHEDULE=once \
  -v /path/to/rclone.conf:/root/.config/rclone/rclone.conf:ro \
  ghcr.io/viperadnan-git/dbstash:mongo-8

CLI Usage

dbstash is also available as a standalone binary. Pre-built binaries for Linux, macOS, and Windows can be downloaded from Releases. Requires rclone and the engine's dump tool (pg_dump, mongodump, mysqldump, redis-cli) to be installed.

dbstash <engine> [flags]

All flags map 1:1 to environment variables (DB_HOST -> --db-host). Run dbstash <engine> --help for details.

Flags and environment variables can be mixed. Precedence: CLI flag > environment variable > default value.

Available Docker Images

DatabaseEngine KeyLatest Alias TagsVersion-Specific TagsLatest Version
PostgreSQLpg:pg, :pg-latest:pg-15, :pg-16, :pg-1717
MongoDBmongo:mongo, :mongo-latest:mongo-7, :mongo-88
MySQLmysql:mysql, :mysql-latest:mysql-8, :mysql-99
MariaDBmariadb:mariadb, :mariadb-latest:mariadb-10, :mariadb-1111
Redisredis:redis, :redis-latest:redis-7, :redis-88

Tag Strategy:

  • :engine-version (e.g. :pg-17) — Pinned to specific database version
  • :engine and :engine-latest (e.g. :pg, :pg-latest) — Both point to the latest version
  • :engine-version-dbstashversion (e.g. :pg-17-0.7.0) — Created on git tag releases

All images: ghcr.io/viperadnan-git/dbstash:<tag>

How It Works

dbstash streams database dump output directly to rclone, avoiding local disk usage:

dump stdout --> rclone rcat remote:path/filename

No intermediate files are created on disk (in stream mode). The Go binary handles process piping, scheduling, retention cleanup, and notifications.

Docker Compose Example

services:
  backup-pg:
    image: ghcr.io/viperadnan-git/dbstash:pg-16
    environment:
      DB_URI_FILE: /run/secrets/pg_uri
      RCLONE_REMOTE: "s3:my-bucket/backups/pg"
      RCLONE_CONFIG_FILE: /run/secrets/rclone_conf
      BACKUP_SCHEDULE: "0 */6 * * *"
      BACKUP_NAME_TEMPLATE: "{db}-{timestamp}"
      BACKUP_TIMEOUT: "1h"
      RETENTION_MAX_FILES: 20
      RETENTION_MAX_DAYS: 30
      NOTIFY_WEBHOOK_URL: ${DISCORD_WEBHOOK_URL}
      NOTIFY_ON: failure
      LOG_LEVEL: info
      TZ: Asia/Kolkata
    secrets:
      - pg_uri
      - rclone_conf
    restart: unless-stopped

secrets:
  pg_uri:
    file: ./secrets/pg_uri.txt
  rclone_conf:
    file: ./secrets/rclone.conf

Configuration

All options can be set via environment variables or CLI flags. In Docker mode, use environment variables. In CLI mode, use flags (which fall back to environment variables automatically).

Connection
VariableFlagRequiredDefaultDescription
DB_URI--db-uriNo*Full connection URI (e.g. postgresql://user:pass@host:5432/mydb)
DB_URI_FILE--db-uri-fileNo*Path to a file containing the connection URI (Docker secrets)
DB_HOST--db-hostNo*Database host
DB_PORT--db-portNoEngine defaultDatabase port
DB_NAME--db-nameNo*Database name
DB_USER--db-userNoDatabase user
DB_PASSWORD--db-passwordNoDatabase password
DB_PASSWORD_FILE--db-password-fileNoPath to file containing the password (Docker secrets)
DB_AUTH_SOURCE--db-auth-sourceNoadminMongoDB auth database

*Either DB_URI/DB_URI_FILE or DB_HOST + DB_NAME must be provided. When BACKUP_ALL_DATABASES=true, DB_NAME is not required. DB_NAME and BACKUP_ALL_DATABASES are mutually exclusive.

Rclone
VariableFlagRequiredDefaultDescription
RCLONE_REMOTE--rclone-remoteYesRclone remote path (e.g. s3:my-bucket/backups)
RCLONE_CONFIG--rclone-configNoBase64-encoded rclone.conf content
RCLONE_CONFIG_FILE--rclone-config-fileNo~/.config/rclone/rclone.confPath to rclone config file
RCLONE_EXTRA_ARGS--rclone-extra-argsNoAdditional rclone flags
Schedule & Naming
VariableFlagRequiredDefaultDescription
BACKUP_SCHEDULE--backup-scheduleNo0 2 * * *Cron expression or once for a single backup
BACKUP_MODE--backup-modeNostreamstream, directory, tar, or file
BACKUP_NAME_TEMPLATE--backup-name-templateNo{db}-{timestamp}Filename template
BACKUP_COMPRESS--backup-compressNofalseEnable native compression via dump tool
BACKUP_EXTENSION--backup-extensionNoautoOverride file extension
BACKUP_ALL_DATABASES--backup-all-databasesNofalseDump all databases (pg, mysql/mariadb, mongo). Alias: BACKUP_ALL_DBS / --backup-all-dbs
BACKUP_ON_START--backup-on-startNofalseRun backup immediately on start
BACKUP_TIMEOUT--backup-timeoutNo0Max duration for a backup (e.g. 1h, 30m)
BACKUP_LOCK--backup-lockNotruePrevent overlapping backup runs
BACKUP_TEMP_DIR--backup-temp-dirNo/tmp/dbstash-workTemp directory for file/directory/tar modes. Stale dirs from crashes are cleaned on startup.
DUMP_EXTRA_ARGS--dump-extra-argsNoAdditional flags for the dump tool
DRY_RUN--dry-runNofalseLog config without executing
TZ--tzNoUTCTimezone for schedule and filenames
Name Template Tokens

The BACKUP_NAME_TEMPLATE value is expanded at backup time by replacing tokens with runtime values. The file extension is appended automatically based on the engine and compression setting (override with BACKUP_EXTENSION). All timestamps respect the TZ environment variable (default UTC).

TokenExpands ToExample
{db}Database name from DB_NAME or parsed from DB_URI (all when BACKUP_ALL_DATABASES=true)myapp
{engine}Engine keypg
{date}Current date as YYYY-MM-DD2026-02-07
{time}Current time as HHmmss020000
{timestamp}ISO 8601 timestamp with timezone20260207T020000Z
{ts}Unix timestamp in seconds1770508800
{uuid}First 8 characters of a UUIDv7 (time-ordered)019c38fb

Default template: {db}-{timestamp} produces filenames like myapp-20260207T020000Z.sql.

Retention
VariableFlagRequiredDefaultDescription
RETENTION_MAX_FILES--retention-max-filesNo0 (unlimited)Keep at most N backup files
RETENTION_MAX_DAYS--retention-max-daysNo0 (unlimited)Delete backups older than N days
Notifications
VariableFlagRequiredDefaultDescription
NOTIFY_WEBHOOK_URL--notify-webhook-urlNoSlack or Discord webhook URL
NOTIFY_ON--notify-onNofailureWhen to notify: always, failure, success
Hooks
VariableFlagRequiredDefaultDescription
HOOK_PRE_BACKUP--hook-pre-backupNoShell command to run before backup
HOOK_POST_BACKUP--hook-post-backupNoShell command to run after backup

Post-backup hooks receive DBSTASH_STATUS (success/failure) and DBSTASH_FILE (remote path) as environment variables.

Logging
VariableFlagRequiredDefaultDescription
LOG_LEVEL--log-levelNoinfodebug, info, warn, error
LOG_FORMAT--log-formatNotextjson or text

Backup Modes

ModeBACKUP_MODEHow It WorksDisk Usage
Stream (default)streamPipes dump stdout directly to rclone rcatZero
FilefileDumps to a temp file, uploads via rclone copy — same output format as stream but no concurrent uploadRequires temp space
DirectorydirectoryDumps to temp dir, uploads via rclone copyRequires temp space
TartarDumps to temp dir, tar streams to rclone rcatRequires temp space
Compression
EngineBACKUP_COMPRESS=trueManual via DUMP_EXTRA_ARGS
PostgreSQL--Fc (custom format)--compress=zstd:9, etc.
MongoDB--gzip--gzip
MySQL/MariaDBNo-op (warning logged)
RedisNo change (RDB already compact)
All Databases

Set BACKUP_ALL_DATABASES=true to dump every database on the server instead of a single one.

EngineTool UsedLimitations
PostgreSQLpg_dumpallStream mode only (plain SQL, no native compression)
MongoDBmongodump (no --db)None
MySQL/MariaDBmysqldump --all-databasesStream mode only (--tab incompatible)
RedisNo changeAlways dumps the full RDB snapshot

DB_NAME and BACKUP_ALL_DATABASES are mutually exclusive. When using a URI, the database name is automatically stripped for engines that would otherwise scope the dump to a single database.

Encryption at Rest

Use rclone's native crypt remote:

[s3-backup]
type = s3
provider = AWS
access_key_id = ...
secret_access_key = ...

[s3-backup-encrypted]
type = crypt
remote = s3-backup:my-bucket/backups
password = ... (obscured)

Set RCLONE_REMOTE=s3-backup-encrypted: and all backups are transparently encrypted.

Docker Secrets

Any sensitive env var supports a _FILE suffix. dbstash reads the secret from the file at startup:

services:
  backup:
    image: ghcr.io/viperadnan-git/dbstash:pg-16
    environment:
      DB_URI_FILE: /run/secrets/db_uri
      RCLONE_CONFIG_FILE: /run/secrets/rclone_conf
    secrets:
      - db_uri
      - rclone_conf

secrets:
  db_uri:
    file: ./secrets/db_uri.txt
  rclone_conf:
    file: ./secrets/rclone.conf

Supported _FILE variants: DB_URI_FILE, DB_PASSWORD_FILE, RCLONE_CONFIG_FILE.

Health Check

When running with a cron schedule, dbstash serves a health endpoint:

GET :8080/healthz

Returns:

{"status": "healthy", "engine": "pg", "last_backup": "2026-02-07T02:00:05Z", "last_status": "success"}

License

MIT

Tag summary

Content type

Image

Digest

sha256:789799e86

Size

63.6 MB

Last updated

6 months ago

docker pull viperadnan/dbstash:mongo