stefanmarkmann/postgres-backup-s3cmd repository overview

⁠postgres-backup-s3cmd

Simple PostgreSQL dump backups to S3-compatible storage using s3cmd.

⁠Overview

This project provides Docker images to periodically back up PostgreSQL databases to S3-compatible object storage, and restore from backups as needed.

⁠Features

• S3-compatible uploads via s3cmd
• Encryption (GPG symmetric)
• Full dumps (pg_dump or pg_dumpall)
• Compression (zstd)
• Scheduled or on-demand runs
• Retention cleanup by age
• Restore from backups

⁠Backup Flow

flowchart TB
  db[(PostgreSQL DB)]

  subgraph container["Backup Container"]
    pg_dump[pg_dump / pg_dumpall]
    zip[zstd compression]
    encrypt[gpg encryption]
    upload[s3cmd upload]
    pg_dump --> zip --> encrypt --> upload
  end

  s3[(S3-Compatible Storage)]

  db --> pg_dump
  upload --> s3

Why s3cmd?

Unlike AWS CLI, s3cmd works reliably with S3-compatible storage providers that don't implement all AWS-specific features (like checksum response headers). This makes it ideal for:

• MinIO
• Ceph RGW
• Wasabi
• DigitalOcean Spaces
• Backblaze B2
• Any S3-compatible storage

Compression + integrity

Backups are compressed with zstd by default. zstd frame checksums are enabled so corruption is detected automatically during restore (decompression fails).

⁠Non-Goals

This tool is intentionally simple. It is not:

• ❌ A Point-in-Time Recovery (PITR) solution — no WAL archiving
• ❌ An incremental backup system — full dumps only
• ❌ A pgBackRest replacement — use pgBackRest for enterprise needs
• ❌ Multi-cluster aware — one database/cluster per container

If you need these features, consider pgBackRest⁠ or Barman⁠.

⁠Quick Start

services:
  postgres:
    image: postgres:18
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: mydb

  backup:
    image: stefanmarkmann/postgres-backup-s3cmd:18
    environment:
      SCHEDULE: '@daily'
      BACKUP_KEEP_DAYS: 7
      S3_ACCESS_KEY_ID: your-access-key
      S3_SECRET_ACCESS_KEY: your-secret-key
      S3_BUCKET: my-backup-bucket
      S3_ENDPOINT: https://s3.example.com  # Optional: for non-AWS S3
      POSTGRES_HOST: postgres
      POSTGRES_DATABASE: mydb
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword

⁠Image Tags

Images are tagged by PostgreSQL major version with semantic versioning:

⁠Environment Variables

⁠Required

⁠Optional

⁠Usage

⁠Scheduled Backups

Set SCHEDULE to enable periodic backups:

environment:
  SCHEDULE: '@daily'       # Daily at midnight
  # SCHEDULE: '@hourly'    # Every hour
  # SCHEDULE: '0 2 * * *'  # Daily at 2 AM

Use standard cron syntax (including @daily, @hourly) for SCHEDULE.

⁠Manual Backup

docker exec <container> sh backup.sh

If you open an interactive shell in the container, backup (along with restore, list, cleanup, delete, run) is available as a convenient alias for the corresponding /backup.sh script.

⁠List Backups

docker exec <container> sh list.sh            # Table format
docker exec <container> sh list.sh --summary  # Count, size, latest
docker exec <container> sh list.sh --latest   # Latest backup only

⁠Delete Specific Backup

docker exec <container> sh delete.sh 2026-01-05T14:30:00 --dry-run  # Preview
docker exec <container> sh delete.sh 2026-01-05T14:30:00            # Delete

⁠Manual Cleanup

docker exec <container> sh cleanup.sh           # Use BACKUP_KEEP_DAYS
docker exec <container> sh cleanup.sh 7         # Override to 7 days
docker exec <container> sh cleanup.sh --dry-run # Preview deletions

⁠Backup All Databases

Omit POSTGRES_DATABASE to use pg_dumpall:

environment:
  POSTGRES_HOST: postgres
  POSTGRES_USER: postgres
  POSTGRES_PASSWORD: secret
  # POSTGRES_DATABASE not set - backs up all databases

⁠Encrypted Backups

Set PASSPHRASE to encrypt backups with GPG:

environment:
  PASSPHRASE: my-secret-passphrase

⁠Restore

⁠Restore Latest Backup

docker exec <container> sh restore.sh

Running without arguments now lists available backups with sizes and prompts for a selection (press Enter for the latest). The script then requires two confirmations: typing YES, followed by either the timestamp or the filename shown in the list. In non-interactive environments, it automatically selects the latest backup and skips confirmations.

⁠Restore Specific Backup

docker exec <container> sh restore.sh 2026-01-07T14:30:00

Restoring a specific timestamp still prompts for the same two confirmations before download and restore. In non-interactive environments, it skips the confirmations.

If backups are compressed (.zst), restore automatically decompresses with zstd -d, which verifies the embedded checksum and fails fast on corruption.

⁠S3 Provider Examples

⁠MinIO

environment:
  S3_ENDPOINT: http://minio:9000
  S3_ACCESS_KEY_ID: minioadmin
  S3_SECRET_ACCESS_KEY: minioadmin
  S3_BUCKET: backups

⁠Wasabi

environment:
  S3_ENDPOINT: https://s3.wasabisys.com
  S3_REGION: us-east-1
  S3_ACCESS_KEY_ID: your-access-key
  S3_SECRET_ACCESS_KEY: your-secret-key
  S3_BUCKET: my-bucket

⁠AWS S3

environment:
  S3_REGION: eu-central-1
  S3_ACCESS_KEY_ID: AKIA...
  S3_SECRET_ACCESS_KEY: ...
  S3_BUCKET: my-bucket
  # S3_ENDPOINT not needed for AWS

⁠Security Considerations

• Credentials are written to /root/.s3cfg inside the container
• Container assumes a trusted runtime environment

Recommendations:

• Use Docker secrets or Kubernetes secrets for credentials
• Prefer mounting a pre-created s3cmd config as a secret (S3CFG_USE_EXISTING=true, S3CFG_PATH=/run/secrets/s3cfg) to avoid generating credential files at runtime
• Run with read-only root filesystem where possible
• Limit container capabilities

⁠Kubernetes

See examples/kubernetes/⁠ for reference manifests.

⁠Development

⁠Build Locally

docker build \
  --build-arg ALPINE_VERSION=3.23 \
  --build-arg PG_MAJOR=18 \
  -t postgres-backup-s3cmd:18 .

⁠Test with Docker Compose

cp template.env .env
# Edit .env with your settings
docker compose up -d

⁠Migration from eeshugerman/postgres-backup-s3

This project is a reimplementation inspired by eeshugerman/postgres-backup-s3⁠.

⁠Breaking Changes

⁠Compatible

• All PostgreSQL connection variables
• SCHEDULE, PASSPHRASE, BACKUP_KEEP_DAYS
• Backup file naming (<db>_<timestamp>.dump[.zst][.gpg])
• Encrypted backups (GPG symmetric)

⁠Acknowledgements

This project is inspired by:

• eeshugerman/postgres-backup-s3⁠ (archived)
• schickling/postgres-backup-s3⁠ (original)

⁠License

MIT License - see LICENSE⁠