arkbotech/pgbackup

By arkbotech

•Updated 10 months ago

Image

4.0K

Overview Tags

arkbotech/pgbackup repository overview

⁠ARKBO PostgreSQL Backup

Simple, production-ready PostgreSQL backup with REST API and support for 40+ cloud storage providers.

⁠Quick Start

⁠1. Create docker-compose.yml

services:
  pgbackup:
    image: arkbotech/pgbackup:latest
    container_name: pgbackup
    init: true
    ports:
      - "8080:8080"
    environment:
      # API Configuration
      PGBACKUP_API_PORT: 8080

      # Database Connection (Required)
      PGBACKUP_HOST: postgres
      PGBACKUP_PORT: 5432
      PGBACKUP_USER: postgres
      PGBACKUP_PASSWORD: yourpassword
      PGBACKUP_DATABASE: mydb

      # Backup Settings
      PGBACKUP_FORMAT: custom
      PGBACKUP_COMPRESSION_LEVEL: 6
      PGBACKUP_RETENTION_DAYS: 7

      # Storage
      PGBACKUP_STORAGE: local

      # Optional: Cloud Storage (uncomment to use)
      # PGBACKUP_STORAGE: rclone
      # PGBACKUP_RCLONE_REMOTE: s3:my-bucket/postgres

      # Optional: Cron Schedule for automatic backups
      # PGBACKUP_CRON_SCHEDULE: "0 2 * * *"

      # Notifications (optional)
      PGBACKUP_NOTIFY_ON_SUCCESS: false
      PGBACKUP_NOTIFY_ON_FAILURE: true
      # PGBACKUP_WEBHOOK_URL: https://hooks.slack.com/services/YOUR/WEBHOOK/URL

    volumes:
      - ./backups:/backups
      # Optional: For rclone cloud storage
      # - ./rclone.conf:/config/rclone.conf:ro

    networks:
      - backup_network

    restart: unless-stopped
    stop_grace_period: 30s

    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 1G
        reservations:
          cpus: '0.25'
          memory: 256M

networks:
  backup_network:
    driver: bridge

⁠2. Start the Container

# Start container
docker compose up -d

# Check status
docker ps --filter name=pgbackup

# View startup logs
docker logs pgbackup

⁠3. Use the REST API

# Health check
curl http://localhost:8080/health

# Get system info
curl http://localhost:8080/system

# Trigger a backup
curl -X POST http://localhost:8080/backup

# List backups
curl http://localhost:8080/backups

# Restore database
curl -X POST http://localhost:8080/restore \
  -H "Content-Type: application/json" \
  -d '{"backup_file": "backup_mydb_20250116_120000.dump"}'

⁠How to Use

⁠Using REST API (Recommended)

The container runs a lightweight REST API server on port 8080 for all backup operations.

⁠From Host Machine

# Health check
curl http://localhost:8080/health

# Trigger backup
curl -X POST http://localhost:8080/backup

# List backups
curl http://localhost:8080/backups

# Get system information
curl http://localhost:8080/system

# Restore from backup
curl -X POST http://localhost:8080/restore \
  -H "Content-Type: application/json" \
  -d '{"backup_file": "backup_mydb_20250116_120000.dump"}'

⁠From Inside Container (docker exec)

# Health check
docker exec $(docker ps --filter "name=pgbackup" --format "{{.ID}}" | head -n 1) curl -s http://localhost:8080/health

# Trigger backup
docker exec $(docker ps --filter "name=pgbackup" --format "{{.ID}}" | head -n 1) curl -v -X POST http://localhost:8080/backup

# List backups
docker exec $(docker ps --filter "name=pgbackup" --format "{{.ID}}" | head -n 1) curl -s http://localhost:8080/backups

# Get system info
docker exec $(docker ps --filter "name=pgbackup" --format "{{.ID}}" | head -n 1) curl -s http://localhost:8080/system

# Restore database
docker exec $(docker ps --filter "name=pgbackup" --format "{{.ID}}" | head -n 1) \
  curl -v -X POST http://localhost:8080/restore \
  -H "Content-Type: application/json" \
  -d '{"backup_file": "backup_mydb_20250116_120000.dump"}'

⁠Using Manual Trigger (Alternative)

# Trigger backup manually
docker exec pgbackup backupnow

# View backup logs
docker exec pgbackup tail /var/log/pgbackup.log

⁠Automatic Scheduled Backups

Set the PGBACKUP_CRON_SCHEDULE environment variable:

environment:
  PGBACKUP_CRON_SCHEDULE: "0 2 * * *"  # Daily at 2 AM

Cron Schedule Examples:

"0 2 * * *" - Daily at 2 AM
"0 */6 * * *" - Every 6 hours
"0 3 * * 0" - Weekly on Sunday at 3 AM
"0 0 1 * *" - Monthly on the 1st at midnight

Or use host cron (if you prefer external scheduling):

# Edit crontab
crontab -e

# Add daily backup at 2 AM
0 2 * * * curl -X POST http://localhost:8080/backup >> /var/log/pgbackup-cron.log 2>&1

⁠View Logs

# Container logs
docker logs -f pgbackup

# Backup script logs
docker exec pgbackup tail -f /var/log/pgbackup.log

# List backup files
ls -lh ./backups/

⁠REST API Endpoints

Method	Endpoint	Description	Example
`GET`	`/health`	Health check + scheduler status	`curl http://localhost:8080/health`
`GET`	`/system`	System info (PostgreSQL version, config)	`curl http://localhost:8080/system`
`GET`	`/backups`	List all backup files	`curl http://localhost:8080/backups`
`POST`	`/backup`	Trigger database backup	`curl -X POST http://localhost:8080/backup`
`POST`	`/restore`	Restore database from backup	`curl -X POST http://localhost:8080/restore -H "Content-Type: application/json" -d '{"backup_file": "mybackup.dump"}'`

⁠API Response Examples

Health Check:

$ curl http://localhost:8080/health
{
  "status": "healthy",
  "service": "pgbackup-api",
  "timestamp": "2025-11-16T10:02:43.825574",
  "scheduler": {
    "enabled": false,
    "schedule": ""
  }
}

System Information:

$ curl http://localhost:8080/system
{
  "status": "success",
  "system": {
    "hostname": "pgbackup-container",
    "postgresql_client": "psql (PostgreSQL) 17.7",
    "rclone": "rclone v1.71.2"
  },
  "configuration": {
    "host": "postgres",
    "port": "5432",
    "database": "mydb",
    "format": "custom",
    "storage": "local",
    "retention_days": "7"
  }
}

List Backups:

$ curl http://localhost:8080/backups
{
  "status": "success",
  "count": 3,
  "backups": [
    {
      "filename": "backup_mydb_20250116_140000.dump",
      "size": 1048576,
      "size_mb": 1.0,
      "modified": "2025-01-16T14:00:00",
      "path": "/backups/backup_mydb_20250116_140000.dump"
    }
  ]
}

Trigger Backup:

$ curl -X POST http://localhost:8080/backup
{
  "status": "success",
  "message": "Backup started"
}

⁠Environment Variables Reference

⁠API Configuration

Variable	Description	Default	Example
`PGBACKUP_API_PORT`	REST API port	`8080`	`8080`, `9000`
`PGBACKUP_CRON_SCHEDULE`	Optional cron schedule for auto backups	(empty)	`"0 2 * * *"` (daily 2AM)

⁠Database Connection (Required)

Variable	Description	Default	Example
`PGBACKUP_HOST`	Database hostname/IP	`postgres`	`postgres`, `192.168.1.10`
`PGBACKUP_PORT`	Database port	`5432`	`5432`
`PGBACKUP_USER`	Database user	`postgres`	`postgres`, `backup_user`
`PGBACKUP_PASSWORD`	Database password	(required)	`mysecretpass`
`PGBACKUP_DATABASE`	Database name	`postgres`	`mydb`, `production`

⁠Backup Configuration

Variable	Description	Default	Options
`PGBACKUP_FORMAT`	Backup format	`custom`	`custom`, `directory`, `plain`, `tar`
`PGBACKUP_COMPRESSION_LEVEL`	Compression level (0-9)	`6`	`0` (none) to `9` (max)
`PGBACKUP_RETENTION_DAYS`	Keep backups for X days	`7`	`7`, `30`, `90`, `0` (forever)
`PGBACKUP_COUNT`	Number of backups to create	`1`	Any positive integer
`PGBACKUP_INTERVAL`	Seconds between backups	`0`	`0`, `3600`, `86400`

Backup Format Details:

custom → .dump (Binary compressed, smallest, recommended)
directory → _dir/ (Parallel dump, fastest for large DBs)
plain → .sql (Plain SQL text, human-readable)
tar → .sql.tar.gz (Compressed tar archive)

⁠Storage Configuration

Variable	Description	Default	Options
`PGBACKUP_STORAGE`	Storage backend	`local`	`local`, `rclone`

⁠Rclone Cloud Storage (when PGBACKUP_STORAGE=rclone)

Variable	Description	Default	Example
`PGBACKUP_RCLONE_REMOTE`	Rclone remote destination	(required)	`s3:my-bucket/postgres`
`PGBACKUP_RCLONE_CONFIG`	Path to rclone config	`/config/rclone.conf`	`/config/rclone.conf`
`PGBACKUP_RCLONE_TRANSFERS`	Parallel transfers	`4`	`4`, `8`, `16`
`PGBACKUP_RCLONE_CHECKERS`	Parallel checkers	`8`	`8`, `16`, `32`
`PGBACKUP_RCLONE_BWLIMIT`	Bandwidth limit	(unlimited)	`10M`, `100M`, `1G`

⁠Notifications

Variable	Description	Default	Example
`PGBACKUP_NOTIFY_ON_SUCCESS`	Notify on success	`false`	`true`, `false`
`PGBACKUP_NOTIFY_ON_FAILURE`	Notify on failure	`true`	`true`, `false`
`PGBACKUP_WEBHOOK_URL`	Webhook URL	(optional)	Slack/Discord/Teams URL

⁠Debug

Variable	Description	Default	Options
`ARKBO_DEBUG`	Enable debug logging	`false`	`true`, `false`

⁠Docker Secrets Support

All variables support _FILE suffix for Docker secrets:

environment:
  PGBACKUP_PASSWORD_FILE: /run/secrets/db_password
  PGBACKUP_RCLONE_REMOTE_FILE: /run/secrets/rclone_remote

secrets:
  - db_password
  - rclone_remote

⁠Complete Docker Compose Examples

⁠Example 1: Basic Local Backup

services:
  pgbackup:
    image: arkbotech/pgbackup:latest
    container_name: pgbackup
    init: true
    ports:
      - "8080:8080"
    environment:
      PGBACKUP_HOST: postgres
      PGBACKUP_USER: postgres
      PGBACKUP_PASSWORD: mypassword
      PGBACKUP_DATABASE: mydb
      PGBACKUP_API_PORT: 8080
    volumes:
      - ./backups:/backups
    restart: unless-stopped

⁠Example 2: Scheduled Backups with S3 Storage

services:
  pgbackup:
    image: arkbotech/pgbackup:latest
    container_name: pgbackup
    init: true
    ports:
      - "8080:8080"
    environment:
      # API
      PGBACKUP_API_PORT: 8080
      PGBACKUP_CRON_SCHEDULE: "0 2 * * *"  # Daily at 2 AM

      # Database
      PGBACKUP_HOST: production-db
      PGBACKUP_PORT: 5432
      PGBACKUP_USER: backup_user
      PGBACKUP_PASSWORD: securepassword
      PGBACKUP_DATABASE: production

      # Backup settings
      PGBACKUP_FORMAT: custom
      PGBACKUP_COMPRESSION_LEVEL: 9
      PGBACKUP_RETENTION_DAYS: 30

      # Cloud storage
      PGBACKUP_STORAGE: rclone
      PGBACKUP_RCLONE_REMOTE: s3:company-backups/postgres
      PGBACKUP_RCLONE_BWLIMIT: 50M

      # Notifications
      PGBACKUP_NOTIFY_ON_FAILURE: true
      PGBACKUP_WEBHOOK_URL: https://hooks.slack.com/services/YOUR/WEBHOOK/URL

    volumes:
      - ./backups:/backups
      - ./rclone.conf:/config/rclone.conf:ro

    restart: unless-stopped

⁠Example 3: Using Docker Secrets (Production)

services:
  pgbackup:
    image: arkbotech/pgbackup:latest
    container_name: pgbackup
    init: true
    ports:
      - "8080:8080"
    environment:
      PGBACKUP_HOST: postgres
      PGBACKUP_USER: postgres
      PGBACKUP_PASSWORD_FILE: /run/secrets/db_password
      PGBACKUP_DATABASE: mydb
      PGBACKUP_API_PORT: 8080
      PGBACKUP_CRON_SCHEDULE: "0 3 * * *"

    secrets:
      - db_password

    volumes:
      - ./backups:/backups

    restart: unless-stopped

secrets:
  db_password:
    file: ./secrets/db_password.txt

⁠Cloud Storage Setup

⁠Amazon S3

1. Configure rclone:

docker run --rm -it -v $(pwd)/rclone.conf:/config/rclone.conf rclone/rclone config

Configure S3:

name> s3
type> s3
provider> AWS
access_key_id> YOUR_ACCESS_KEY
secret_access_key> YOUR_SECRET_KEY
region> us-east-1

2. Update docker-compose.yml:

environment:
  PGBACKUP_STORAGE: rclone
  PGBACKUP_RCLONE_REMOTE: s3:my-bucket-name/postgres-backups

volumes:
  - ./rclone.conf:/config/rclone.conf:ro

⁠Google Cloud Storage

rclone.conf:

[gcs]
type = google cloud storage
project_number = 123456789
service_account_file = /config/gcs-key.json

docker-compose.yml:

environment:
  PGBACKUP_STORAGE: rclone
  PGBACKUP_RCLONE_REMOTE: gcs:my-bucket/backups

volumes:
  - ./rclone.conf:/config/rclone.conf:ro
  - ./gcs-key.json:/config/gcs-key.json:ro

⁠Other Supported Providers

AWS S3
Google Cloud Storage
Azure Blob Storage
Backblaze B2
Wasabi
DigitalOcean Spaces
MinIO
Dropbox
Google Drive
Microsoft OneDrive
SFTP/FTP
WebDAV
And 30+ more!

See rclone.org⁠ for complete list.

⁠Docker Swarm Deployment

⁠compose-swarm.yml

version: '3.8'

services:
  pgbackup:
    image: arkbotech/pgbackup:latest
    init: true

    environment:
      PGBACKUP_API_PORT: 8080
      PGBACKUP_HOST: postgres
      PGBACKUP_PORT: 5432
      PGBACKUP_USER: postgres
      PGBACKUP_PASSWORD: yourpassword
      PGBACKUP_DATABASE: mydb
      PGBACKUP_FORMAT: custom
      PGBACKUP_COMPRESSION_LEVEL: 6
      PGBACKUP_RETENTION_DAYS: 7
      PGBACKUP_STORAGE: local

    volumes:
      - /opt/backups/postgres:/backups

    networks:
      - backup_network

    ports:
      - target: 8080
        published: 8080
        protocol: tcp
        mode: host

    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == manager
      restart_policy:
        condition: unless-stopped
      resources:
        limits:
          cpus: '1'
          memory: 1G
        reservations:
          cpus: '0.25'
          memory: 256M

networks:
  backup_network:
    driver: overlay

⁠Deploy to Swarm

# Create backup directory on manager node
sudo mkdir -p /opt/backups/postgres
sudo chown 1001:1001 /opt/backups/postgres

# Deploy stack
docker stack deploy -c compose-swarm.yml pgbackup

# Check status
docker stack ls
docker service ls | grep pgbackup
docker service logs -f pgbackup_pgbackup

# Trigger backup via API (from manager or worker node)
curl -X POST http://localhost:8080/backup

# View backups
curl http://localhost:8080/backups

⁠Troubleshooting

⁠Container Won't Start

# Check logs
docker logs pgbackup

# Check if port 8080 is already in use
sudo netstat -tulpn | grep 8080

# Restart container
docker restart pgbackup

⁠Cannot Connect to Database

# Test database connection
docker exec pgbackup psql -h $PGBACKUP_HOST -U $PGBACKUP_USER -d $PGBACKUP_DATABASE -c "SELECT version();"

# Check environment variables
docker exec pgbackup env | grep PGBACKUP

# View connection logs
docker logs pgbackup | grep -i "connection\|error"

⁠API Not Responding

# Check if API server is running
docker exec pgbackup ps aux | grep python

# Check API port
docker exec pgbackup netstat -tulpn | grep 8080

# Test from inside container
docker exec pgbackup curl -s http://localhost:8080/health

⁠Backup Failed

# View detailed logs
docker logs pgbackup

# Check backup script logs
docker exec pgbackup cat /var/log/pgbackup.log

# Check disk space
df -h ./backups/

# Test backup manually
docker exec pgbackup backupnow

⁠Permission Issues

# Fix backup directory permissions
chmod 777 ./backups/

# Or set correct ownership
sudo chown -R 1001:1001 ./backups/

⁠Rclone Not Working

# Verify rclone config
docker exec pgbackup rclone config show

# List remotes
docker exec pgbackup rclone listremotes

# Test rclone connection
docker exec pgbackup rclone ls s3:my-bucket/

⁠Features

✅ REST API Interface - Full HTTP API for all operations ✅ Optional Scheduler - Built-in Python-based cron scheduler ✅ Cloud Storage - Support for 40+ cloud providers via rclone ✅ Multiple Formats - Custom (compressed), directory, plain SQL, tar ✅ Smart Retention - Automatic cleanup of old backups ✅ Webhooks - Slack/Discord/Teams notifications ✅ Docker Secrets - Secure credential management ✅ Health Checks - Built-in Docker health monitoring ✅ Non-Root - Runs as UID 1001 for security ✅ Multi-Platform - Works on amd64 and arm64 ✅ Production Ready - Battle-tested with comprehensive logging

⁠Support & License

Tag summary

Recent tags

Content type

Image

Digest

sha256:e217b8512…

Size

87.1 MB

Last updated

10 months ago

docker pull arkbotech/pgbackup