spinkever/backup-manager repository overview

⁠Backup Manager

A web-based backup management solution for Docker volumes, flat files, and remote rsync synchronization.

⁠AI disclaimer

⁠Description

Backup Manager is a Dockerized application that provides a user-friendly web interface to manage backup jobs. It supports:

• Docker Volume Backups: Creates consistent backups of Docker container data by stopping containers through the compose file, creating tar.gz archives, and restarting containers
• Flat File Backups: Backs up regular directories and files
• Remote Rsync Sync: Synchronizes backups to remote servers over SSH with automatic cleanup of old remote backups
• Scheduled Jobs: Cron-based scheduling for automated backups
• Pushover Notifications: Receive alerts on backup success/failure
• Automatic Pruning: Configurable retention policy to delete old backups

Backup Manager works best by having a common name for your backup job, your docker stack location and the stack data volume mount. For example: if you set the "Job name" to nextcloud, it will automatically use /opt/stacks/nextcloud for the stack location, and /opt/nextcloud for the nextcloud volume. You can manually specify a different location per backup job if required.

⁠Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Backup Manager Container                     │
│                                                                  │
│  ┌─────────────────────┐      ┌─────────────────────────────┐  │
│  │   Web UI (Flask)    │      │      Scheduler (APScheduler)│  │
│  │   Port 5001         │◄────►│      Port 5002              │  │
│  │   (Gunicorn)        │ API  │      (Gunicorn)             │  │
│  └─────────┬───────────┘      └─────────────┬───────────────┘  │
│            │                                │                   │
│            │         ┌──────────────────────┘                   │
│            │         │                                          │
│            ▼         ▼                                          │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │                    SQLite Database                        │  │
│  │  - jobs (backup job configs)                              │  │
│  │  - logs (backup execution logs)                           │  │
│  │  - rsync_jobs (rsync job configs)                         │  │
│  │  - rsync_logs (rsync execution logs)                      │  │
│  │  - pushover_config (notification settings)                │  │
│  └──────────────────────────────────────────────────────────┘  │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │              Backup Execution Engine                      │  │
│  │  - tar (for local archives)                               │  │
│  │  - rsync (for remote sync)                                │  │
│  │  - docker compose (for container management)              │  │
│  └──────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              │ Volumes
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                        Host System                              │
│  /var/run/docker.sock  - Docker socket access                   │
│  /opt                  - Docker stacks/data directories         │
│  /mnt/storage          - Backup destination storage             │
│  /config               - Database & logs persistence            │
└─────────────────────────────────────────────────────────────────┘

⁠Components

1. Web Application (app.py)

   • Flask-based REST API and web interface
   • Manages job configurations via SQLite database
   • Handles backup orchestration (stop containers → archive → restart)
   • Provides Pushover notification integration

2. Scheduler (scheduler.py)

   • APScheduler-based job scheduler
   • Executes backup jobs based on cron expressions
   • REST API for manual job triggering
   • API-key protected endpoints

3. Process Manager (supervisord.conf)

   • Runs both web app and scheduler via Gunicorn
   • Handles process monitoring and restarts

4. Frontend

   • Tailwind CSS-based responsive UI
   • Dark/light mode support
   • Real-time log viewing
   • Job management (CRUD operations)

⁠Features

⁠Local Backup Jobs

• Docker compose stack backups with automatic container lifecycle management
• Option to skip restarting the Docker stack after backup (per-job setting)
• Folder-only backups for flat file structures
• Configurable exclusion patterns (tar-compatible)
• Customizable retention policies
• Cron-based scheduling or manual execution
• Sort jobs by name or scheduled time

⁠Remote Rsync Jobs

• SSH-based rsync synchronization
• SSH key authentication
• Exclusion patterns for rsync
• Automatic remote cleanup based on age
• Safety checks to prevent accidental deletion of critical directories

⁠Notifications

• Pushover integration for real-time alerts
• Enable/disable toggle for Pushover notifications
• Configurable notifications for success/failure
• Priority levels for different alert types (-2 to 2, see Pushover API docs⁠)

⁠Logging & Monitoring

• Detailed execution logs with timestamps
• Backup file size tracking
• Status indicators (SUCCESS, PARTIAL_SUCCESS, FAILURE)
• Real-time job status with animated spinner and completion badge after manual trigger
• Manual deletion of backup files and/or log entries from the web UI
• Health check endpoint for container monitoring

⁠Usage

⁠Quick Start with Docker Compose

1. Clone or copy the project files to your server

2. Configure the compose.yml file:

   services:
     backup-manager:
       image: spinkever/backup-manager:latest
       container_name: backup-manager
       restart: unless-stopped
       ports:
         - "5001:5001"
       volumes:
         # Mount the docker socket for compose stop/start. Optionally, include a docker socket proxy container.
         - /var/run/docker.sock:/var/run/docker.sock
         - /opt:/opt                               # Source directory
         - /mnt/storage:/mnt/storage               # Destination directory
         - /opt/backup-manager/config:/config      # Config directory
         - /opt/backup-manager/ssh:/root/.ssh      # Optional -- .ssh for remote rsync operations
       environment:
         - TZ=Europe/Amsterdam
         - STACKS_DIR=/opt/stacks                  # Stacks directory, prefills the compose path in the web. Defaults to /opt/stacks if not set
         - SOURCE_DATA_DIR=/opt                    # Source data directory, prefills the source path in the web UI
         - BACKUP_DEST_DIR=/mnt/storage/backup     # Backup destination directory, prefills the backup path in the web UI
         - SCHEDULER_API_KEY=super-secret-api-key  # Optional -- scheduler API key. Generated automatically if unset.
   
         # Optional settings
         # Auth: set a password to enable local login
         # - LOGIN_PASSWORD=my-secret-password
         # Set to false to hide the local password form when only OIDC is used
         # - LOCAL_LOGIN_ENABLED=true
   
         # OIDC: set these three to enable OIDC login
         # - OIDC_CLIENT_ID=my-client-id
         # - OIDC_CLIENT_SECRET=my-client-secret
         # - OIDC_PROVIDER_URL=https://my-oidc-provider
         # Optional: custom button text for OIDC (default "Sign in with OIDC")
         # - OIDC_APP_NAME=MyApp
         # Required for OIDC when behind a reverse proxy: set to the external URL
         # - EXTERNAL_URL=https://backups.example.com
         # Optional: set a persistent session secret (auto-generated if empty)
         # - SESSION_SECRET=my-session-secret
   

   Copy

3. Build and start the container:

   docker-compose up -d
   

   Copy

4. Access the web interface at http://your-server:5001

⁠Creating a Backup Job

1. Navigate to Local Jobs tab
2. Fill in the job configuration:
   • Job name: Unique identifier (no / or ?)
   • Job type: Docker stack or Folder only
   • Schedule: Cron expression (e.g., 0 2 * * * for 2 AM daily)
   • Max backups: Number of backups to retain (0 for unlimited)
   • Exclusions: Comma-separated patterns to exclude
   • Auto-start stack (Docker jobs only): Enable to restart the stack after backup, disable to leave it stopped
3. Click Add job

Jobs can be sorted by name or scheduled time using the sort buttons in the job list header.

⁠Creating an Rsync Job

1. Navigate to Rsync Jobs tab
2. Fill in the configuration:
   • Job name: Unique identifier
   • Local source path: Directory to sync from
   • Remote user/host: SSH destination
   • Remote path: Destination directory
   • SSH key path: Path to private key (default: /root/.ssh/id_rsa)
   • Max age: Days to keep remote backups (0 to disable cleanup)
3. Click Add rsync job

⁠Configuring Pushover Notifications

1. Sign up at pushover.net⁠
2. Create an application to get your API token
3. In Backup Manager, expand Pushover configuration
4. Enter your API token and user key
5. Set notification preferences and priority
6. Click Save Settings

⁠Authentication

Backup Manager supports two authentication methods: local password login and OIDC (OpenID Connect). You can use either one or both at the same time. Authentication is disabled if local password login and OIDC are not configured making the app accessible without login.

⁠Local Password Login

Set the LOGIN_PASSWORD environment variable to enable password-based login. When only this is set, the login page shows a simple password field.

Optionally, set LOCAL_LOGIN_ENABLED to false to hide the local login field if you have OIDC login enabled.

⁠OIDC Login

To enable OIDC, set these environment variables:

• OIDC_CLIENT_ID — your OIDC client ID
• OIDC_CLIENT_SECRET — your OIDC client secret
• OIDC_PROVIDER_URL — the OIDC issuer URL (the issuer field from your provider's /.well-known/openid-configuration, e.g. https://auth.example.com or https://auth.example.com/oidc)

Configure the redirect/callback URI in your OIDC provider as:

https://<your-domain>/oidc/callback

Replace https://<your-domain> with the actual external URL of your Backup Manager instance.

If the app is behind a reverse proxy, also set EXTERNAL_URL to the same base URL (e.g. https://backups.example.com). This ensures the callback URL is consistent and matches what's registered with your provider.

Optionally set OIDC_APP_NAME to customize the button text on the login page (defaults to OIDC).

⁠What You'll See

• Only LOGIN_PASSWORD set — a password field and a "Sign in" button
• Only OIDC configured — a single button to sign in via your OIDC provider (the password field is hidden)
• Both configured — both the password field and the OIDC sign-in button are shown. You can use either method.

If neither is configured, authentication is disabled and the app is accessible without login.

⁠Environment Variables

⁠Cron Expression Format

Standard cron format: minute hour day month weekday

Examples:

• 0 2 * * * - Daily at 2:00 AM
• 0 0 * * 0 - Weekly on Sunday at midnight
• 0 */6 * * * - Every 6 hours
• 30 18 * * 1-5 - Weekdays at 6:30 PM

⁠Security Considerations

• SSH Keys: Mount SSH keys into the container for rsync operations
• API Keys: The scheduler API key should be kept secure
• File Permissions: Backup files are created with standard permissions
• Network: Expose only necessary ports; consider using a reverse proxy

⁠Troubleshooting

⁠Common Issues

1. Docker socket access denied: Ensure /var/run/docker.sock is mounted, or proper permissions are passed through to a docker socket proxy
2. Permission denied on paths: Verify volume mounts and container permissions
3. SSH connection failed: Check SSH key is mounted and has correct permissions (600)
4. Cron schedule not working: Validate cron expression format

⁠Logs

View application logs:

docker logs backup-manager

View detailed job logs in the web UI under Logs for each job.

⁠API Endpoints

⁠Backup Jobs

• GET / - List all backup jobs
• GET /job/<id> - Get job details as JSON
• POST /add_job - Create new backup job
• POST /edit_job/<id> - Update backup job
• POST /delete_job/<id> - Delete backup job
• POST /run_now/<id> - Trigger immediate backup
• GET /logs/<id> - Get job execution logs
• GET /latest_log/<id>?since=<log_id> - Poll for latest log after job trigger
• POST /delete_backup/<id> - Delete backup file and its log entry
• POST /delete_log/<id> - Delete a log entry without deleting the backup

⁠Rsync Jobs

• GET /rsync - List all rsync jobs
• GET /rsync_job/<id> - Get rsync job details as JSON
• POST /add_rsync_job - Create new rsync job
• POST /edit_rsync_job/<id> - Update rsync job
• POST /delete_rsync_job/<id> - Delete rsync job
• POST /run_rsync_now/<id> - Trigger immediate sync
• GET /rsync_logs/<id> - Get rsync execution logs
• POST /delete_rsync_log/<id> - Delete an rsync log entry

⁠Auth

• GET /login - Login page (GET shows form, POST validates password)
• GET /logout - Log out and clear session
• GET /oidc/authorize - Redirect to OIDC provider for authorization
• GET /oidc/callback - OIDC callback handler (receives auth code)

⁠System

• GET /healthz - Health check endpoint
• GET /version - Get application version
• POST /pushover_config - Update Pushover settings

⁠Development

⁠Requirements

• Python 3.13+
• Docker (for containerized deployment)
• Docker Compose

⁠Dependencies

See requirements.txt:

• Flask
• APScheduler
• Gunicorn
• cron-validator
• pytz
• requests

⁠License

This project is available through the MIT license. See the LICENSE file for the full license details.

⁠Support

For issues, questions, or contributions, please refer to the project repository.