djas19/gcp-sql-exporter repository overview

⁠GCP Cloud SQL Exporter

Automated Cloud SQL database backup exporter for Google Cloud Platform. This application automatically discovers and exports Cloud SQL instances to Google Cloud Storage buckets based on instance labels and schedules.

⁠Quick Start

⁠1. Add the Terraform Module to Your Infrastructure

Add this module to your existing Terraform configuration:

module "sql_exporter" {
  source = "github.com/yourusername/gcp-sql-exporter//infra"

  project_id         = "your-gcp-project-id"
  region             = "us-central1"
  backup_bucket_name = "your-backup-bucket-name"
  docker_image       = "yourdockerhub/gcp-sql-exporter:latest"
}

Or use a local path if you've cloned the repository:

module "sql_exporter" {
  source = "./path/to/gcp-sql-exporter/infra"

  project_id         = "your-gcp-project-id"
  region             = "us-central1"
  backup_bucket_name = "your-backup-bucket-name"
  docker_image       = "yourdockerhub/gcp-sql-exporter:latest"
}

⁠2. Label Your Cloud SQL Instances

Add labels to the instances you want to back up:

# Enable daily backups for an instance
gcloud sql instances patch my-production-db \
  --update-labels=autosql-backup=true,autosql-schedule=daily

# Enable hourly backups for a critical database
gcloud sql instances patch my-critical-db \
  --update-labels=autosql-backup=true,autosql-schedule=hourly

Required Labels:

⁠3. How It Works

Once deployed, the system automatically:

1. Cloud Scheduler triggers the exporter on a schedule (default: hourly)
2. Exporter discovers instances by listing all Cloud SQL instances in your project
3. Filters by labels - Only processes instances with autosql-backup=true
4. Checks if backup is needed - Looks at recent operations to avoid duplicates:
   • For hourly: Skips if a backup completed in the last 55 minutes
   • For daily: Skips if a backup completed in the last 23 hours
5. Triggers the export - Calls Cloud SQL Admin API to export to GCS
6. Organizes backups - Stores in bucket with structure: gs://bucket/instance-name/YYYY-MM-DD-HHMMSS-instance-name.sql.gz

Example Workflow:

Cloud Scheduler (every hour)
    ↓
Cloud Run Service
    ↓
Checks all instances with autosql-backup=true
    ↓
Filters by schedule (hourly vs daily)
    ↓
Checks for recent/running backups
    ↓
Triggers export to GCS bucket