Sign inSign up

diogopms/postgres-gcs-backup

By diogopms

•Updated about 2 years ago

PostgreSQL GCP (Google cloud) Kubernetes backup

Image
Databases & storage
1

10K+

diogopms/postgres-gcs-backup repository overview

⁠postgres-gcs-backup

GitHub last commit (branch) GitHub issues GitHub license GitHub stars

Docker Pulls Hacktoberfest

This project aims to provide a simple way to perform a PostgreSQL server/db backup using pg_dump and to upload it to Google Cloud Storage. It also provides the ability to send Slack alerts in case of failure.

We provide a kubernetes support thanks to the helm chart located in the chart folder of this repository.

⁠Docker image

You can pull the public image from Docker Hub:

docker pull diogopms/postgres-gcs-backup:latest

Or pull the public image from Github Registry:

 docker pull ghcr.io/diogopms/postgres-gcs-backup:latest
⁠Configuration

The following table lists the configurable parameters you can set up.

Environment VariableRequiredDefaultDescription
JOB_NAMENodefault-nameJob name
BACKUP_DIRNo/tmpThe path where the pg_dump result will be temporarily stored.
BOTO_CONFIG_PATHNo/root/.botoThe path where gsutil will search for the boto configuration file.
GCS_BUCKETYesThe bucket you want to upload the backup archive to.
GCS_KEY_FILE_PATHYesThe location where the GCS serviceaccount key file will be mounted.
POSTGRES_HOSTNolocalhostThe PostgreSQL server host.
POSTGRES_PORTNo5432The PostgreSQL port.
POSTGRES_DBNoThe database to backup. By default, a backup of all the databases will be performed.
POSTGRES_USERNoThe PostgreSQL user if any.
POSTGRES_PASSWORDNoThe PostgreSQL password if any.
SLACK_AUTHOR_NAMENopostgres-gcs-backuptrue slack author name.
SLACK_ALERTSNotrue if you want to send Slack alerts in case of failure.
SLACK_WEBHOOK_URLNoThe Incoming WebHook URL to use to send the alerts.
SLACK_CHANNELNoThe channel to send Slack messages to.
SLACK_USERNAMENoThe user to send Slack messages as.
SLACK_ICONNoThe Slack icon to associate to the user/message.

You can set all of these variables within your values.yaml file under the env dict key.

⁠Usage
⁠Run locally

You can run the script locally:

cd /path/to/postgres-gcs-backup
chmod +x backup.sh
GCS_BUCKET=<gs://bucket_name> \
./backup.sh

Please note that you can set any environment variable described in the previous section! As an example, to enable the Slack alerts on failure:

SLACK_ALERTS=true \
SLACK_WEBHOOK_URL=<webhook_url> \
SLACK_CHANNEL=<slack_channel> \
SLACK_USERNAME=<slack_username> \
SLACK_ICON=<slack_icon> \
GCS_BUCKET=<gs://bucket_name> \
./backup.sh
⁠Run within Kubernetes
⁠Installing the Chart

To install the chart with the release name my-release within you Kubernetes cluster:

$ helm install --name my-release chart/postgres-gcs-backup

The command deploys the chart on the Kubernetes cluster in the default namespace. The configuration section lists the parameters that can be configured during installation.

⁠Uninstalling the Chart

To uninstall/delete the my-release deployment:

$ helm delete my-release

The command removes all the Kubernetes components associated with the chart and deletes the release.

⁠Authenticate with GCS
⁠Using the gcloud CLI

If you are running the script locally, the easiest solution is to sign in to the google account associated with your Google Cloud Storage data:

gcloud init --console-only

More information on how to setup gsutil locally here⁠.

⁠Using a service account within Kubernetes

You can create a service account⁠ with the required roles to write to GCS attached.

To use the resulting JSON key file within Kubernetes you can create a secret from it by running the following command:

  kubectl create secret generic postgres-gcs-backup \
  --from-file=credentials.json=/path/to/your/key.json

Then you will need to specify this secret name via the --set secretName=<your_secret_name> argument to the helm install command or by specifying it directly in your values.yaml file (by default, the secret name is set to postgres-gcs-backup). The key file will be mounted by default under /secrets/gcp/credentials.json and the GCS_KEY_FILE_PATH variable should point to it.

Tag summary

Content type

Image

Digest

sha256:d92a95479…

Size

74.9 MB

Last updated

about 2 years ago

docker pull diogopms/postgres-gcs-backup