Bitnami package for PostgreSQL HA

This PostgreSQL cluster solution includes the PostgreSQL replication manager, an open-source tool for managing replication and failover on PostgreSQL clusters.

Overview of PostgreSQL HA⁠

Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

TL;DR

helm install my-release oci://registry-1.docker.io/bitnamicharts/postgresql-ha

Looking to use PostgreSQL HA in production? Try VMware Tanzu Application Catalog⁠, the commercial edition of the Bitnami catalog.

Introduction

This Helm⁠ chart installs PostgreSQL⁠ with HA architecture in a Kubernetes cluster. Welcome to contribute⁠ to Helm Chart for PostgreSQL HA.

This Helm chart has been developed based on bitnami/postgresql⁠ chart but including some changes to guarantee high availability such as:

• A new deployment, service have been added to deploy Pgpool-II⁠ to act as proxy for PostgreSQL backend. It helps to reduce connection overhead, acts as a load balancer for PostgreSQL, and ensures database node failover.
• Replacing bitnami/postgresql with bitnami/postgresql-repmgr which includes and configures repmgr⁠. Repmgr ensures standby nodes assume the primary role when the primary node is unhealthy.

Bitnami charts can be used with Kubeapps⁠ for deployment and management of Helm Charts in clusters.

Differences between the PostgreSQL-HA and PostgreSQL Helm charts

There are two different ways to deploy a PostgreSQL cluster, using the PostgreSQL Helm chart or the PostgreSQL High Availability (HA) Helm chart. Both solutions provide a simple and reliable way to run PostgreSQL in a production environment. Keep reading to discover the differences between them and check which one better suits your needs.

• Both the PostgreSQL HA and the PostgreSQL chart configures a cluster with a primary/replica topology. The primary node has writing permissions while replication is on the replica nodes which have read-only permissions.
• The PostgreSQL HA Helm chart deploys a cluster with three nodes by default, one for pgpool, one primary, and one replica for PostgreSQL. The PostgreSQL chart configures a cluster with two nodes by default (one primary and one replica).
• The PostgreSQL HA Helm chart uses pgpool to handle the connection to the nodes. pgpool is responsible to spread the queries among nodes.
• The PostgreSQL HA Helm chart includes a repmgr module that ensures high-availability thanks to automatic membership control. If the primary is down, any of the replica nodes will be promoted as primary to avoid data loss.

The following diagram shows you the options you have for using Bitnami's PostgreSQL solutions in your deployments:

Prerequisites

• Kubernetes 1.23+
• Helm 3.8.0+

Installing the Chart

To install the chart with the release name my-release:

helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/postgresql-ha

Configuration and installation details

Resource requests and limits

Bitnami charts allow setting resource requests and limits for all containers inside the chart deployment. These are inside the resources value (check parameter table). Setting requests is essential for production workloads and these should be adapted to your specific use case.

To make this process easier, the chart contains the resourcesPreset values, which automatically sets the resources section according to different presets. Check these presets in the bitnami/common chart⁠. However, in production workloads using resourcesPreset is discouraged as it may not fully adapt to your specific needs. Find more information on container resource management in the official Kubernetes documentation⁠.

Prometheus metrics

This chart can be integrated with Prometheus by setting metrics.enabled to true. This will deploy a sidecar container with postgres_exporter⁠ in all pods. It will also create metrics services that can be configured under the metrics.service section. These services will be have the necessary annotations to be automatically scraped by Prometheus.

Prometheus requirements

It is necessary to have a working installation of Prometheus or Prometheus Operator for the integration to work. Install the Bitnami Prometheus helm chart⁠ or the Bitnami Kube Prometheus helm chart⁠ to easily have a working Prometheus in your cluster.

Integration with Prometheus Operator

The chart can deploy ServiceMonitor objects for integration with Prometheus Operator installations. To do so, set the value metrics.serviceMonitor.enabled=true. Ensure that the Prometheus Operator CustomResourceDefinitions are installed in the cluster or it will fail with the following error:

no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"

Install the Bitnami Kube Prometheus helm chart⁠ for having the necessary CRDs and the Prometheus Operator.

Update credentials

Bitnami charts configure credentials at first boot. Any further change in the secrets or credentials require manual intervention. Follow these instructions:

• Update the user password following the upstream documentation⁠
• Update the password secret with the new values (replace the SECRET_NAME, POSTGRES_PASSWORD and PASSWORD and REPMGR_PASSWORD placeholders)

kubectl create secret generic SECRET_NAME --from-literal=postgres-password=POSTGRES_PASSWORD --from-literal=password=PASSWORD --from-literal=repmgr-password=REPMGR_PASSWORD --dry-run -o yaml | kubectl apply -f -

Rolling VS Immutable tags⁠

It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.

Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.

Use a different PostgreSQL version

To modify the application version used in this chart, specify a different version of the image using the image.tag parameter and/or a different repository using the image.repository parameter.

Use a volume for /dev/shm

When working with huge databeses, /dev/shm can run out of space. A way to fix this is to use the postgresql.extraVolumes and postgresql.extraVolumeMounts values. In the example below, we set an emptyDir volume with 512Mb:

postgresql:
  extraVolumes:
    - name: dshm
      emptyDir:
        medium: Memory
        sizeLimit: 512Mi
  extraVolumeMounts:
    - name: dshm
      mountPath: /dev/shm

Configure the way how to expose PostgreSQL

• ClusterIP: Exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster. Set service.type=ClusterIP to choose this service type.
• NodePort: Exposes the service on each Node's IP at a static port (the NodePort). You will be able to contact the NodePort service, from outside the cluster, by requesting NodeIP:NodePort. Set service.type=NodePort to choose this service type.
• LoadBalancer: Exposes the service externally using a cloud provider's load balancer. Set service.type=LoadBalancer to choose this service type.

Adjust permissions of persistent volume mountpoint

As the images run as non-root by default, it is necessary to adjust the ownership of the persistent volumes so that the containers can write data into it.

By default, the chart is configured to use Kubernetes Security Context to automatically change the ownership of the volume. However, this feature does not work in all Kubernetes distributions. As an alternative, this chart supports using an initContainer to change the ownership of the volume before mounting it in the final destination.

You can enable this initContainer by setting volumePermissions.enabled to true.

LDAP

LDAP support can be enabled in the chart by specifying the ldap. parameters while creating a release. The following parameters should be configured to properly enable the LDAP support in the chart.

• ldap.enabled: Enable LDAP support. Defaults to false.
• ldap.uri: LDAP URL beginning in the form ldap[s]://<hostname>:<port>. No defaults.
• ldap.base: LDAP base DN. No defaults.
• ldap.binddn: LDAP bind DN. No defaults.
• ldap.bindpw: LDAP bind password. No defaults.
• ldap.bslookup: LDAP base lookup. No defaults.
• ldap.nss_initgroups_ignoreusers: LDAP ignored users. root,nslcd.
• ldap.scope: LDAP search scope. No defaults.
• ldap.tls_reqcert: LDAP TLS check on server certificates. No defaults.

For example:

ldap.enabled="true"
ldap.uri="ldap://my_ldap_server"
ldap.base="dc=example\,dc=org"
ldap.binddn="cn=admin\,dc=example\,dc=org"
ldap.bindpw="admin"
ldap.bslookup="ou=group-ok\,dc=example\,dc=org"
ldap.nss_initgroups_ignoreusers="root\,nslcd"
ldap.scope="sub"
ldap.tls_reqcert="demand"

Next, login to the PostgreSQL server using the psql client and add the PAM authenticated LDAP users.

Securing traffic using TLS

The chart handles two main flows of traffic information:

• Connections between end-clients and PgPool (sometimes referred to as frontend connections).
• Internal connections between PgPool and PostgreSQL nodes (sometimes referred to as backend connections).

The Bitnami postgresql-ha chart allows configuring the securitization of both types of traffic using TLS.

Encrypt traffic between clients and Pgpool (frontend)

TLS for end-client connections can be enabled in the chart by specifying the pgpool.tls.* parameters when installing a release. Below you can find detailed information about these parameters:

• pgpool.tls.enabled: Enable TLS support. Defaults to false.
• pgpool.tls.certificatesSecret: Name of an existing secret that contains the certificates. No defaults.
• pgpool.tls.certFilename: Certificate filename. No defaults.
• pgpool.tls.certKeyFilename: Certificate key filename. No defaults.

For example:

• First, create a secret with the certificates files. You will need to generate previously the certificate files:

  kubectl create secret generic pgpool-tls-secret --from-file=./cert.crt --from-file=./cert.key --from-file=./ca.crt
  

  Copy

• Then, install the chart using the following parameters:

  pgpool.tls.enabled=true
  pgpool.tls.certificatesSecret="pgpool-tls-secret"
  pgpool.tls.certFilename="cert.crt"
  pgpool.tls.certKeyFilename="cert.key"
  

  Copy

Enable client certificate authentication

When TLS is configured for frontend connections, the server can be configured to authenticate clients by verifying their provided TLS certificate is valid and trusted. Hence, the client will not be sent a password prompt.

You can enable this authentication feature additionally specifying the following parameter:

• postgresql.tls.certCAFilename: CA Certificate filename. No defaults.

  $ psql --host postgresql-ha-pgpool -d "dbname=XXXXX user=YYYYYY sslcert=client.crt sslkey=client.key sslmode=require"
  psql (14.4)
  SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
  Type "help" for help.
  
  postgres=>
  

  Copy

Clients using this method to authenticate will be required to provide a certificate with the CN (Common Name) field matching the requested database user name. Please, refer to the official documentation⁠ for further information.

Encrypt traffic between Pgpool and PostgreSQL nodes (backend)

TLS for backend connections can be enabled in the chart by specifying the postgresql.tls.* parameters while creating a release. Below you can find detailed information about these parameters:

• postgresql.tls.enabled: Enable TLS support. Defaults to false
• postgresql.tls.certificatesSecret: Name of an existing secret that contains the certificates. No defaults.
• postgresql.tls.certFilename: Certificate filename. No defaults.
• postgresql.tls.certKeyFilename: Certificate key filename. No defaults.

For example:

• First, create a secret with the certificates files. You will need to generate previously the certificate files:

  kubectl create secret generic postgresql-tls-secret --from-file=./cert.crt --from-file=./cert.key --from-file=./ca.crt
  

  Copy

• Then, install the chart using the following parameters:

  postgresql.tls.enabled=true
  postgresql.tls.certificatesSecret="postgresql-tls-secret"
  postgresql.tls.certFilename="cert.crt"
  postgresql.tls.certKeyFilename="cert.key"
  

  Copy

If you want to encrypt both frontend and backend traffics, you may use the same secret for Pgpool and PostgreSQL TLS configuration.

repmgr.conf / postgresql.conf / pg_hba.conf / pgpool.conf files as configMap

This helm chart also supports to customize the whole configuration file.

You can specify the Pgpool, PostgreSQL and Repmgr configuration using the pgpool.configuration, postgresql.configuration, postgresql.pgHbaConfiguration, and postgresql.repmgrConfiguration parameters. The corresponding files will be mounted as ConfigMap to the containers and it will be used for configuring Pgpool, Repmgr and the PostgreSQL server.

In addition to this option, you can also set an external ConfigMap(s) with all the configuration files. This is done by setting the postgresql.configurationCM and pgpool.configurationCM parameters. Note that this will override the previous options.

Allow settings to be loaded from files other than the default postgresql.conf

If you don't want to provide the whole PostgreSQL configuration file and only specify certain parameters, you can specify the extended configuration using the postgresql.extendedConf parameter. A file will be mounted as configMap to the containers adding/overwriting the default configuration using the include_dir directive that allows settings to be loaded from files other than the default postgresql.conf.

In addition to this option, you can also set an external ConfigMap with all the extra configuration files. This is done by setting the postgresql.extendedConfCM parameter. Note that this will override the previous option.

Initialize a fresh instance

The Bitnami PostgreSQL with Repmgr⁠ image allows you to use your custom scripts to initialize a fresh instance. You can specify custom scripts using the initdbScripts parameter as dict so they can be consumed as a ConfigMap.

In addition to this option, you can also set an external ConfigMap with all the initialization scripts. This is done by setting the initdbScriptsCM parameter. Note that this will override the two previous options. If your initialization scripts contain sensitive information such as credentials or passwords, you can use the initdbScriptsSecret parameter.

The above parameters (initdbScripts, initdbScriptsCM, and initdbScriptsSecret) are supported in both StatefulSet by prepending postgresql or pgpool to the parameter, depending on the use case (see above parameters table).

The allowed extensions are .sh, .sql and .sql.gz in the postgresql container while only .sh in the case of the pgpool one.

+info: https://github.com/bitnami/containers/tree/main/bitnami/postgresql#initializing-a-new-instance⁠ and https://github.com/bitnami/containers/tree/main/bitnami/pgpool#initializing-with-custom-scripts⁠

Use of global variables

In more complex scenarios, we may have the following tree of dependencies

                     +--------------+
                     |              |
        +------------+   Chart 1    +-----------+
        |            |              |           |
        |            --------+------+           |
        |                    |                  |
        |                    |                  |
        |                    |                  |
        |                    |                  |
        v                    v                  v
+-------+------+    +--------+------+  +--------+------+
|              |    |               |  |               |
| PostgreSQL HA |  | Sub-chart 1 |  | Sub-chart 2 |
|---------------|--|-------------|--|-------------|
+--------------+    +---------------+  +---------------+

The three charts below depend on the parent chart Chart 1. However, subcharts 1 and 2 may need to connect to PostgreSQL HA as well. In order to do so, subcharts 1 and 2 need to know the PostgreSQL HA credentials, so one option for deploying could be deploy Chart 1 with the following parameters:

postgresql.postgresqlPassword=testtest
subchart1.postgresql.postgresqlPassword=testtest
subchart2.postgresql.postgresqlPassword=testtest
postgresql.postgresqlDatabase=db1
subchart1.postgresql.postgresqlDatabase=db1
subchart2.postgresql.postgresqlDatabase=db1

If the number of dependent sub-charts increases, installing the chart with parameters can become increasingly difficult. An alternative would be to set the credentials using global variables as follows:

global.postgresql.postgresqlPassword=testtest
global.postgresql.postgresqlDatabase=db1

This way, the credentials will be available in all of the subcharts.

Setting Pod's affinity

This chart allows you to set your custom affinity using the XXX.affinity paremeter(s). Find more information about Pod's affinity in the kubernetes documentation⁠.

As an alternative, you can use of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common⁠ chart. To do so, set the XXX.podAffinityPreset, XXX.podAntiAffinityPreset, or XXX.nodeAffinityPreset parameters.

Backup and restore

To back up and restore Helm chart deployments on Kubernetes, you need to back up the persistent volumes from the source deployment and attach them to a new deployment using Velero⁠, a Kubernetes backup/restore tool. Find the instructions for using Velero in this guide⁠.

Persistence

The data is persisted by default using PVC templates in the PostgreSQL statefulset. You can disable the persistence setting the persistence.enabled parameter to false. A default StorageClass is needed in the Kubernetes cluster to dynamically provision the volumes. Specify another StorageClass in the persistence.storageClass or set persistence.existingClaim if you have already existing persistent volumes to use.

Parameters

Global parameters

Note: the README for this chart is longer than the DockerHub length limit of 25000, so it has been trimmed. The full README can be found at https://github.com/bitnami/charts/blob/main/bitnami/postgresql-ha/README.md⁠