bitnamicharts/thanos
Bitnami Helm chart for Thanos
1M+
Thanos is a highly available metrics system that can be added on top of existing Prometheus deployments, providing a global query view across all Prometheus installations.
Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.
helm install my-release oci://registry-1.docker.io/bitnamicharts/thanos
Looking to use Thanos in production? Try VMware Tanzu Application Catalog, the commercial edition of the Bitnami catalog.
This chart bootstraps a Thanos deployment on a Kubernetes cluster using the Helm package manager.
Bitnami charts can be used with Kubeapps for deployment and management of Helm Charts in clusters.
To install the chart with the release name my-release
:
helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/thanos
Note: You need to substitute the placeholders
REGISTRY_NAME
andREPOSITORY_NAME
with a reference to your Helm chart registry and repository. For example, in the case of Bitnami, you need to useREGISTRY_NAME=registry-1.docker.io
andREPOSITORY_NAME=bitnamicharts
.
These commands deploy Thanos on the Kubernetes cluster with the default configuration. The configuration section lists the parameters that can be configured during installation.
Tip: List all releases using
helm list
This charts allows you install several Thanos components, so you deploy an architecture as the one below:
+--------------+ +--------------+ +--------------+
| Thanos |----------------> | Thanos Store | | Thanos |
| Query | | | Gateway | | Compactor |
+--------------+ | +--------------+ +--------------+
push | | |
+--------------+ alerts +--------------+ | | storages | Downsample &
| Alertmanager | <----------| Thanos | <----| | query metrics | compact blocks
| (*) | | Ruler | | | |
+--------------+ +--------------+ | \/ |
^ | | +----------------+ |
| push alerts +--------------|----> | MinIO® (*) | <---------+
| | | |
+------------------------------+ | +----------------+
|+------------+ +------------+| | ^
|| Prometheus |->| Thanos || <----------------+ |
|| (*) |<-| Sidecar (*)|| query | inspect
|+------------+ +------------+| metrics | blocks
+------------------------------+ |
+--------------+
| Thanos |
| Bucket Web |
+--------------+
Note: Components marked with (*) are provided by subchart(s) (such as the Bitnami MinIO® chart) or external charts (such as the Bitnami kube-prometheus chart).
Check the section Integrate Thanos with Prometheus and Alertmanager for detailed instructions to deploy this architecture.
Bitnami charts allow setting resource requests and limits for all containers inside the chart deployment. These are inside the resources
value (check parameter table). Setting requests is essential for production workloads and these should be adapted to your specific use case.
To make this process easier, the chart contains the resourcesPreset
values, which automatically sets the resources
section according to different presets. Check these presets in the bitnami/common chart. However, in production workloads using resourcesPreset
is discouraged as it may not fully adapt to your specific needs. Find more information on container resource management in the official Kubernetes documentation.
This chart can be integrated with Prometheus by setting metrics.enabled
to true
. This will expose Thanos native Prometheus endpoint in the service. It will have the necessary annotations to be automatically scraped by Prometheus.
Prometheus requirements
It is necessary to have a working installation of Prometheus or Prometheus Operator for the integration to work. Install the Bitnami Prometheus helm chart or the Bitnami Kube Prometheus helm chart to easily have a working Prometheus in your cluster.
Integration with Prometheus Operator
The chart can deploy ServiceMonitor
objects for integration with Prometheus Operator installations. To do so, set the value metrics.serviceMonitor.enabled=true
. Ensure that the Prometheus Operator CustomResourceDefinitions
are installed in the cluster or it will fail with the following error:
no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
Install the Bitnami Kube Prometheus helm chart for having the necessary CRDs and the Prometheus Operator.
Thanos can encrypt communications by setting *.tls.enabled=true
(under the query.grpc.client
, query.grpc.server
, storegateway.grpc.server
and receive.grpc.server
sections) value. The chart allows three configuration options:
*.tls.ca
, *.tls.cert
, *.tls.key
(under the query.grpc.client
, query.grpc.server
, storegateway.grpc.server
and receive.grpc.server
sections) values.*.tls.existingSecret
(under the query.grpc.client
, query.grpc.server
, storegateway.grpc.server
and receive.grpc.server
sections) values. Apart from providing the secret name also set a key mapping to the ca-cert
, tls-cert
and tls-key
elements. For example:receive:
grpc:
server:
tls:
existingSecret:
name: foo
keyMapping:
ca-cert: ca.pem
tls-cert: cert.pem
tls-key: key.pem
*.tls.autoGenerated=true
(under the query.grpc.client
, query.grpc.server
, storegateway.grpc.server
and receive.grpc.server
sections).It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.
Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.
In case you want to add extra flags to any Thanos component, you can use XXX.extraFlags
parameter(s), where XXX is placeholder you need to replace with the actual component(s). For instance, to add extra flags to Thanos Store Gateway, use:
storegateway:
extraFlags:
- --sync-block-duration=3m
- --chunk-pool-size=2GB
This also works for multi-line flags. This can be useful when you want to configure caching for a particular component without using a configMap. For example, to configure the query-range response cache of the Thanos Query Frontend, use:
queryFrontend:
extraFlags:
- |
--query-range.response-cache-config=
type: MEMCACHED
config:
addresses:
- <MEMCACHED_SERVER>:11211
timeout: 500ms
max_idle_connections: 100
max_async_concurrency: 10
max_async_buffer_size: 10000
max_get_multi_concurrency: 100
max_get_multi_batch_size: 0
dns_provider_update_interval: 10s
expiration: 24h
This helm chart supports using custom Objstore configuration.
You can specify the Objstore configuration using the objstoreConfig
parameter.
In addition, you can also set an external Secret with the configuration file. This is done by setting the existingObjstoreSecret
parameter. Note that this will override the previous option. If needed you can also provide a custom Secret Key with existingObjstoreSecretItems
, please be aware that the Path of your Secret should be objstore.yml
.
This helm chart supports using custom Service Discovery configuration for Query.
You can specify the Service Discovery configuration using the query.sdConfig
parameter.
In addition, you can also set an external ConfigMap with the Service Discovery configuration file. This is done by setting the query.existingSDConfigmap
parameter. Note that this will override the previous option.
This helm chart supports using custom Ruler configuration.
You can specify the Ruler configuration using the ruler.config
parameter.
In addition, you can also set an external ConfigMap with the configuration file. This is done by setting the ruler.existingConfigmap
parameter. Note that this will override the previous option.
This helm charts supports using HTTPS and basic authentication. The underlying feature is experimental and might change in the future, so are the associated settings in the chart. For more information, please refer to Thanos documentation.
This feature can be enabled by using the following values:
https.enabled=true
. Enabling HTTPS requires the user to provide the TLS certificate and Key for Thanos, which can be done using one of the following options:
https.existingSecret
. The secret must contain the keys tls.crt
or tls.key
(key names can be renamed using the values https.keyFilename
and https.certFilename
).https.cert
and https.key
.https.autoGenerated=true
, using this value Helm will generate a self-signed key pair during the chart initialization. Not recommended for production environments.auth.basicAuthUsers.*
. An dictionary of key / values, where the keys corresponds to the users that will have access to Thanos and the values are the plaintext passwords. Passwords will be later encrypted with bcrypt.
Alternatively, provide your own Thanos http config file using the value httpConfig
or existingHttpConfigSecret
. This may cause any settings under https.*
or auth.*
to be ignored, except for the settings related to the TLS certificates. When providing a configuration file using these parameters, the chart Probes will fail to initialize unless one of the following fixes are applied:
https.enabled
or auth.basicAuthUsers
with at least one user, matching the configuration file you provided. That way Probes will be configured with HTTPS and/or basic authentication accordingly.<component>.customLivenessProbe
, <component>.customReadinessProbe
and <component>.customStartupProbe
.Thanos store supports partion based on time.
Setting time partitions will create N number of store statefulsets based on the number of items in the timePartitioning
list. Each item must contain the min and max time for querying in the supported format (find more details at Thanos documentation).
Note: leaving the
timePartitioning
list empty ([]
) will create a single store for all data.
For instance, to use 3 stores you can use a values.yaml like the one below:
timePartitioning:
# One store for data older than 6 weeks
- min: ""
max: -6w
# One store for data newer than 6 weeks and older than 2 weeks
- min: -6w
max: -2w
# One store for data newer than 2 weeks
- min: -2w
max: ""
You can also specify different resources and limits configurations for each storegateway statefulset. This is done by adding a resources.requests
and resources.limits
to each item you wish to change, as shown below:
timePartitioning:
# One store for data older than 6 weeks
- min: ""
max: -6w
# One store for data newer than 6 weeks and older than 2 weeks
- min: -6w
max: -2w
resources: #optional resources declaration for partition
requests:
cpu: 10m
memory: 100Mi
limits:
cpu: 20m
memory: 100Mi
# One store for data newer than 2 weeks
- min: -2w
max: ""
You can integrate Thanos with Prometheus & Alertmanager using this chart and the Bitnami kube-prometheus chart following the steps below:
Note: in this example we will use MinIO® (subchart) as the Objstore. Every component will be deployed in the "monitoring" namespace.
objstoreConfig: |-
type: s3
config:
bucket: thanos
endpoint: {{ include "thanos.minio.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:9000
access_key: minio
secret_key: minio123
insecure: true
query:
dnsDiscovery:
sidecarsService: kube-prometheus-prometheus-thanos
sidecarsNamespace: monitoring
bucketweb:
enabled: true
compactor:
enabled: true
storegateway:
enabled: true
ruler:
enabled: true
alertmanagers:
- http://kube-prometheus-alertmanager.monitoring.svc.cluster.local:9093
config: |-
groups:
- name: "metamonitoring"
rules:
- alert: "PrometheusDown"
expr: absent(up{prometheus="monitoring/kube-prometheus"})
metrics:
enabled: true
serviceMonitor:
enabled: true
minio:
enabled: true
auth:
rootPassword: minio123
rootUser: minio
monitoringBuckets: thanos
accessKey:
password: minio
secretKey:
password: minio123
For Helm 3:
$ kubectl create namespace monitoring
helm install kube-prometheus \
--set prometheus.thanos.create=true \
--namespace monitoring \
bitnami/kube-prometheus
helm install thanos \
--values values.yaml \
--namespace monitoring \
oci://REGISTRY_NAME/REPOSITORY_NAME/thanos
Note: You need to substitute the placeholders
REGISTRY_NAME
andREPOSITORY_NAME
with a reference to your Helm chart registry and repository. For example, in the case of Bitnami, you need to useREGISTRY_NAME=registry-1.docker.io
andREPOSITORY_NAME=bitnamicharts
.
That's all! Now you have Thanos fully integrated with Prometheus and Alertmanager.
There are cases where you may want to deploy extra objects, such a ConfigMap containing your app's configuration or some extra deployment with a micro service used by your app. For covering this case, the chart allows adding the full specification of other objects using the extraDeploy
parameter.
This chart allows you to set your custom affinity using the XXX.affinity
parameter(s). Find more information about Pod's affinity in the kubernetes documentation.
As an alternative, you can use of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common chart. To do so, set the XXX.podAffinityPreset
, XXX.podAntiAffinityPreset
, or XXX.nodeAffinityPreset
parameters.
To back up and restore Helm chart deployments on Kubernetes, you need to back up the persistent volumes from the source deployment and attach them to a new deployment using Velero, a Kubernetes backup/restore tool. Find the instructions for using Velero in this guide.
The data is persisted by default using PVC(s) on Thanos components. You can disable the persistence setting the XXX.persistence.enabled
parameter(s) to false
. A default StorageClass
is needed in the Kubernetes cluster to dynamically provision the volumes. Specify another StorageClass in the XXX.persistence.storageClass
parameter(s) or set XXX.persistence.existingClaim
if you have already existing persistent volumes to use.
Note: you need to substitute the XXX placeholders above with the actual component(s) you want to configure.
As the images run as non-root by default, it is necessary to adjust the ownership of the persistent volumes so that the containers can write data into it.
By default, the chart is configured to use Kubernetes Security Context to automatically change the ownership of the volumes. However, this feature does not work in all Kubernetes distributions. As an alternative, this chart supports using an initContainer to change the ownership of the volumes before mounting it in the final destination.
You can enable this initContainer by setting volumePermissions.enabled
to true
.
Name | Description | Value |
---|---|---|
global.imageRegistry | Global Docker image registry | "" |
global.imagePullSecrets | Global Docker registry secret names as an array | [] |
global.defaultStorageClass | Global default StorageClass for Persistent Volume(s) | "" |
global.storageClass | DEPRECATED: use global.defaultStorageClass instead | "" |
global.security.allowInsecureImages | Allows skipping image verification | false |
global.compatibility.openshift.adaptSecurityContext | Adapt the securityContext sections of the deployment to make them compatible with Openshift restricted-v2 SCC: remove runAsUser, runAsGroup and fsGroup and let the platform use their allowed default IDs. Possible values: auto (apply if the detected running cluster is Openshift), force (perform the adaptation always), disabled (do not perform adaptation) | auto |
Name | Description | Value |
---|---|---|
kubeVersion | Force target Kubernetes version (using Helm capabilities if not set) | "" |
nameOverride | String to partially override common.names.fullname template (will maintain the release name) | "" |
fullnameOverride | String to fully override common.names.fullname template | "" |
commonLabels | Add labels to all the deployed resources | {} |
commonAnnotations | Add annotations to all the deployed resources | {} |
clusterDomain | Kubernetes Cluster Domain | cluster.local |
extraDeploy | Array of extra objects to deploy with the release | [] |
Name | Description | Value |
---|---|---|
image.registry | Thanos image registry | REGISTRY_NAME |
image.repository | Thanos image repository | REPOSITORY_NAME/thanos |
image.digest | Thanos image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag |
Note: the README for this chart is longer than the DockerHub length limit of 25000, so it has been trimmed. The full README can be found at https://github.com/bitnami/charts/blob/main/bitnami/thanos/README.md