Introducing our new CEO Don Johnson - Read More

bitnamicharts/flink

Verified Publisher

By VMware

Updated 1 day ago

Bitnami Helm chart for Apache Flink

Image
Helm
Data Science
Internet of Things
Machine Learning & AI
0

50K+

Bitnami package for Apache Flink

Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams.

Overview of Apache Flink

Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

TL;DR

helm install my-release oci://registry-1.docker.io/bitnamicharts/flink

Looking to use Apache Flink in production? Try VMware Tanzu Application Catalog, the commercial edition of the Bitnami catalog.

Introduction

This chart bootstraps a flink deployment on a Kubernetes cluster using the Helm package manager.

Bitnami charts can be used with Kubeapps for deployment and management of Helm Charts in clusters.

Prerequisites

  • Kubernetes 1.23+
  • Helm 3.8.0+
  • PV provisioner support in the underlying infrastructure
  • ReadWriteMany volumes for deployment scaling

Installing the Chart

To install the chart with the release name my-release:

helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/flink

Note: You need to substitute the placeholders REGISTRY_NAME and REPOSITORY_NAME with a reference to your Helm chart registry and repository. For example, in the case of Bitnami, you need to use REGISTRY_NAME=registry-1.docker.io and REPOSITORY_NAME=bitnamicharts.

These commands deploy flink on the Kubernetes cluster in the default configuration. The Parameters section lists the parameters that can be configured during installation.

Tip: List all releases using helm list

Configuration and installation details

Resource requests and limits

Bitnami charts allow setting resource requests and limits for all containers inside the chart deployment. These are inside the resources value (check parameter table). Setting requests is essential for production workloads and these should be adapted to your specific use case.

To make this process easier, the chart contains the resourcesPreset values, which automatically sets the resources section according to different presets. Check these presets in the bitnami/common chart. However, in production workloads using resourcesPreset is discouraged as it may not fully adapt to your specific needs. Find more information on container resource management in the official Kubernetes documentation.

Rolling VS Immutable tags

It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.

Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.

Backup and restore

To back up and restore Helm chart deployments on Kubernetes, you need to back up the persistent volumes from the source deployment and attach them to a new deployment using Velero, a Kubernetes backup/restore tool. Find the instructions for using Velero in this guide.

Additional environment variables

In case you want to add extra environment variables (useful for advanced operations like custom init scripts), you can use the extraEnvVars property inside each of the subsections: jobmanager, taskmanager.

jobmanager:
  extraEnvVars:
    - name: ENV_VAR_NAME
      value: ENV_VAR_VALUE
taskmanager:
  extraEnvVars:
    - name: ENV_VAR_NAME
      value: ENV_VAR_VALUE

Alternatively, you can use a ConfigMap or a Secret with the environment variables. To do so, use the extraEnvVarsCM or the extraEnvVarsSecret values.

Sidecars

If additional containers are needed in the same pod as flink (such as additional metrics or logging exporters), they can be defined using the sidecars parameter inside each of the subsections: jobmanager, taskmanager .

sidecars:
- name: your-image-name
  image: your-image
  imagePullPolicy: Always
  ports:
  - name: portname
    containerPort: 1234

If these sidecars export extra ports, extra port definitions can be added using the service.extraPorts parameter (where available), as shown in the example below:

service:
  extraPorts:
  - name: extraPort
    port: 11311
    targetPort: 11311

NOTE: This Helm chart already includes sidecar containers for the Prometheus exporters (where applicable). These can be activated by adding the --enable-metrics=true parameter at deployment time. The sidecars parameter should therefore only be used for any extra sidecar containers.

If additional init containers are needed in the same pod, they can be defined using the initContainers parameter. Here is an example:

initContainers:
  - name: your-image-name
    image: your-image
    imagePullPolicy: Always
    ports:
      - name: portname
        containerPort: 1234

Learn more about sidecar containers and init containers.

Pod affinity

This chart allows you to set your custom affinity using the affinity parameter. Find more information about Pod affinity in the kubernetes documentation.

As an alternative, use one of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common chart. To do so, set the podAffinityPreset, podAntiAffinityPreset, or nodeAffinityPreset parameters inside each of the subsections: jobmanager, taskmanager.

Persistence

The Bitnami Flink image stores the trace onto an external database. Persistent Volume Claims are used to keep the data across deployments.

Parameters

Global parameters
NameDescriptionValue
global.imageRegistryGlobal Docker image registry""
global.imagePullSecretsGlobal Docker registry secret names as an array[]
global.defaultStorageClassGlobal default StorageClass for Persistent Volume(s)""
global.storageClassDEPRECATED: use global.defaultStorageClass instead""
global.security.allowInsecureImagesAllows skipping image verificationfalse
global.compatibility.openshift.adaptSecurityContextAdapt the securityContext sections of the deployment to make them compatible with Openshift restricted-v2 SCC: remove runAsUser, runAsGroup and fsGroup and let the platform use their allowed default IDs. Possible values: auto (apply if the detected running cluster is Openshift), force (perform the adaptation always), disabled (do not perform adaptation)auto
Common parameters
NameDescriptionValue
nameOverrideString to partially override common.names.fullname""
fullnameOverrideString to fully override common.names.fullname""
kubeVersionForce target Kubernetes version (using Helm capabilities if not set)""
commonLabelsLabels to add to all deployed objects (sub-charts are not considered){}
commonAnnotationsAnnotations to add to all deployed objects{}
clusterDomainDefault Kubernetes cluster domaincluster.local
extraDeployArray of extra objects to deploy with the release[]
diagnosticMode.enabledEnable diagnostic mode (all probes will be disabled and the command will be overridden)false
diagnosticMode.commandCommand to override all containers in the deployment["sleep"]
diagnosticMode.argsArgs to override all containers in the deployment["infinity"]
Apache Flink parameters
NameDescriptionValue
image.registryApache Flink image registryREGISTRY_NAME
image.repositoryApache Flink image repositoryREPOSITORY_NAME/flink
image.digestApache Flink image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag""
image.pullPolicyimage pull policyIfNotPresent
image.pullSecretsApache Flink image pull secrets[]
image.debugEnable image debug modefalse
Jobmanager deployment parameters
NameDescriptionValue
jobmanager.commandCommand for running the container (set to default if not set). Use array form[]
jobmanager.argsArgs for running the container (set to default if not set). Use array form[]
jobmanager.lifecycleHooksOverride default etcd container hooks{}
jobmanager.automountServiceAccountTokenMount Service Account token in podfalse
jobmanager.hostAliasesSet pod host aliases[]
jobmanager.extraEnvVarsExtra environment variables to be set on flink container[]
jobmanager.extraEnvVarsCMName of existing ConfigMap containing extra env vars""
jobmanager.extraEnvVarsSecretName of existing Secret containing extra env vars""
jobmanager.replicaCountNumber of Apache Flink Jobmanager replicas1
jobmanager.livenessProbe.enabledEnable livenessProbe on Jobmanager nodestrue
jobmanager.livenessProbe.initialDelaySecondsInitial delay seconds for livenessProbe20
jobmanager.livenessProbe.periodSecondsPeriod seconds for livenessProbe10
jobmanager.livenessProbe.timeoutSecondsTimeout seconds for livenessProbe1
jobmanager.livenessProbe.failureThresholdFailure threshold for livenessProbe3
jobmanager.livenessProbe.successThresholdSuccess threshold for livenessProbe1
jobmanager.startupProbe.enabledEnable startupProbe on Jobmanager containerstrue
jobmanager.startupProbe.initialDelaySecondsInitial delay seconds for startupProbe20
jobmanager.startupProbe.periodSecondsPeriod seconds for startupProbe10
jobmanager.startupProbe.timeoutSecondsTimeout seconds for startupProbe1
jobmanager.startupProbe.failureThresholdFailure threshold for startupProbe15
jobmanager.startupProbe.successThresholdSuccess threshold for startupProbe1
jobmanager.readinessProbe.enabledEnable readinessProbetrue
jobmanager.readinessProbe.initialDelaySecondsInitial delay seconds for readinessProbe20
jobmanager.readinessProbe.periodSecondsPeriod seconds for readinessProbe10
jobmanager.readinessProbe.timeoutSecondsTimeout seconds for readinessProbe1
jobmanager.readinessProbe.failureThresholdFailure threshold for readinessProbe15
jobmanager.readinessProbe.successThresholdSuccess threshold for readinessProbe1
jobmanager.customLivenessProbeCustom livenessProbe that overrides the default one{}
jobmanager.customStartupProbeOverride default startup probe{}
jobmanager.customReadinessProbeOverride default readiness probe{}
jobmanager.resourcesPresetSet container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if jobmanager.resources is set (jobmanager.resources is recommended for production).small
jobmanager.resourcesSet container requests and limits for different resources like CPU or memory (essential for production workloads){}
jobmanager.extraVolumeMountsOptionally specify extra list of additional volumeMounts for flink container[]
jobmanager.containerPorts.rpcPort for RPC6123
`job

Note: the README for this chart is longer than the DockerHub length limit of 25000, so it has been trimmed. The full README can be found at https://github.com/bitnami/charts/blob/main/bitnami/flink/README.md

Docker Pull Command

docker pull bitnamicharts/flink
Bitnami