bitnamicharts/logstash
Bitnami Helm chart for Logstash
100K+
Logstash is an open source data processing engine. It ingests data from multiple sources, processes it, and sends the output to final destination in real-time. It is a core component of the ELK stack.
Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.
helm install my-release oci://registry-1.docker.io/bitnamicharts/logstash
Looking to use Logstash in production? Try VMware Tanzu Application Catalog, the commercial edition of the Bitnami catalog.
This chart bootstraps a logstash deployment on a Kubernetes cluster using the Helm package manager.
Bitnami charts can be used with Kubeapps for deployment and management of Helm Charts in clusters.
To install the chart with the release name my-release
:
helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/logstash
Note: You need to substitute the placeholders
REGISTRY_NAME
andREPOSITORY_NAME
with a reference to your Helm chart registry and repository. For example, in the case of Bitnami, you need to useREGISTRY_NAME=registry-1.docker.io
andREPOSITORY_NAME=bitnamicharts
.
These commands deploy logstash on the Kubernetes cluster in the default configuration. The configuration section lists the parameters that can be configured during installation.
Tip: List all releases using
helm list
Bitnami charts allow setting resource requests and limits for all containers inside the chart deployment. These are inside the resources
value (check parameter table). Setting requests is essential for production workloads and these should be adapted to your specific use case.
To make this process easier, the chart contains the resourcesPreset
values, which automatically sets the resources
section according to different presets. Check these presets in the bitnami/common chart. However, in production workloads using resourcesPreset
is discouraged as it may not fully adapt to your specific needs. Find more information on container resource management in the official Kubernetes documentation.
It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.
Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.
The service(s) created by the deployment can be exposed within or outside the cluster using any of the following approaches:
ingress.enabled=true
to expose the corresponding service(s) through Ingress.service.type=ClusterIP
to choose this approach.service.type=NodePort
to choose this approach.service.type=LoadBalancer
to choose this approach.By default, this Helm chart provides a basic configuration for Logstash: listening to HTTP requests on port 8080 and writing them to the standard output.
This Logstash configuration can be adjusted using the input, filter, and output parameters, which allow specification of the input, filter and output plugins configuration respectively. In addition to these options, the chart also supports reading configuration from an external ConfigMap via the existingConfiguration parameter. Note that this will override the parameters discussed previously.
The chart supports the use of multiple pipelines by setting the enableMultiplePipelines
parameter to true
.
To do this, place the pipelines.yml
file in the files/conf
directory, together with the rest of the desired configuration files. If the enableMultiplePipelines
parameter is set to true
but the pipelines.yml
file does not exist in the mounted volume, a dummy file is created using the default configuration (a single pipeline).
The chart also supports setting an external ConfigMap with all the configuration filesvia the existingConfiguration
parameter.
Here is an example of creating multiple pipelines using a ConfigMap:
Create a ConfigMap with the configuration files:
$ cat bye.conf
input {
file {
path => "/tmp/bye"
}
}
output {
stdout { }
}
$ cat hello.conf
input {
file {
path => "/tmp/hello"
}
}
output {
stdout { }
}
$ cat pipelines.yml
- pipeline.id: hello
path.config: "/opt/bitnami/logstash/config/hello.conf"
- pipeline.id: bye
path.config: "/opt/bitnami/logstash/config/bye.conf"
$ kubectl create cm multipleconfig --from-file=pipelines.yml --from-file=hello.conf --from-file=bye.conf
Deploy the Helm chart with the enableMultiplePipelines
parameter:
helm install logstash . --set enableMultiplePipelines=true --set existingConfiguration=multipleconfig
Create dummy events in the tracked files and check the result in the Logstash output:
kubectl exec -ti logstash-0 -- bash -c 'echo hi >> /tmp/hello'
kubectl exec -ti logstash-0 -- bash -c 'echo bye >> /tmp/bye'
To add extra environment variables, use the extraEnvVars
property.
extraEnvVars:
- name: ELASTICSEARCH_HOST
value: "x.y.z"
To add extra environment variables from an external ConfigMap or secret, use the extraEnvVarsCM
and extraEnvVarsSecret
properties. Note that the secret and ConfigMap should be already available in the namespace.
extraEnvVarsSecret: logstash-secrets
extraEnvVarsCM: logstash-configmap
This chart allows you to set custom Pod affinity using the affinity
parameter. Find more information about Pod affinity in the kubernetes documentation.
As an alternative, use one of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common chart. To do so, set the podAffinityPreset
, podAntiAffinityPreset
, or nodeAffinityPreset
parameters.
To back up and restore Helm chart deployments on Kubernetes, you need to back up the persistent volumes from the source deployment and attach them to a new deployment using Velero, a Kubernetes backup/restore tool. Find the instructions for using Velero in this guide.
The Bitnami Logstash image stores the Logstash data at the /bitnami/logstash/data
path of the container.
Persistent Volume Claims (PVCs) are used to keep the data across deployments. This is known to work in GCE, AWS, and minikube.
See the Parameters section to configure the PVC or to disable persistence.
Name | Description | Value |
---|---|---|
global.imageRegistry | Global Docker image registry | "" |
global.imagePullSecrets | Global Docker registry secret names as an array | [] |
global.defaultStorageClass | Global default StorageClass for Persistent Volume(s) | "" |
global.storageClass | DEPRECATED: use global.defaultStorageClass instead | "" |
global.security.allowInsecureImages | Allows skipping image verification | false |
global.compatibility.openshift.adaptSecurityContext | Adapt the securityContext sections of the deployment to make them compatible with Openshift restricted-v2 SCC: remove runAsUser, runAsGroup and fsGroup and let the platform use their allowed default IDs. Possible values: auto (apply if the detected running cluster is Openshift), force (perform the adaptation always), disabled (do not perform adaptation) | auto |
Name | Description | Value |
---|---|---|
kubeVersion | Force target Kubernetes version (using Helm capabilities if not set) | "" |
nameOverride | String to partially override logstash.fullname template (will maintain the release name) | "" |
fullnameOverride | String to fully override logstash.fullname template | "" |
clusterDomain | Default Kubernetes cluster domain | cluster.local |
commonAnnotations | Annotations to add to all deployed objects | {} |
commonLabels | Labels to add to all deployed objects | {} |
extraDeploy | Array of extra objects to deploy with the release (evaluated as a template). | [] |
diagnosticMode.enabled | Enable diagnostic mode (all probes will be disabled and the command will be overridden) | false |
diagnosticMode.command | Command to override all containers in the deployment | ["sleep"] |
diagnosticMode.args | Args to override all containers in the deployment | ["infinity"] |
Name | Description | Value |
---|---|---|
image.registry | Logstash image registry | REGISTRY_NAME |
image.repository | Logstash image repository | REPOSITORY_NAME/logstash |
image.digest | Logstash image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag | "" |
image.pullPolicy | Logstash image pull policy | IfNotPresent |
image.pullSecrets | Specify docker-registry secret names as an array | [] |
image.debug | Specify if debug logs should be enabled | false |
automountServiceAccountToken | Mount Service Account token in pod | false |
hostAliases | Add deployment host aliases | [] |
configFileName | Logstash configuration file name. It must match the name of the configuration file mounted as a configmap. | logstash.conf |
enableMonitoringAPI | Whether to enable the Logstash Monitoring API or not Kubernetes cluster domain | true |
monitoringAPIPort | Logstash Monitoring API Port | 9600 |
extraEnvVars | Array containing extra env vars to configure Logstash | [] |
extraEnvVarsSecret | To add secrets to environment | "" |
extraEnvVarsCM | To add configmaps to environment | "" |
input | Input Plugins configuration | "" |
extraInput | Extra Input Plugins configuration | "" |
filter | Filter Plugins configuration | "" |
output | Output Plugins configuration | "" |
existingConfiguration | Name of existing ConfigMap object with the Logstash configuration (input , filter , and output will be ignored). | "" |
extraConfigurationFiles | Extra configuration files to be added to the configuration ConfigMap and mounted at /bitnami/logstash/config. Rendered as a template. | {} |
enableMultiplePipelines | Allows user to use multiple pipelines | false |
extraVolumes | Array to add extra volumes (evaluated as a template) | [] |
extraVolumeMounts | Array to add extra mounts (normally used with extraVolumes, evaluated as a template) | [] |
serviceAccount.create | Enable creation of ServiceAccount for Logstash pods | true |
serviceAccount.name | The name of the service account to use. If not set and create is true , a name is generated | "" |
serviceAccount.automountServiceAccountToken | Allows automount of ServiceAccountToken on the serviceAccount created | false |
serviceAccount.annotations | Additional custom annotations for the ServiceAccount | {} |
containerPorts | Array containing the ports to open in the Logstash container (evaluated as a template) | [] |
extraContainerPorts | Array containing extra ports to open in the Logstash container (evaluated as a template) | [] |
initContainers | Add additional init containers to the Logstash pod(s) | [] |
sidecars | Add additional sidecar containers to the Logstash pod(s) |
Note: the README for this chart is longer than the DockerHub length limit of 25000, so it has been trimmed. The full README can be found at https://github.com/bitnami/charts/blob/main/bitnami/logstash/README.md