etlworks/kubernetes-lifecycle-adapter repository overview

⁠Etlworks Kubernetes Lifecycle Adapter

Kubernetes controller for safe capacity changes and upgrades of Etlworks managed application clusters.

Image: etlworks/kubernetes-lifecycle-adapter

Use this image with the matching Etlworks managed application image⁠ and its managed-cluster release bundle. The adapter runs separately from the application pods so lifecycle operations can continue while application members start, drain or restart.

⁠What the adapter does

The adapter reconciles desired capacity with actual application members. Before a planned removal, it closes new execution admission, withdraws traffic and waits for the application's removal approval. It also coordinates approved managed application/embedded-engine upgrades and reconciles application-stored elasticity policy with the installed autoscaling configuration.

The adapter does not execute flows, select cloud machines or replace the application scheduler. A Kubernetes process restart does not authorize automatic replay of the business executions that were running in the failed process.

⁠Select a compatible release

Obtain the Etlworks managed-cluster release bundle through your authorized Etlworks release channel. Use its immutable controller image reference and matching application, installer, charts and configuration contracts.

Available images are listed on Docker Hub's Tags tab. Use registry manifest digests supplied by the release; a local Docker image configuration ID is not a manifest digest. Do not substitute an arbitrary tag or mix controller/application releases without an approved compatibility plan.

The currently documented installation profile is local-nfs-v1. Hosted AWS, Azure and Google Cloud environments, alternative storage and infrastructure HA require their own supported profile. This image does not provision the Kubernetes cluster itself.

New matrix releases provide Linux AMD64 and ARM64 variants. Select the release ZIP matching the target Kubernetes nodes; the installation workstation may use a different architecture. Each ZIP pins platform-specific app/controller digests. Architecture-specific tags end in -amd64 or -arm64; an architecture-neutral candidate tag identifies both variants. Check the selected tag's actual platforms: older ARM64-only releases remain ARM64-only. Upgrades preserve the installation's architecture; publishing both variants does not permit a rolling architecture change.

⁠Install with the release client

The release bundle includes the installation client and its installation, operations and recovery guides. Installation requires an existing Kubernetes cluster, explicit kubeconfig/context, licensed environment inputs, supported storage and the selected profile's platform permissions.

Verify the archive against the SHA-256 supplied through a trusted release channel before extraction. Keep the extracted release unchanged and store protected installation inputs outside it. From that release directory:

python3 deployment/installer/release_archive.py verify --root .

sh deployment/installer/etlworks-kubernetes.sh preflight \
  --config /absolute/protected/install.json --json

sh deployment/installer/etlworks-kubernetes.sh install \
  --config /absolute/protected/install.json --wait 20m --json

sh deployment/installer/etlworks-kubernetes.sh status \
  --config /absolute/protected/install.json --json

Use the supplied configuration example and installation guide to prepare install.json. The installation client runs with Java 25 and uses kubectl, Helm 3 and Python; use the exact workstation prerequisites supplied by the release. The installer provisions or reuses compatible platform dependencies and deploys the controller with its application configuration, credentials and permissions.

This controller is not a standalone docker run application. It needs the installed Kubernetes resources, application lifecycle endpoints, signed capacity authorization, traffic integration and storage contracts.

⁠Runtime and credentials

The supplied configuration binds the namespace, cluster name, environment, member template, lifecycle endpoints, traffic handling and credential policy. Use the release-generated configuration instead of constructing a partial configuration from these examples.

The controller uses its own application identity; it does not need a human SuperAdmin login or the license server's private issuer signing keys. Kubernetes RBAC and application lifecycle authorization are separate permissions. The installed role grants the operations needed by that release; do not replace it with cluster-admin access.

The supported local installation provisions cert-manager and the component credentials/trust needed for its internal TLS connections. Mounted leaf credentials are renewed and reloaded through the installed credential workflow. Keep projected credential directories intact; replacing them with subPath mounts prevents the intended update behavior. Root trust changes are a separate operational procedure, not an ordinary leaf-certificate renewal.

The same image can supply additional credential projection/reload processes selected by the release packaging. Those processes have their own configured entrypoints; they do not become additional lifecycle leaders.

⁠How planned removal works

1. The adapter observes the native desired count, licensed bounds and current member state. Existing pending, creating and draining capacity stays accounted for; retries do not allocate an unlimited fleet.
2. It records an operation for the selected member and exact process/pod incarnation, closes new flow-execution admission and withdraws that member from the supported HAProxy traffic path.
3. Ordinary executions finish. Only healthy continuous executions with current relocation eligibility may be stopped and replaced. Scheduler ownership, replacement receipts and other application barriers must settle.
4. The adapter waits for actual execution and owned-work termination, application removal approval and traffic-quiescence evidence. It then requests removal of the approved workload and observes completion before retiring its capacity.

Readiness, a missing heartbeat, a canceled audit row or an elapsed timer is not removal approval. A blocked drain may keep actual member count above desired count. Status reports this difference and the blocking condition.

If load rises during drain, the adapter stops selecting further scale-in victims. Already closed admission follows the recorded lifecycle operation; it is not silently reopened. Reconciliation remains subject to licensed capacity and surge bounds.

Voluntary node maintenance must respect the supported disruption protections. Direct pod/workload deletion, force deletion and external hard shutdown deadlines can bypass those protections. They cannot be represented as successful planned drains. Do not delete state or bypass a hold to make the counts match.

⁠Monitor and troubleshoot

List pods and follow controller logs with the release client:

sh deployment/installer/etlworks-kubernetes.sh pods \
  --config /absolute/protected/install.json

sh deployment/installer/etlworks-kubernetes.sh logs \
  --config /absolute/protected/install.json --pod CONTROLLER_POD \
  --container controller --tail 200 --follow

sh deployment/installer/etlworks-kubernetes.sh diagnostics \
  --config /absolute/protected/install.json \
  --output /absolute/protected/support.zip

Direct kubectl access is also available with the appropriate Kubernetes permissions:

kubectl --kubeconfig /absolute/kubeconfig --context CONTEXT \
  -n NAMESPACE logs CONTROLLER_POD -c controller --tail=200 --follow

Inspect both controller replicas when investigating leadership or a blocked operation. A healthy standby is expected. A log-follow session remains attached to its selected pod; select a replacement pod explicitly after removal. Use platform log collection for retention beyond pod lifetime.

The controller exposes these internal HTTP endpoints on port 8081:

Keep these endpoints private. They are monitoring interfaces, not authenticated operator command endpoints or proof that a member is safe to delete. Missing, stale or unavailable metrics are not zero load and must not grant scale-in. Configure alert routing through your monitoring platform.

If reconciliation is held, inspect installation status, the recorded operation and controller logs. Restore the named dependency or use the release's documented recovery procedure. A controller restart resumes durable operations; it does not erase uncertainty about an accepted application command or replay the command as a new business execution.

⁠Updates and recovery

Application/embedded-engine upgrades use the managed upgrade UI or the release client's update commands. They do not automatically upgrade the controller, cert-manager, KEDA, storage or database-server software.

A controller image update is a platform operation. Follow the matching release's operations guide, verify controller/CRD/application protocol compatibility and use the approved immutable image reference and accompanying configuration/RBAC. Retain controller availability and one active Lease owner during the rollout. Credential sidecar images and trust/configuration must remain compatible too.

If a new controller cannot become healthy, preserve the existing compatible controller and recorded operations while investigating. Do not clear the Lease, operation resources, identity records or PVCs as a reset mechanism. Removing the controller stops lifecycle progress; it does not safely drain application members or remove their retained data.

Restore and cross-deployment migration require the release's explicit offline recovery/cutover workflow. Preserve storage, credentials and authority evidence; do not resume autoscaling or execution merely because restored pods are Running.

For help, use the guides included in the selected release and your existing Etlworks support channel. Share diagnostics through that protected channel and keep credentials, private keys and backup archives out of public issue reports.