etlworks/etlworks-managed repository overview

⁠Etlworks Managed Application

Etlworks application and embedded ETL engine for managed, elastic Kubernetes clusters.

Image: etlworks/etlworks-managed

Each application member runs the full Etlworks application and embedded engine. The matching Etlworks lifecycle adapter coordinates safe member creation, draining and upgrades. KEDA and Kubernetes HPA determine desired application capacity from the configured policy and metrics.

⁠Choose the matching release

Install using the Etlworks managed-cluster release bundle supplied through your authorized Etlworks release channel. The bundle includes the installation client, charts, platform assets, release descriptor and operating instructions. The application, controller and configuration contracts must belong to a compatible release.

Use the immutable image references in the bundle's release.json. Available tags are listed on Docker Hub's Tags tab; architecture and supported infrastructure are specified by the selected release. A local Docker image configuration ID is not a registry manifest digest.

The currently documented installation profile is local-nfs-v1, using an existing Kubernetes cluster with the release's pinned networking, storage and autoscaling components. Hosted AWS, Azure and Google Cloud profiles, infrastructure HA and alternative storage need their own supported release profile. Successful installation of the local profile does not establish production qualification for those other deployments.

New matrix releases provide Linux AMD64 and ARM64 variants. Select the release ZIP matching the target Kubernetes nodes; the installation workstation may use a different architecture. Each ZIP pins platform-specific app/controller digests. Architecture-specific tags end in -amd64 or -arm64; an architecture-neutral candidate tag identifies both variants. Check the selected tag's actual platforms: older ARM64-only releases remain ARM64-only. Upgrades preserve the installation's architecture; publishing both variants does not permit a rolling architecture change.

This image contains the versioned application/engine, Java 25 and Tomcat runtime. Installation uses Kubernetes and the supplied client. The image is not a native host installer and does not download updated WARs during application startup.

⁠Prerequisites

Prepare:

• An existing Kubernetes cluster and a kubeconfig with an explicit context.
• A supported installation workstation with Java 25, Python 3.12 or later, kubectl, Helm 3 and archive-extraction tools.
• Platform installation permissions for the selected profile's CRDs, networking, storage, cert-manager and KEDA components. Runtime service accounts receive narrower permissions.
• Sufficient worker CPU/RAM and retained storage for the signed starting fleet, databases, shared application data and platform components.
• A signed environment-level Cluster installation bundle, separate execution authorization and independently trusted issuer public keys from Etlworks.
• A reachable application hostname, the intended frontend TLS/DNS arrangement, and an initial administrator email/password.
• Registry access for the exact image digests in the release. Private repositories additionally require the configured Kubernetes image-pull credentials.

The local profile can provision PostgreSQL, Redis, NFS and HAProxy through the installer. Use its documented worker/storage bindings. External infrastructure choices are supported only when included in your selected release profile. Machine provisioning and node-pool limits are platform-operator responsibilities.

⁠1. Verify and extract the release

Obtain the release archive and its expected SHA-256 through a trusted Etlworks channel. Substitute the actual archive name for RELEASE.zip below.

On Linux:

sha256sum RELEASE.zip

On macOS:

shasum -a 256 RELEASE.zip

Compare the result with the independently supplied expected hash before extracting. Then extract into a new directory:

mkdir etlworks-release
unzip RELEASE.zip -d etlworks-release
cd etlworks-release
python3 deployment/installer/release_archive.py verify --root .

Keep this directory unchanged. Its inventory rejects missing, changed and extra files. Store installation inputs, secrets, logs and backups outside it, and retain the original archive for recovery.

⁠2. Prepare installation inputs

Create a protected sibling directory and copy the supplied configuration example:

mkdir -p ../etlworks-private
chmod 700 ../etlworks-private
cp -n deployment/installer/example.json ../etlworks-private/install.json
chmod 600 ../etlworks-private/install.json

Edit install.json for your environment. Use absolute paths for the release and protected input files. The important fields are:

The license supplies starting member count and CPU/RAM profile. The installer validates those signed values; installation JSON does not author sizing. Subsequent operational policy stays within the licensed limits.

Private issuer signing keys stay on the license server. Do not place private keys, passwords or bearer tokens in the configuration JSON, image, public repository or release directory. Refer to protected files/Secrets as specified by the installer. The initial password must satisfy the application's existing password policy.

⁠3. Install and inspect progress

From the extracted release directory:

sh deployment/installer/etlworks-kubernetes.sh preflight \
  --config ../etlworks-private/install.json --json

sh deployment/installer/etlworks-kubernetes.sh install \
  --config ../etlworks-private/install.json --wait 20m --json

sh deployment/installer/etlworks-kubernetes.sh status \
  --config ../etlworks-private/install.json --json

The installer uses the specified context; it does not switch the workstation's active context. Compatible platform prerequisites are reused. The installer does not silently upgrade an existing shared Kubernetes platform component.

If the client is interrupted or its wait ends while work is pending, inspect status and resume the same installation:

sh deployment/installer/etlworks-kubernetes.sh resume \
  --config ../etlworks-private/install.json --wait 20m --json

A client timeout is not permission to terminate application work or start another installation. Preserve the configuration, release, namespace, operation receipts, credentials and storage. A blocked result includes the condition to investigate. There is no timeout-based force deletion or automatic business-flow replay.

The native Etlworks CLI can delegate to this client when ETLWORKS_MANAGED_HOME points to the extracted release. A PowerShell launcher, deployment/installer/etlworks-kubernetes.ps1, is also supplied; use it only with a release supporting your installation workstation/platform combination.

⁠4. Access Etlworks

Use the configured HTTPS hostname and initial administrator credentials once installation reports the admitting fleet ready. DNS, client reachability and frontend certificate trust must match that hostname. Installation does not configure workstation DNS automatically.

The local profile uses an internal frontend CA. For local access, an operator can forward the proxy Service, using the actual context, namespace and cluster name:

kubectl --kubeconfig /absolute/kubeconfig --context CONTEXT \
  -n NAMESPACE port-forward service/CLUSTER_NAME-proxy 8443:443

Map the configured hostname to loopback for this access path, trust the approved frontend CA and open https://CONFIGURED_HOSTNAME:8443/. Port forwarding does not provide public ingress or DNS. Keep TLS hostname/certificate verification enabled.

⁠Capacity and scaling

Etlworks operational policy controls minimum/warm capacity, load thresholds and stabilization within the signed entitlement. CPU, container RAM and JVM heap observations have distinct meanings; RAM includes more than JVM heap.

KEDA/HPA recommends desired application replicas. The lifecycle adapter creates members and controls planned removal. Initializing a maximum of two members does not itself request two starting members or guarantee one member on each machine; the signed starting configuration and Kubernetes placement rules determine that.

Before planned removal, new execution admission closes and traffic is withdrawn. Ordinary executions finish. Healthy continuous executions can be stopped and replaced only when their current lifecycle eligibility allows it. Removal waits for actual execution/owned-work termination and required handoff evidence. A blocked drain can therefore keep actual capacity above desired capacity.

A larger pod fleet can receive new work; it cannot divide an existing indivisible execution. Kubernetes process replacement after a crash does not authorize restarting its business executions. Node autoscaling and voluntary machine maintenance must use a supported integration respecting these drain constraints.

⁠Logs and diagnostics

List pods and follow a specific app member's console/Tomcat output:

sh deployment/installer/etlworks-kubernetes.sh pods \
  --config ../etlworks-private/install.json

sh deployment/installer/etlworks-kubernetes.sh logs \
  --config ../etlworks-private/install.json --pod APP_POD --tail 200 --follow

sh deployment/installer/etlworks-kubernetes.sh logs \
  --config ../etlworks-private/install.json --pod APP_POD --previous

sh deployment/installer/etlworks-kubernetes.sh diagnostics \
  --config ../etlworks-private/install.json --output ../etlworks-private/support.zip

The app container is named app-engine. Use --container controller for a controller pod. Direct kubectl get pods, describe and logs remain available. A log-follow session stays with the selected pod; select its replacement explicitly. Retention after pod deletion requires your platform's logging collection.

Diagnostics are read-only and produce a bounded status bundle. Send support files through your authorized support channel; keep credentials and private keys private.

⁠Upgrades

Use the managed upgrade UI as an Etlworks SuperAdmin, or the release client's update commands with the required Kubernetes permissions. The backend/UI and CLI use the managed upgrade protocol; UI actions do not invoke a host CLI.

The platform operator must first register an approved source/target release pair and immutable image references. Inspect the plan before starting:

sh deployment/installer/etlworks-kubernetes.sh update plan \
  --config ../etlworks-private/install.json \
  --source SOURCE_MANIFEST_DIGEST --target TARGET_MANIFEST_DIGEST --json

sh deployment/installer/etlworks-kubernetes.sh update start \
  --config ../etlworks-private/install.json --request REQUEST_UUID \
  --source SOURCE_MANIFEST_DIGEST --target TARGET_MANIFEST_DIGEST \
  --plan-digest PLAN_DIGEST_FROM_PLAN --json

sh deployment/installer/etlworks-kubernetes.sh update status \
  --config ../etlworks-private/install.json --request REQUEST_UUID --json

Use the identifiers supplied by the registered release pair and retain the request UUID for retries. update pause and update resume operate on that same request. Follow the shipped installer/operations guide for their arguments and any recovery command shown by status.

The approved compatibility plan determines rolling versus maintenance behavior. App startup owns database migrations under the managed upgrade controls. Image rollback alone does not undo a schema migration. Controller/platform/database-server upgrades are separate operations; app upgrades do not silently perform them.

⁠Backup and recovery

Use the matching installed release and its shipped recovery guide. Backup/restore has offline preparation requirements: settle executions and maintenance operations, stop the relevant reconciliation/autoscaling writers and preserve exact storage and credential identities before starting the supported offline procedure.

A checkpoint includes PostgreSQL, the dedicated Redis database's logical contents, shared application data and protected installation credentials. Keep backups in a separate protected location. Inspect an archive before restoring:

sh deployment/installer/etlworks-kubernetes.sh restore inspect \
  --config ../etlworks-private/install.json \
  --archive /absolute/backups/checkpoint.zip --json

restore start replaces selected data. A successful restore remains held for explicit authority reconciliation/cutover; it does not automatically restart business executions. Regular-to-managed or managed-to-regular migration needs the release's explicit migration workflow and source isolation, not an ordinary in-place restore or per-pod image change.

⁠Container and data contract

The installer supplies mounts, identities, Secrets, JVM arguments and resource limits. JAVA_OPTS, CATALINA_OPTS and JDK_JAVA_OPTIONS overrides are rejected by this image's startup contract. Select supported sizing through licensed profiles and managed policy. Removing a pod does not remove its retained data. Do not delete PVCs, member identities or coordination state to clear a hold.

For installation-specific help, use the installer, operations and recovery guides included in the same release and contact Etlworks support through your existing support channel.