n8500x/kafka-connect-oss repository overview

⁠kafka-connect-oss

A 100% open-source Kafka Connect image built on the official Apache Kafka distribution. No licence key, no trial period, no runtime licence validation, no proprietary or copyleft-heavy drivers. Every jar in plugin.path is audited on every build.

Tags: 4.3.1-<date>-<time> = Apache Kafka 4.3.1 + build timestamp. Pull the newest dated tag; there is no latest.

docker pull n8500x/kafka-connect-oss:4.3.1-20260917-155426

⁠What's inside

JDBC drivers: PostgreSQL (BSD), MariaDB Connector/J (LGPL-2.1, also serves MySQL), mssql-jdbc (MIT), Snowflake and SQLite (Apache-2.0). The Oracle ojdbc8, GPL mysql-connector-j and jtds jars that the Aiven package bundles are stripped at build time. No TIBCO, no Elasticsearch, no Confluent components.

Redis Streams is pinned to connector 0.9.1 on purpose: 1.0.0+ ships a Business Source Licence dependency.

⁠Run

services:
  kafka-connect:
    image: n8500x/kafka-connect-oss:4.3.1-20260917-155426
    ports: ["8083:8083"]
    environment:
      CONNECT_BOOTSTRAP_SERVERS: kafka:29092
      CONNECT_GROUP_ID: oss-connect
      CONNECT_CONFIG_STORAGE_TOPIC: _oss-connect-configs
      CONNECT_OFFSET_STORAGE_TOPIC: _oss-connect-offsets
      CONNECT_STATUS_STORAGE_TOPIC: _oss-connect-status
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1

CONNECT_FOO_BAR=x becomes foo.bar=x, the same convention as the Confluent images. REST port: CONNECT_PORT=<port> (or CONNECT_LISTENERS=http://0.0.0.0:<port>; Kafka 4.x has no rest.port); the healthcheck and self-test follow it. A static self-test (plugins, drivers, licence guarantees) runs at every start unless SKIP_SELF_TEST=true.

⁠Redis Streams

Source: com.redis.kafka.connect.RedisStreamSourceConnector (XREADGROUP, at-least-once). Sink: com.redis.kafka.connect.RedisSinkConnector with redis.command=XADD. Add the bundled SMT to publish a stream entry's own fields as the Kafka key and value:

"transforms": "streamToKv",
"transforms.streamToKv.type": "io.github.n8500x.connect.transforms.RedisStreamToKeyValue",
"transforms.streamToKv.keyField": "key",
"transforms.streamToKv.valueField": "value"

Configs that already name the transform MapRedisStreamRecord keep working: the same transform is also registered as io.github.n8500x.connect.transforms.MapRedisStreamRecord (identical behaviour and properties), so only the package prefix of the type line changes.

⁠Verify it on-prem (65 checks, no internet needed)

Every supported connector moves a real record through its backend in the test (Redis SET, JDBC sink and source against PostgreSQL, OpenSearch, MongoDB sink and change-stream source, an ActiveMQ round trip, and Redis Streams in both directions).

Works with Podman + podman-compose (CONTAINER_CLI=podman COMPOSE_CMD=podman-compose) or Docker. Missing images are pulled by the script up front (--pull-only prepares a host); compose never pulls.

Everything required is inside the image at /usr/share/doc/kafka-connect-oss/: the test harness, the compose stack definition, licence inventory, Dockerfile and SMT sources.

docker run --rm n8500x/kafka-connect-oss:4.3.1-20260917-155426 \
  tar -C /usr/share/doc/kafka-connect-oss -cf - integration-test.sh docker-compose.yml | tar -xf -
chmod +x integration-test.sh
CONNECT_IMAGE=n8500x/kafka-connect-oss:4.3.1-20260917-155426 CONNECT_PORT=9083 ./integration-test.sh --skip-build   # CONNECT_PORT optional, default 8083
# expected: Summary   PASS: 65   FAIL: 0

The suite starts Kafka, Redis, PostgreSQL, OpenSearch, MongoDB and ActiveMQ (pinned open-source images; set REGISTRY_PREFIX= to pull them from an internal registry), registers every connector and requires RUNNING tasks, runs Redis Streams end-to-end in both directions, then scans every plugin jar for restricted licence text and licence-enforcement classes. Full details in README.md and ONPREM-REFERENCE.html inside the image.

⁠Retired predecessor

n8500x/cp-kafka-connect is retired. Its 8.2.0-ubi10-* tags were a Confluent Platform build (Confluent Community runtime, commercial ActiveMQ/TIBCO connectors, proprietary Oracle driver); its 4.3.1-oss-* tags were earlier builds of this image. Use this repository for all new deployments.