averemee/oracdc
Oracle Database CDC and Data Streaming using Kafka Connect
1.2K
You can dynamically specify configuration values with environment variables. You can use the Docker -e/--env or --env-file flags to specify various settings. You need to convert connect-distributed.properties file variables as described below and pass them as environment variables:
Standard Kafka Connect port 8083 is exposed, for storing secrets and oracdc memory mapped files mountpoint /var/lib/oracdc is present
A2_ORACDC_BOOTSTRAP_SERVERS A list of Kafka brokers to connect to
A2_ORACDC_GROUP_ID A unique string that identifies the Connect cluster group this Worker belongs to
A2_ORACDC_OFFSET_STORAGE_TOPIC The name of the existing topic where connector and task configuration offsets are stored. This must be the same for all Workers with the same A2_ORACDC_GROUP_ID (group.id). Always create it as a compacted, highly replicated (3x or more) topic with a large number of partitions (e.g., 25 or 50, just like Kafka’s built-in __consumer_offsets topic) to support large Kafka Connect clusters.
A2_ORACDC_CONFIG_STORAGE_TOPIC The name of the existing topic where connector and task configuration data are stored. This must be the same for all Workers with the same A2_ORACDC_GROUP_ID (group.id). Always create this topic as a compacted topic with a single partition and a high replication factor (3x or more).
A2_ORACDC_STATUS_STORAGE_TOPIC The name of the existing topic where connector and task configuration status updates are stored. This must be the same for all Workers with the same A2_ORACDC_GROUP_ID (group.id). Always create this topic as a compacted, highly replicated (3x or more) topic with multiple partitions
For complete example please read How to Set Up Oracle Database Tables Replication in Minutes
OR
Perform RDBMS configuration steps according to the Required Oracle database setup for oracdc Source connector
Full list of oracdc configuration parameters is available here
Create JSON config file (do not forget to replace connection details with your actual data) e.g.
{
"name" : "oracdc-test",
"config" : {
"connector.class" : "solutions.a2.cdc.oracle.OraCdcLogMinerConnector",
"tasks.max" : "1",
"a2.resiliency.type" : "fault-tolerant,
"a2.wallet.location" : "/var/lib/oracdc/admin/wallet",
"a2.jdbc.url" : "jdbc:oracle:thin:@VISION?TNS_ADMIN=/var/lib/oracdc/admin/network",
"a2.include" : "SCOTT.DEPT,SCOTT.EMP"
}
}
Create connector by invoking REST API at exposed port
curl -s -X POST -H 'Content-Type: application/json' --data @connector.json http://<CONTAINER_IP>:8083/connectors
docker pull averemee/oracdc