quboleinc/streamx repository overview

StreamX is a kafka-connect based connector to copy data from Kafka to Object Stores like Amazon s3, Google Cloud Storage and Azure Blob Store. It focusses on reliable and scalable data copying. It can write the data out in different formats (like parquet, so that it can readily be used by analytical tools) and also in different partitioning requirements.

Usage : docker run -d --net=host --env CONNECT_BOOTSTRAP_SERVERS=localhost:29092 --env CONNECT_AWS_SECRET_KEY=yyyy --env CONNECT_AWS_ACCESS_KEY=xxxx --env CONNECT_GROUP_ID=connect-cluster-2 --env CONNECT_CONFIG_STORAGE_TOPIC=connect-cluster-2 --env CONNECT_REST_PORT=8085 quboleinc/streamx