pnda/gobblin repository overview

Docker Image Configuration Configuration is done through the following environment variables:

• HDFS_URL=hdfs://hdfs-namenode:8020
• MAX_MAPPERS=4
• KAFKA_BROKERS=kafka:9092
• MASTER_DATASET_DIRECTORY=/user/pnda/PNDA_datasets/datasets
• MASTER_DATASET_QUARANTINE_DIRECTORY=/user/pnda/PNDA_datasets/quarantine
• MASTER_DATASET_COMPACTION_DIRECTORY=/user/pnda/PNDA_datasets/compaction
• DATASET_COMPACTION_PATTERN=d