Public | Automated Build

Last pushed: 4 months ago
Short Description
Base image with heavy dependencies for pacman
Full Description

Pacman aka DAE



Pacman is a synchronous tool using multiprocessing to crunch bundles of data from sources and flow them into destination.

Ex: Pacman gets Adwords Reports (Bundles) from the API (Source) and put each line into Kafka (Destination)



Typical scenarios


This project uses Ansible for deployment. Pacman is a schedule task launched thanks to an hourly cron.

Playbooks are within the deployment folder. Client specific configuration are in the conf folder inside the client folder :
To launch the right playbook a command would looks like this :

ansible-playbook -i deployment/inventory -e ansible_user=ubuntu -e env=prod -e client=orange --vault-password-file ~/.vault_pass.txt deployment/pacman.yml

The password required by ansible is the canonical Artefact R&D password.


  • docker
  • docker-compose
  • ask DAE team to access vault within internal VPN

Docker commands

  • To build pacman main container image: make docker_build
  • To run unit and functional tests: make docker_test

Validation tests

Validation tests are end to end tests that run pacman jobs interacting with external APIs and assert conditions on generated data
To run validation tests:

  1. create a json file with your vault credentials {"login": ..., "password": ...}
  2. export VAULT_PASSWORD_FILE=~/paht_to_your_creds_file
  3. python bin/

TODO sphynx documentation on existing connectors

Docker Pull Command
Source Repository