Public | Automated Build

Last pushed: 21 days ago
Short Description
Base image with heavy dependencies for pacman
Full Description

Pacman aka DAE

Master:

Develop:

Pacman is a synchronous tool using multiprocessing to crunch bundles of data from sources and flow them into destination.

Ex: Pacman gets Adwords Reports (Bundles) from the API (Source) and put each line into Kafka (Destination)

Concepts

Architecture

Typical scenarios

Deployment

This project uses Ansible for deployment. Pacman is a schedule task launched thanks to an hourly cron.

Playbooks are within the deployment folder. Client specific configuration are in the conf folder inside the client folder :
To launch the right playbook a command would looks like this :

ansible-playbook -i deployment/inventory -e ansible_user=ubuntu -e env=prod -e client=orange --vault-password-file ~/.vault_pass.txt deployment/pacman.yml

The password required by ansible is the canonical Artefact R&D password.

Prerequisites

  • docker
  • docker-compose
  • ask DAE team to access vault within internal VPN

Docker commands

  • To build pacman main container image: make docker_build
  • To run unit and functional tests: make docker_test

TODO sphynx documentation on existing connectors

Docker Pull Command
Owner
artefact
Source Repository