ampathke/etl-services repository overview

⁠ETL REST Server

This is a node project that uses hapi⁠ to expose REST endpoints which provide access to data hosted in ETL⁠ flat tables. These flat tables are flattened derived database tables containing data from OpenMRS. Data generation is done via MySQL stored procedures (found here⁠). This project is currently being battle-tested in production by AMPATH POC⁠.

To setup the project, run:

$ git clone https://github.com/AMPATH/etl-rest-server.git

$ cd etl-rest-server

$ npm install

$ mkdir conf && cd conf

Create a config.json file

$ cat config.json

With the following content

{
  "openmrs": {
    "host": "The IP or hostname of the OpenMRS server",
    "port": 8080
  },
  "etl": {
    "host": "The IP or hostname of the server running etl-rest-server",
    "port": 8002,
    "key": "path to the private key for tls",
    "cert": "path to the public key for tls",
    "tls": true
  },
  "mysql": {
    "connectionLimit": 10,
    "host": "The IP or hostname of the  mysql server",
    "port": "3306",
    "user": "<mysql user>",
    "password": "<mysql password>",
    "multipleStatements": true
  }
}

You can set tls:false if you don't care about https and don't provide the keys but if you set it to true you have to provide the keys.

npm start

Now visit https://<Your Host>:<Your Port> You should see the welcome message

Welcome to ETL reset server for OpenMRS

⁠Using Docker Compose 1.6+

docker-compose up -d

Confirm by looking for the server at host port 8002 using TLS:

curl -k https://docker:8007

⁠Without Docker Compose

⁠Building

docker build -t etl .

⁠Running

⁠Using Docker for MySQL

docker run -d --name mysql4etl -e MYSQL_ROOT_PASSWORD=supersecret \
  -e MYSQL_USER=etl_user -e MYSQL_PASSWORD=etl_password mysql

⁠Running with openmrs instance running at localhost on port 8080

docker run -d --name etl --link mysql4etl:db -p 8002:8002 etl

⁠Running with openmrs running at 1.2.3.4 on port 8081 under app name "omrs"

docker run -d --name etl --link mysql4etl:db -p 8002:8002 \
  -e OPENMRS_HOST_ADDR=1.2.3.4 -e OPENMRS_NAME=omrs -e OPENMRS_PORT=8081 etl

⁠Using your own MySQL server

If you have MySQL running at 1.2.3.4 with username "myuser" and password "mypassword": (This assumes openmrs at localhost port 8080, to use different settings pass environmental variables as above)

docker run -d --name etl \
  -e DB_PORT_3306_TCP_ADDR=1.2.3.4 -e DB_PORT_3306_TCP_PORT=3306 \
  -e MYSQL_USER=myuser -e MYSQL_PASSWORD=mypassword \
  -p 8002:8002 etl

⁠Using custom SSL certificate

docker run -d -name etl -v /path/to/keys/:/keys -p 8002:8002 etl

The folder /path/to/keys/ should contain SSL certificate and private key in server.crt and server.key.

⁠Setup CI Integration

1. Request addition of your dockerhub account to the ampathke docker team.
2. Activate the etl-rest-server repo on your Travis account.
3. Setup DOCKER_PASSWORD & DOCKER_USERNAME under environment variables on your forked repo's settings page on Travis.
4. Update your local etl-rest-server fork i.e. git pull upstream master. The CD setup should be in master.
5. Checkout a branch from master e.g. git checkout -b test-etl
6. Push that branch i.e. git push origin <branchname>
7. Access your test build via the following url: https://ngx.ampath.or.ke/etl-backend/branchname/etl⁠ where branchname is the name of your branch.