randysimpson/ingestor repository overview

⁠Ingestor

This project was created to allow for the ingestion of metrics into a mongodb. The metrics will be queued up when there are too many, but there is no local storage. The purpose of this is that when there is a large amount of metrics incoming, there could be more containers added to handle the workload.

When this is running you can post metrics to port 3000 and endpoint /api/v1/data and the metrics will be queued and stored into the mongodb.

Read more about the ingestor at https://github.com/randysimpson/ingestor⁠.

⁠Prerequisites

There must be a running mongodb instance running that is accessible on the network. If you want to setup the mongodb as a deployment on Kubernetes environment, please follow the blog on Setup MongoDB on Kubernetes⁠

⁠Docker

To run the ingestor as a container with exposed port 3000, then issue the following command:

docker run --name ingestor -e "DB_URL=mongodb://mongosvc:27017" -e "DB_NAME=metrics" -d -p 3000:3000 randysimpson/ingestor:latest

⁠Kubernetes

Creating a deployment for the ingestor if you are using the DB_URL of mongodb://mongosvc:27017 and DB_NAME of metrics:

kubectl create -f https://raw.githubusercontent.com/randysimpson/ingestor/master/ingestor.yaml --edit

The yaml file includes a deployment as well as a service and can be found at https://raw.githubusercontent.com/randysimpson/ingestor/master/ingestor.yaml⁠.

⁠Scale Deployment

To scale the deployment you will only need to adjust the number of ingestors:

kubectl scale deployment.v1.apps/ingestor --replicas=3