lastcallmedia/metrics-sync repository overview

⁠Metrics Sync

This tool is designed to consume aggregate metrics from several different cloud providers into a single Elasticsearch instance for graphing and measurement.

⁠Configuration

This tool is configured through a sync.yml file that lives in the working directory. See sync.yml.example⁠ for more information.

⁠Running

To invoke a sync, run bin/metricsync. The tool will go out and fetch the latest metrics, and push them into Elasticsearch. It doesn't particularly matter how often you run the sync - all sources generate an ID that will prevent the records from being duplicated in Elasticsearch.

Alternatively, you can run this application with Docker:

docker run -v ./sync.yml:/app/sync.yml lastcallmedia/metrics-sync 

⁠Sources

⁠CircleCI

The CircleCI source consumes job information from the CircleCI APIs. Specifically, it requests the last N builds and pushes them into an Elasticsearch index.

⁠Acquia

The Acquia source consumes some of the Stack Metrics data that Acquia's APIs provide. It aggregates them into hourly chunks, and pushes them into an Elasticsearch index.

⁠New Relic

The New Relic source consumes hourly aggregates for the HTTPDispatcher metrics on New Relic.

⁠Credits

This project is an independent effort of Last Call Media, but it was born out of a high-level monitoring need discovered while working on the Mass.gov project.