Prometheus remote storage adapter for Elasticsearch
100K+
Please note: This project is no longer under active development.
A read and write adapter for prometheus persistent storage.
| Port | Path | Description |
|---|---|---|
| 8000 | /read | Prometheus remote read endpoint |
| 8000 | /write | Prometheus remote write endpoint |
| 9000 | /metrics | Surface Prometheus metrics |
| 9000 | /live | Http probe endpoint to reflect service liveness |
| 9000 | /ready | Http probe endpoint reflecting the connection to and state of the Elasticsearch cluster |
| Env Variables | Default | Description |
|---|---|---|
| ES_URL | http://localhost:9200 | Elasticsearch URL |
| ES_USER | Elasticsearch User | |
| ES_PASSWORD | Elasticsearch User Password | |
| ES_WORKERS | 1 | Number of batch workers |
| ES_BATCH_MAX_AGE | 10 | Max period in seconds between bulk Elasticsearch insert operations |
| ES_BATCH_MAX_DOCS | 1000 | Max items for bulk Elasticsearch insert operation |
| ES_BATCH_MAX_SIZE | 4096 | Max size in bytes for bulk Elasticsearch insert operation |
| ES_ALIAS | prom-metrics | Elasticsearch alias pointing to active write index |
| ES_INDEX_DAILY | false | Create daily indexes and disable index rollover |
| ES_INDEX_SHARDS | 5 | Number of Elasticsearch shards to create per index |
| ES_INDEX_REPLICAS | 1 | Number of Elasticsearch replicas to create per index |
| ES_INDEX_MAX_AGE | 7d | Max age of Elasticsearch index before rollover |
| ES_INDEX_MAX_DOCS | 1000000 | Max number of docs in Elasticsearch index before rollover |
| ES_INDEX_MAX_SIZE | Max size of index before rollover eg 5gb | |
| ES_SEARCH_MAX_DOCS | 1000 | Max number of docs returned for Elasticsearch search operation |
| ES_SNIFF | false | Enable Elasticsearch sniffing |
| STATS | true | Expose Prometheus metrics endpoint |
| DEBUG | false | Display extra debug logs |
Although prometheus-es-adapter will create and rollover Elasticsearch indicies it is expected that a tool such as Elasticsearch Curator will be used to maintain quiescent indicies eg deleting, shrinking and merging old indexes.
Automated builds of Docker image are available at https://hub.docker.com/r/pwillie/prometheus-es-adapter/.
Local development requires Go to be installed. On OS X with Homebrew you can just run brew install go.
Running it then should be as simple as:
$ make build
$ ./bin/prometheus-es-adapter
make test
To run end to end tests using docker-compose, from the "test" directory:
docker-compose up -d
docker-compose ps
docker-compose up -d --build prometheus-es-adapter
docker-compose logs -f prometheus-es-adapter
Content type
Image
Digest
Size
10.4 MB
Last updated
over 6 years ago
docker pull pwillie/prometheus-es-adapter