mintel/elasticsearch-cloudwatcher repository overview

⁠elasticsearch-asg

A number of little applications I needed setting up Elasticsearch on AWS Autoscaling Groups. These target Elasticsearch >= 7.0.

• cloudwatcher⁠ () - Push metrics about an Elasticsearch cluster to AWS CloudWatch, mainly to inform AWS Target Tracing Scaling Policies⁠.

• drainer⁠ () - Remove shards from Elasticsearch nodes on EC2 instances that are about to be terminated - either by an AWS AutoScaling Group downscaling or by Spot Instance interruption - by consuming CloudWatch Events⁠ from an SQS Queue⁠.

• throttler⁠ () - Regulate an AWS AutoScaling Group running Elasticsearch by preventing new autoscaling actions until the cluster is stable (not red, no relocating shards, etc).

• snapshooter⁠ () - Take snapshots of Elasticsearch cluster on a schedule, and clean up old ones with downsampling.

⁠Directory layout

• /cmd - A little bit of glue to run each application.
• /pkg - Packages that should theoretically be reusable in other projects.
• /internal - Code that isn't reusable in other projects.
  • /internal/app - The implementation of each application.
  • /internal/pkg - Packages used across multiple applications.

⁠How-To

⁠Start developing

You need a working Go >= 1.13 environment⁠. Then clone this repository:

mkdir -p $GOPATH/src/github.com/mintel/
cd $GOPATH/src/github.com/mintel/
git clone [email protected]:mintel/elasticsearch-asg.git
cd elasticsearch-asg

⁠Run tests

go test ./...

⁠Build binaries

for app in cloudwatcher drainer snapshooter throttler; do
    go build "./cmd/$app"
done

⁠Build Docker containers

for app in cloudwatcher drainer snapshooter throttler; do
    docker build . -f "Dockerfile.$app" -t "mintel/elasticsearch-$app:latest"
done

⁠Debug

Install Delve⁠.

To debug one of the apps:

dlv debug ./cmd/cloudwatcher -- --elasticsearch.url=http://localhost:9200

To debug tests in the current directory:

dlv test

See the Delve Getting Started⁠ guide for more details.

⁠Make a new release

If you tag a commit with a semantic version, Docker Hub will build it as a separate tag. For example:

git tag -a v1.2.3 -m "Release v1.2.3"
git push --follow-tags

Will build Docker images like mintel/elasticsearch-cloudwatcher:1.2.3.