mirzamerdovic/alembic repository overview

⁠Alembic:alembic:

⁠What is it?

Alembic :alembic: is a simple .net core (3.1) app that monitors the health status of your Docker containers.

⁠Motivation

I needed a small utility app that I could use for local development and potentially for small systems where I would like to have as few 3rd party tools involved in my infrastructure setup as possible. So, if you are using Docker Swarm, Kubernetes or similar you are not going to need this, but if you are working with only Docker Compose or just Docker as far as it is now there is no out of the box way to have your containers restarted or killed in case they become unhealthy which is not really an optimal solution.

:alembic: tries to solve this problem by running a simple application written in C# running on .Net Core. The application is listening for containers' "health_status" events and act if the status is unhealthy.

You should also be aware of docker-autoheal⁠ project which is tackling the same issue.

⁠How it works

Docker is transmitting events about almost everything that's going on with your containers, :alembic: is interested in "health_status" container events and once it receives one it will check if the received status is "healthy" or "unhealthy" and act accordingly.

By default :alembic: is going to try and restart container 3 times and upon 3 consecutive tries if the container is still reported as unhealthy :alembic: is going to kill the container. Both actions: restart and kill are going to be reported via webhook. For starters only Slack webhooks will be supported.

⁠How to use :alembic:

Add this configuration in your docker-compose file:

alembic:
    image: mirzamerdovic/alembic
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

If you want to change the default values you can bind your own appsettings.json (you can name what ever you want) with override values

volumes:
  - ${path_to_appsettings_override}:/app/appsettings.json:ro

If you only need to update the webhook URL then you can just add environment variable: WebHookReporterOptions__Url=${my_url} to your docker-compose

environment:
  - WebHookReporterOptions__Url=services/{my_webhook}

⁠Configuration

There are a couple of things that can be configured when using :alembic:

• RestartCount: Tells :alembic: how many time it will try to restart an unhealthy container consecutively
• Kill: Tells :alembic: whether it should kill containers that cannot auto-heal. In other words tells :alembic: what to do once the maximum number of retries have been reached. With true meaning kill the container and with false meaning leave the container as it is.
• ReportsWebHook: A webhook to where :alembic: will send the notifications about restart and/or kill actions

⁠Known Issues

The appsettings.json hot reaload is not working. Which means if you change a value in your bound override of appesttings.json it will not be applied until you restart the container. The hot reload works if you are running from VS or just executing the dll, but when it is in container nothing is happening. I saw there are issues with binding to a file and hot-reload, and there was a suggestion to use folder bind, I tried that too, with the same result. Maybe I will get back to this, or maybe someone else will figure out what's the problem.