danielsan05/conmonpusher repository overview

⁠danielsan05/conmonpusher

⁠Description

Imagine you are hosting some Docker containers to ease your digital daily life. You are interested in if something went wrong with your containers on your VPS or self-hosted Docker VM and they died or stopped unexpectedly. The Reason could be a failed Watchtower container upgrade or something else.

Prometheus with Grafana or Uptime Kuma is an overkill for your small home setup - here comes ConMonPusher into play.

ConMonPusher is monitoring your containers and is sending out a push message immediately (currently with Pushover and self-hosted Gotify server only) if a container stopped or died unexpectedly so that you can troubleshoot it. A Push message for recovery after fixing will also be sent out.

⁠Prerequisites

Following prerequisites are recommended:

• Using a Docker Socket Proxy container
• Having a registered Pushover account
• Added a custom application in Pushover Account for using it with ConMonPusher

⁠Environment variables

⁠TZ (default: Europe/Berlin)

Set-up your preferred time zone.

⁠DOCKER_HOST (default: tcp://socket-proxy:2375)

Sets the environment variable where the Docker Socket can be reached. Examples:

• DOCKER_HOST=/var/run/docker.sock
• DOCKER_HOST=tcp://192.168.59.103:2375
• DOCKER_HOST=tcp://localhost:2375

⁠HOST_LABEL

Sets a Hostname in the push message for the Docker Host where you are monitoring your running Docker containers. You can much easier differ on which Docker Host the current problematic container is running when you are monitoring multiple Docker Hosts with ConMonPusher.

⁠EVENTS (default: die,start,restart,stop,kill,health_status: unhealthy,health_status: healthy)

You can set "die", "stop", "restart", "kill", "health_status: unhealthy" or "health_status: healthy" for container status as push notification events.

⁠INCLUDE_STOP (default: false)

More granular control over the "stop" container event monitoring. Sometimes you might want to have notifications even for container stop events. Be careful in using it, it can produce unwanted false positives push notifications in your Docker container setup when you are running Watchtower for updating your container environment for example.

⁠TITLE_PREFIX (default: Docker alert)

Defines the title prefix of your push message.

⁠SOUND (default: pushover)

Defines the sound of your notification push message.

⁠PRIORITY (default: 0)

Defines the priority of your notification push message. You can use any value the pushover API documentation is referncing to.

⁠RETRY (default: -30)

Defines the retry parameter for the Pushover emergency priority notifications based on the pushover API documentation.

⁠EXPIRE (default: -120)

Defines the expiry parameter for the Pushover emergency priority notifications based on the pushover API documentation.

⁠ALERT_ON_SOCKET_FAILURE (default: true)

ConMonPusher will send out a push notification if it cannot connect to the Docker Socket Proxy via Docker Socket volume or via Socket Proxy container anymore.

⁠SOCKET_RETRY_SECONDS (default: -30)

Defines the retry parameter for the Pushover emergency priority notifications based on the pushover API documentation for Docker Socket connection errors. So you can differ in Docker Socket connection notification message and container event notification messages frequency if you want to.

⁠SOCKET_FAILURE_PRIORITY (default: -2)

Defines the priority of your notification push message. You can use any value the pushover API documentation is referncing to.

⁠NOTIFY_BACKEND (defalut: pushover)

Defines the backend over which you will receive the push notifications. You can choose between "pushover", "gotify" or "both".

⁠PUSHOVER_TOKEN

Defines the Pushover token of your registered account. Docker Secret support is enabled.

⁠PUSHOVER_USER

Defines the pushover application token of your created pushover application. Docker Secret support is enabled.

⁠PUSHOVER_USER

Defines the pushover application token of your created pushover application. Docker Secret support is enabled.

⁠GOTIFY_URL

Defines the URL for your self-hosted Gotify Server.

⁠GOTIFY_TOKEN or GOTIFY_TOKEN_FILE

Defines the Gotify application token under which application category the push notifications will be summarized on the Gotify Server. If you want to use Docker secrets management, please choose GOTIFY_TOKEN_FILE.

⁠Example docker-compose.yml

Here is an example of a default docker-compose.yml file using all possible environment variables of ConMonPusher container with Secrets and using connection against a Docker Socket proxy container for monitoring:

services:
  conmonpusher:
    image: danielsan05/conmonpusher:latest
    container_name: conmonpusher
    restart: unless-stopped
    labels:
      - traefik.enable=false
    depends_on:
      socket-proxy:
        condition: service_started
        restart: true
    environment:
      TZ: Europe/Berlin
      DOCKER_HOST: tcp://socket-proxy:2375
      PUSHOVER_TOKEN: /run/secrets/conmonpusher_pushover_token
      PUSHOVER_USER: /run/secrets/conmonpusher_pushover_user
      GOTIFY_TOKEN_FILE: /run/secrets/conmonpusher_gotify_token
      GOTIFY_URL: https://gotify.example.com
      NOTIFY_BACKEND: both
      HOST_LABEL: hostname.example.com
      EVENTS: "die,start,restart,stop,kill,health_status: unhealthy,health_status: healthy"
      INCLUDE_STOP: "false"
      TITLE_PREFIX: Docker alert
      SOUND: siren
      PRIORITY: "2"
      RETRY: "60"
      EXPIRE: "120"
      ALERT_ON_SOCKET_FAILURE: "true"
      SOCKET_RETRY_SECONDS: "60"
      SOCKET_FAILURE_PRIORITY: "2"
    secrets:
      - conmonpusher_pushover_token
      - conmonpusher_pushover_user
      - conmonpusher_gotify_token
    networks:
      socket_proxy:

  socket-proxy:
    image: lscr.io/linuxserver/socket-proxy:latest
    container_name: socket-proxy
    security_opt:
      - no-new-privileges:true
    restart: always
    labels:
      - traefik.enable=false
    networks:
      socket_proxy:
    privileged: true
    ports:
      - "2375:2375"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    read_only: true
    tmpfs:
      - /run
    environment:
      LOG_LEVEL: info
      EVENTS: 1
      PING: 1
      VERSION: 1
      INFO: 1
      CONTAINERS: 1
      POST: 0

secrets:
  conmonpusher_pushover_token:
    file: ./conmonpusher_pushover_token
  conmonpusher_pushover_user:
    file: ./conmonpusher_pushover_user

networks:
  socket_proxy:
    name: socket_proxy
    driver: bridge

Please note: Please modify the sample docker-compose.yml file to your needs and requirements on your own Docker Host.

Create both Docker Secret files and paste your appropriate Pushover Tokens into them.

If you are running Watchtower or Portainer for example in combination with a Docker Socket Proxy the Socket Proxy environment variables needs to be modified to their needs as well.

The file tree of the above mentioned sample ConMonPusher setup is configured as follows:

conmonpusher/
├── docker-compose.yml           # main configuration file
├── conmonpusher_gotify_token    # Docker Secret file for GOTIFY_TOKEN
├── conmonpusher_pushover_user   # Docker Secret file for PUSHOVER_USER
└── conmonpusher_pushover_token  # Docker Secret file for PUSHOVER_TOKEN

Finally run sudo docker-compose up -d to start the containers.