decryptus/monit-docker repository overview

⁠monit-docker project

monit-docker is a free and open-source, we develop it to monitor container status or resources and execute some commands inside containers or manage containers with dockerd, for example:

• reload php-fpm if memory usage is too high
• reload php-fpm if no free space in /dev/shm
• restart container if status is not running
• remove all containers

⁠Table of contents

1. Quickstart⁠
2. Installation⁠
3. Environment variables⁠
4. Sub-command: monit⁠
   1. Basic commands⁠
   2. Advanced commands⁠
   3. Container informations with exit codes⁠
   4. monit-docker with M/Monit⁠
5. Sub-command: stats⁠
   1. Basic commands⁠
   2. Advanced commands⁠

⁠Quickstart

Using monit-docker in Docker with crond

docker-compose up -d

See docker-compose.yml⁠ and MONIT_DOCKER_CRONS environment variable to configure commands.

⁠Installation

pip install monit-docker

⁠Environment variables

⁠Sub-command: monit

⁠Basic commands

Restart containers with name starts with foo if memory usage percentage > 60% or cpu usage percentage > 90%:

monit-docker --name 'foo*' monit --cmd-if 'mem_percent > 60 ? restart' --cmd-if 'cpu_percent > 90 ? restart'

Stop containers with name starts with bar or foo and if cpu usage percentage greater than 60% and less than 70%:

monit-docker --name 'bar*' --name 'foo*' monit --cmd-if '60 > cpu_percent < 70 ? stop'

Kill containers with name starts with bar and status equal to pause or running:

monit-docker --name 'bar*' monit --cmd-if 'status in (pause,running) ? kill'

You can also use status argument, for example, restart containers with status paused or exited:

monit-docker -s paused -s exited monit --cmd 'restart'

Generate containers pidfile:

monit-docker monit --rsc pid

Reload php-fpm in container with image name contains /php-fpm/ if memory usage greater than 100 MiB:

monit-docker --image '*/php-fpm/*' monit --cmd-if 'mem_usage > 100 MiB ? (kill -USR2 1)'

Reload php-fpm in container with image name contains /php-fpm/ if /dev/shm percentage usage greater than 80%:

monit-docker --image '*/php-fpm/*' monit --cmd '(bash -c "[ $(df /dev/shm | sed \"s/\%//;\$!d\" | awk \"{print \$5}\") -gt 80 ] && kill -USR2 1")'

⁠Advanced commands with configuration file or environment variable MONIT_DOCKER_CONFIG

⁠Run commands with aliases declared in configuration file (e.g.: monit-docker.yml.example⁠):

Restart container id 4c01db0b339c if condition alias @status_not_running is true:

monit-docker --id 4c01db0b339c monit --cmd-if '@status_not_running ? restart'

Execute commands alias @start_pause containers with name starts with foo if condition alias @status_not_running is true:

monit-docker --name 'foo*' monit --cmd-if '@status_not_running ? @start_pause'

Remove force container group php if status is equal to running:

monit-docker --ctn-group php monit --cmd-if 'status == running ? @remove_force'

Restart containers group nodejs if memory usage percentage > 10% and cpu usage percentage > 60%:

monit-docker --ctn-group nodejs monit --cmd-if '@mem_gt_10pct_and_cpu_gt_60pct ? restart'

Remove force all containers:

monit-docker monit --cmd '@remove_force'

⁠Container informations with exit codes

⁠Container status

Run command below to get status with exit code for container named foo_php_fpm:

monit-docker --name foo_php_fpm monit --rsc status

An error occurred if exit code is greater than 100.

⁠Container CPU usage percentage

Run command below to get CPU usage percentage with exit code for container named foo_php_fpm:

monit-docker --name foo_php_fpm monit --rsc cpu_percent

An error occurred if exit code is greater than 100.

⁠Container memory usage percentage

Run command below to get memory usage percentage with exit code for container named foo_php_fpm:

monit-docker --name foo_php_fpm monit --rsc mem_percent

An error occurred if exit code is greater than 100.

⁠monit-docker with M/Monit

We can also monitoring containers cpu_percent and mem_percent resources with M/Monit⁠.

⁠Configuration examples

check program docker.foo_php_fpm.status with path "/usr/bin/monit-docker --name foo_php_fpm monit --rsc status"
    group monit-docker
    if status = 114 for 2 cycles then alert # container not found
    if status != 0 for 2 cycles then exec "/usr/bin/monit-docker --name foo_php_fpm monit --cmd restart" # container not running

check program docker.foo_php_fpm.cpu with path "/usr/bin/monit-docker -s running --name foo_php_fpm monit --rsc cpu_percent"
    group monit-docker
    if status > 100 for 2 cycles then alert
    if status > 70 for 2 cycles then alert
    if status > 80 for 4 cycles then exec "/usr/bin/monit-docker --name foo_php_fpm monit --cmd reload"

check program docker.foo_php_fpm.mem with path "/usr/bin/monit-docker -s running --name foo_php_fpm monit --rsc mem_percent"
    group monit-docker
    if status > 100 for 2 cycles then alert
    if status > 70 for 2 cycles then alert
    if status > 80 for 4 cycles then exec "/usr/bin/monit-docker --name foo_php_fpm monit --cmd '(kill -USR2 1)'"

check program docker.foo_php_fpm.pid with pidfile /run/monit-docker/foo_php_fpm.pid
    group monit-docker
    if changed pid then alert

⁠Sub-command: stats

⁠Basic commands

Get all resources statistics for all containers in json format:

monit-docker stats --output json

{
  "flamboyant_chaplygin": {
    "status": "running",
    "mem_percent": 0.03,
    "net_tx": "0.0 B",
    "cpu_percent": 0,
    "mem_usage": "2.52 MiB",
    "io_read": "3.5 MB",
    "io_write": "0.0 B",
    "net_rx": "25.2 kB",
    "mem_limit": "7.27 GiB",
    "pid": "3943"
  }
}
{
  "practical_proskuriakova": {
    "status": "running",
    "mem_percent": 0.04,
    "net_tx": "0.0 B",
    "cpu_percent": 0,
    "mem_usage": "2.61 MiB",
    "io_read": "24.6 kB",
    "io_write": "0.0 B",
    "net_rx": "25.0 kB",
    "mem_limit": "7.27 GiB",
    "pid": "3990"
  }
}

Get all resources statistics for all containers in text format:

monit-docker stats --output text

flamboyant_chaplygin|mem_usage:2.52 MiB|mem_limit:7.27 GiB|mem_percent:0.03|cpu_percent:0.0|io_read:3.5 MB|io_write:0.0 B|net_tx:0.0 B|net_rx:43.5 kB|status:running
practical_proskuriakova|mem_usage:2.61 MiB|mem_limit:7.27 GiB|mem_percent:0.04|cpu_percent:0.0|io_read:24.6 kB|io_write:0.0 B|net_tx:0.0 B|net_rx:43.3 kB|status:running

⁠Advanced commands with configuration file or environment variable MONIT_DOCKER_CONFIG

Get status and memory usage for group nodejs:

monit-docker --ctn-group nodejs stats --rsc status --rsc mem_usage