Sign inSign up

swaya1125/docker-autoheal

By swaya1125

•Updated 10 months ago

Image
0

10K+

swaya1125/docker-autoheal repository overview

⁠Docker Auto-Heal Service

Docker Pulls Docker Image Size Version

A production-ready Docker container monitoring and auto-healing service with a modern React web interface. Automatically monitors your Docker containers for failures and unhealthy states, restarting them intelligently based on configurable policies.

ā šŸš€ Quick Start

docker run -d \
  --name docker-autoheal \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -v ./data:/data \
  -p 3131:3131 \
  -p 9090:9090 \
  --restart unless-stopped \
  swaya1125/docker-autoheal:latest

Access the Web UI: http://localhost:3131⁠

ā šŸ“¦ What's Included

  • Python 3.11 backend with FastAPI
  • React 18 modern web interface with Vite
  • Automated health monitoring for all Docker containers
  • Smart restart logic with exponential backoff
  • Prometheus metrics endpoint on port 9090
  • Persistent storage in /data volume

⁠🌟 Key Features

⁠Core Monitoring
  • āœ… Monitor containers by label (autoheal=true) or all containers
  • āœ… React to Docker health checks and container exit codes
  • āœ… Configurable health check intervals and timeouts
  • āœ… Custom health checks (HTTP, TCP, Exec)
⁠Smart Restart Logic
  • āœ… Exponential backoff to prevent restart storms
  • āœ… Configurable cooldown periods between restarts
  • āœ… Maximum restart thresholds to prevent infinite loops
  • āœ… Automatic quarantine for containers that restart too frequently
  • āœ… Respect manual stops (exit code 0)
⁠Web Interface
  • āœ… Real-time dashboard with all container statuses
  • āœ… Per-container auto-heal enable/disable
  • āœ… Live event log with restart history
  • āœ… Full configuration management through UI
  • āœ… Config export/import as JSON
  • āœ… Maintenance mode support
⁠Enterprise Features
  • āœ… Persistent state across restarts (stored in /data)
  • āœ… Prometheus metrics for monitoring
  • āœ… Webhook alerts for critical events
  • āœ… Structured JSON logging
  • āœ… Configurable log levels (DEBUG, INFO, WARNING, ERROR)

ā šŸ“‹ Requirements

  • Docker Engine 20.10+
  • Docker socket access (/var/run/docker.sock)
  • Recommended: 2GB RAM, 1 CPU core

ā šŸ”§ Usage

⁠Docker Run (Basic)
docker run -d \
  --name docker-autoheal \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -p 3131:3131 \
  swaya1125/docker-autoheal:latest
⁠Docker Run (Full Options)
docker run -d \
  --name docker-autoheal \
  --restart unless-stopped \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -v /path/to/data:/data \
  -p 3131:3131 \
  -p 9090:9090 \
  -e AUTOHEAL_INTERVAL=30 \
  -e AUTOHEAL_LOG_LEVEL=INFO \
  swaya1125/docker-autoheal:latest
⁠Docker Compose
version: '3.8'

services:
  autoheal:
    image: swaya1125/docker-autoheal:latest
    container_name: docker-autoheal
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./data:/data  # Persist configuration and state
    ports:
      - "3131:3131"  # Web UI
      - "9090:9090"  # Prometheus metrics
    environment:
      - AUTOHEAL_INTERVAL=30
      - AUTOHEAL_LOG_LEVEL=INFO

  # Example monitored container
  webapp:
    image: nginx:latest
    labels:
      autoheal: "true"  # Enable auto-healing for this container
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost"]
      interval: 30s
      timeout: 10s
      retries: 3

Then start:

docker-compose up -d

ā šŸ·ļø Container Labels

Enable auto-healing on specific containers using labels:

services:
  myapp:
    image: myapp:latest
    labels:
      autoheal: "true"  # Enable monitoring
      autoheal.stop.timeout: "30"  # Custom stop timeout
⁠Available Labels
LabelDescriptionDefault
autohealEnable monitoring (true or false)Matches config
autoheal.stop.timeoutSeconds to wait before force-stopping10

ā āš™ļø Configuration

⁠Environment Variables
VariableDescriptionDefault
AUTOHEAL_INTERVALMonitoring interval in seconds30
AUTOHEAL_LOG_LEVELLog level (DEBUG, INFO, WARNING, ERROR)INFO
AUTOHEAL_LABEL_KEYLabel key to filter containersautoheal
AUTOHEAL_LABEL_VALUELabel value to filter containerstrue
⁠Web UI Configuration

All settings can be configured through the web interface at http://localhost:3131:

  • Monitor Settings: Interval, label filtering
  • Restart Policies: Cooldowns, max restarts, backoff strategies
  • Container Selection: Whitelist/blacklist containers
  • Alerts: Webhook configuration
  • Observability: Metrics and logging settings
⁠Config File

Configuration is automatically persisted to /data/config.json. You can:

  • Export config as JSON from the web UI
  • Edit the file directly
  • Import config from JSON backup

ā šŸ“Š Monitoring & Metrics

⁠Prometheus Metrics

Metrics are exposed on port 9090 at /metrics:

curl http://localhost:9090/metrics

Available metrics:

  • Container restart counts
  • Health check failures
  • Quarantine events
  • Processing times
⁠Health Check

Service health endpoint:

curl http://localhost:3131/health
⁠Logs

View logs:

docker logs -f docker-autoheal

Logs are also persisted to /data/logs/autoheal.log

ā šŸ” Troubleshooting

⁠Container Not Being Monitored
  1. Check if container has the autoheal=true label (if label filtering is enabled)
  2. Verify container is not in the exclusion list
  3. Check logs: docker logs docker-autoheal
  4. Enable DEBUG logging: Set AUTOHEAL_LOG_LEVEL=DEBUG
⁠Auto-Heal Service Won't Start
  1. Verify Docker socket is accessible:
    docker run --rm -v /var/run/docker.sock:/var/run/docker.sock alpine ls -l /var/run/docker.sock
    
  2. Check port availability (3131, 9090)
  3. Review logs for specific errors
⁠Container Quarantined

When a container restarts too frequently, it's automatically quarantined:

  1. View quarantined containers in the Web UI
  2. Investigate the root cause of failures
  3. Fix the underlying issue
  4. Unquarantine from the UI or API

ā šŸ” Security

⁠Docker Socket Access

This service requires read-only access to the Docker socket. While necessary for monitoring, this grants significant privileges. Best practices:

  • Use read-only socket mount: :ro
  • Run in isolated network segment
  • Limit access to the web UI (use reverse proxy with auth)
  • Review container logs regularly
⁠Production Deployment

For production:

  1. Use TLS for the web UI (reverse proxy recommended)
  2. Implement authentication (OAuth, basic auth via reverse proxy)
  3. Use Prometheus for metrics collection
  4. Set up alerting for quarantine events
  5. Regular config backups

ā šŸ—ļø Architecture

ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│  Docker Auto-Heal Container             │
│                                          │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”      ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”   │
│  │ React UI   │─────►│ FastAPI      │   │
│  │ (Port 3131)│      │ Backend      │   │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜      ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜   │
│                             │           │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā–¼ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”  │
│  │  Monitor Service                  │  │
│  │  - Health checks                  │  │
│  │  - Restart logic                  │  │
│  │  - Event logging                  │  │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜  │
│                     │                   │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”¼ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜
                      │
                      ā–¼
            /var/run/docker.sock
                      │
                      ā–¼
            ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
            │ Docker Engine   │
            │ (Host System)   │
            ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜

ā šŸ“š Additional Resources

ā šŸ“ License

MIT License - see LICENSE file for details

ā šŸ¤ Contributing

Contributions welcome! Please see the GitHub repository for guidelines.


Built with ā¤ļø using Python, FastAPI, React, and Docker

Tag summary

Content type

Image

Digest

sha256:f7e290c1f…

Size

75.6 MB

Last updated

10 months ago

docker pull swaya1125/docker-autoheal