bionicburn/rec-worker-updater repository overview

⁠Red Eyed Crows Worker Updater

Automated worker deployment and lifecycle management system that monitors Git repositories, builds packages, and manages worker processes.

⁠Quick Start

docker run -d \
  --name rec-worker-updater \
  -p 3003:3003 \
  -v /path/to/config:/app/config \
  -v /path/to/repos:/app/repos \
  -v /path/to/logs:/app/logs \
  -v /path/to/secrets:/run/secrets:ro \
  -v /path/to/ssh:/root/.ssh:ro \
  -v /var/run/docker.sock:/var/run/docker.sock \
  bionicburn/rec-worker-updater:latest

⚠️ IMPORTANT: Create the secrets directory and files before starting:

# Create secrets directory
mkdir -p /path/to/secrets

# Generate update secret (required for POST /update authentication)
openssl rand -base64 32 > /path/to/secrets/update_secret
chmod 600 /path/to/secrets/update_secret

# Optionally create GitHub webhook secret
openssl rand -base64 32 > /path/to/secrets/github_webhook_secret
chmod 600 /path/to/secrets/github_webhook_secret

⁠What It Does

The Worker Updater automates the deployment and management of worker packages with two deployment strategies:

⁠Docker Image Deployment (Recommended)

1. Clones your repository from Git to fetch package configuration
2. Monitors for new Git tags via polling or GitHub webhooks
3. Pulls pre-built Docker images from container registries (GHCR, Docker Hub)
4. Manages worker containers as sibling containers via Docker socket
5. Restarts workers with zero-downtime deployments

Benefits: ~30 second deployments (vs 100+ minutes), no yarn install on slow NAS.

⁠Git-Clone Deployment (Legacy)

1. Clones your repository from Git (full source code)
2. Monitors for changes via Git polling or GitHub webhooks
3. Builds worker packages with yarn install and yarn build
4. Manages worker processes
5. Restarts workers with zero-downtime deployments

Use when: Pre-built Docker images are not available or you need custom builds.

⁠Volume Mounts

⚠️ CRITICAL: The /run/secrets mount is required for authentication to work. Without it:

• POST /update endpoint will be unauthenticated (security risk)
• GitHub webhooks will fail signature validation
• Signature authentication won't work

See the Security Setup⁠ section for how to create secrets.

⁠API Endpoints

The Worker Updater exposes REST API endpoints for monitoring and control:

⁠Health Check

GET /health

Returns lightweight health status. For detailed status with git and worker info, use GET /health/detail.

Example response:

• Updater version and build information
• Git repository status
• Worker process status
• Service health indicators

Example Response:

{
  "status": "healthy",
  "updater": {
    "version": "0.0.15-alpha.1",
    "commit": "abc123d",
    "buildTime": "2026-03-04T21:00:00Z",
    "branch": "main",
    "environment": "production"
  },
  "git": {
    "status": "ready",
    "branch": "main",
    "commit": "abc123def",
    "lastCheck": "2026-03-04T22:00:00Z",
    "lastTag": "worker-v1.0.0"
  },
  "worker": {
    "running": true,
    "status": "running",
    "startedAt": "2026-03-04T20:00:00Z",
    "health": {
      "status": "ok",
      "version": "1.0.0",
      "uptime": 3600
    }
  }
}

⁠Version Information

GET /version

Returns lightweight version information:

{
  "version": "0.0.15-alpha.1",
  "commit": "abc123d",
  "buildTime": "2026-03-04T21:00:00Z",
  "branch": "main",
  "environment": "production"
}

⁠Manual Update Trigger

POST /update

Manually triggers worker update check and deployment.

HMAC Signature Authentication (Recommended):

timestamp=$(date +%s%3N); signature=$(echo -n "${timestamp}" | openssl dgst -sha256 -hmac "your-secret" -hex | cut -d" " -f2); curl -X POST -H "X-Signature: ${signature}" -H "X-Timestamp: ${timestamp}" http://localhost:3003/update

RSA/Ed25519 Signature (Maximum Security):

curl -X POST -H "X-Client-Id: github-ci" -H "X-Signature: <base64-rsa-sig>" -H "X-Timestamp: 1710456789123" http://localhost:3003/update

Shared Secret (DEPRECATED):

curl -X POST -H "X-Update-Secret: your-secret" http://localhost:3003/update

⁠GitHub Webhook

POST /webhook/github

Receives GitHub push events for automatic deployments. Secured with HMAC-SHA256 signature validation.

Configure in GitHub repository settings:

• Payload URL: https://your-server:3003/webhook/github
• Content type: application/json
• Secret: Set in config.json (github.webhookSecretPath)
• Events: Just the push event

⁠Port Mapping

The Worker Updater and managed workers may need multiple ports exposed:

⁠Worker Updater Ports

Configure in config.json:

{
  "updater": {
    "port": 3003
  }
}

⁠Worker Process Ports

Workers managed by the updater may expose their own API endpoints. Configure port mapping in Docker run command or docker-compose:

Example: Worker with API on port 3002

docker run -d \
  --name rec-worker-updater \
  -p 3003:3003 \
  -p 3002:3002 \
  -v /path/to/config:/app/config \
  ...
  bionicburn/rec-worker-updater:latest

Docker Compose Example:

services:
  rec-worker-updater:
    image: bionicburn/rec-worker-updater:latest
    ports:
      - "3003:3003"  # Updater API
      - "3002:3002"  # Worker API (example)
    volumes:
      - ./config:/app/config
      - ./repos:/app/repos
      - ./logs:/app/logs

Port Mapping Strategy:

• Worker Updater uses port 3003 (or configured port)
• Workers typically use ports 3002, 3001, etc.
• Expose only the ports you need externally
• Use packages[].environment.PORT to configure worker listen ports
• Health check endpoints automatically use the configured PORT
• Use packages[].environment.HOST to customise hostname (defaults to localhost)

Example config.json with worker port:

{
  "packages": [
    {
      "name": "@your-org/worker",
      "environment": {
        "PORT": "3002",
        "API_URL": "https://api.example.com"
      }
    }
  ]
}

The worker application reads process.env.PORT and listens on that port. The worker-updater container must expose this port to make it accessible from the host.

⁠Configuration

Create a config.json file in /app/config/ with your deployment settings.

⁠Complete Configuration Example

{
  "$schema": "./config.schema.json",
  "version": "1.0",
  "updater": {
    "port": 3003,
    "logLevel": "info",
    "updateCheckInterval": 300000,
    "healthCheckInterval": 60000
  },
  "git": {
    "repositoryUrl": "[email protected]:user/repo.git",
    "branch": "main",
    "trackingMode": "tags",
    "tagPattern": "worker-v*",
    "tagEnvironment": "production",
    "sshKeyPath": "/root/.ssh/id_rsa",
    "clonePath": "/app/repos/your-repo",
    "author": {
      "name": "Worker Updater",
      "email": "[email protected]"
    }
  },
  "github": {
    "webhookEnabled": true,
    "webhookSecretPath": "/run/secrets/github_webhook_secret",
    "webhookPath": "/webhook/github",
    "token": "ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  },
  "security": {
    "updateSecretPath": "/run/secrets/update_secret",
    "signatureAuthEnabled": false,
    "clientPublicKeysPath": "/run/secrets/client-keys"
  },
  "docker": {
    "socketPath": "/var/run/docker.sock"
  },
  "packages": [
    {
      "name": "@your-org/worker",
      "enabled": true,
      "manifestPath": "packages/worker/package-config.json",
      "containerName": "your-worker",
      "environment": {
        "PORT": "3002",
        "DATABASE_URL": "postgresql://...",
        "API_URL": "https://api.example.com",
        "API_KEY": "your-api-key-here"
      }
    }
  ],
  "storage": {
    "basePath": "/storage",
    "secrets": "/storage/secrets",
    "data": "/storage/data",
    "config": "/storage/config",
    "logs": "/app/logs"
  },
  "logging": {
    "level": "info",
    "directory": "/app/logs"
  }
}

⁠Git Tracking Modes

The Worker Updater supports three tracking modes:

1. Branch Tracking - Monitors branch for new commits (dev/staging) 2. Tag Tracking - Deploys on new tags (production) 3. Commit Tracking - Pins to specific commit SHA (testing)

Example tag mode:

{
  "git": {
    "trackingMode": "tags",
    "tagPattern": "worker-v*",
    "tagEnvironment": "production"
  }
}

⁠GitHub Token Configuration

Required for:

• Fetching tags from private repositories (HTTPS authentication)
• Pulling private Docker images from GHCR (GitHub Container Registry)

Token permissions needed:

• repo (full control of private repositories)
• read:packages (download packages from GitHub Package Registry)
• write:packages (upload packages - optional, only for CD pipelines)

Configuration:

{
  "github": {
    "token": "ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  }
}

⚠️ Security: Store tokens in gitignored config files, never commit them to version control.

How it works:

• Git authentication: Token embedded in HTTPS URLs for tag fetching
• GHCR authentication: docker login ghcr.io with token before pulling images

Alternative: Use SSH for Git (no token needed for Git, but still required for GHCR).

⁠Package Manifest (package-config.json)

Each worker package includes a manifest that defines its deployment strategy and lifecycle:

⁠Deployment Strategies

Docker Image (Recommended):

{
  "$schema": "./package-config.schema.json",
  "type": "nodejs-workspace",
  "deployment": {
    "strategy": "docker-image",
    "dockerImage": {
      "enabled": true,
      "source": {
        "type": "github-container-registry",
        "registry": "ghcr.io",
        "repository": "username/worker-name",
        "tagPattern": "{tag}"
      },
      "fallbackToGitClone": false
    },
    "docker": {
      "containerName": "my-worker",
      "network": "my-network",
      "ports": [{ "container": 3002, "protocol": "tcp" }],
      "restart": "unless-stopped"
    }
  },
  "lifecycle": {
    "health": {
      "type": "http",
      "endpoint": "/health",
      "expectedStatus": 200,
      "timeout": 5000
    }
  }
}

Git-Clone (Legacy):

{
  "$schema": "./package-config.schema.json",
  "type": "nodejs-workspace",
  "deployment": {
    "strategy": "git-clone"
  },
  "lifecycle": {
    "install": {
      "command": "yarn workspaces focus @your-org/worker @your-org/shared",
      "description": "Install only worker and shared workspace dependencies"
    },
    "build": {
      "command": "yarn workspace @your-org/shared build && yarn workspace @your-org/worker build",
      "description": "Build shared package and worker package"
    },
    "start": {
      "command": "node dist/worker/src/main.js",
      "workingDirectory": "packages/worker",
      "description": "Start worker service"
    },
    "health": {
      "type": "http",
      "endpoint": "/health",
      "expectedStatus": 200,
      "timeout": 5000,
      "description": "Lightweight health check (no expensive operations)"
    },
    "healthDetail": {
      "type": "http",
      "endpoint": "/health/detail",
      "expectedStatus": 200,
      "timeout": 10000,
      "description": "Detailed health check with dependencies"
    }
  },
  "update": {
    "watchPaths": [
      "packages/worker/**",
      "packages/shared/**",
      "yarn.lock",
      "package.json"
    ],
    "excludePaths": [
      "packages/worker/dist/**",
      "packages/worker/node_modules/**",
      "**/*.test.ts",
      "**/*.spec.ts"
    ]
  }
}

Lifecycle Commands:

• install: Install dependencies before building
• build: Compile/build the worker package
• start: Command to run the worker
• health: Lightweight health check endpoint (path-only format)
• healthDetail: (Optional) Detailed health check with dependency validation

Health Endpoint Format: Health endpoints use path-only format (e.g., /health) instead of full URLs. The Worker-Updater automatically constructs the full URL using the worker's PORT and HOST environment variables:

• Path-only: /health → Constructed as http://localhost:${PORT}/health
• Full URL: http://custom-host:8080/health → Used as-is (backward compatible)

This allows the same package-config.json to work across different environments without hardcoded ports or hostnames.

Update Configuration:

• watchPaths: File patterns to monitor for changes
• excludePaths: Files to ignore when detecting changes

⁠Environment Variables

Pass environment variables to workers via the packages[].environment section:

{
  "packages": [
    {
      "name": "@your-org/worker",
      "environment": {
        "PORT": "3002",
        "DATABASE_URL": "postgresql://...",
        "API_KEY": "secret-key-here"
      }
    }
  ]
}

Environment variables are:

• Passed to worker process at startup
• Available during build and runtime
• Can be loaded from Docker secrets
• Support for .env file mounting

Automatic NODE_ENV Injection: NODE_ENV is automatically set by the Worker-Updater based on git.tagEnvironment:

• tagEnvironment: "preview" → NODE_ENV=development
• tagEnvironment: "production" → NODE_ENV=production

You don't need to manually set NODE_ENV in the environment configuration.

Automatic Version Injection: When using git tag tracking mode, the Worker-Updater automatically injects the APP_VERSION environment variable to worker processes. This provides accurate version reporting without manual version management:

{
  "git": {
    "trackingMode": "tags",
    "tagPattern": "worker-v*"
  }
}

When a tag like worker-v1.0.0 is deployed, the Worker-Updater:

1. Extracts version 1.0.0 from the tag
2. Injects APP_VERSION=1.0.0 environment variable to the worker
3. Worker reports this version in its /health endpoint

This ensures version consistency between deployment tags and runtime reporting, eliminating manual version management in package.json.

⁠Volume Mapping

Configure persistent storage for workers:

{
  "packages": [
    {
      "name": "@your-org/worker",
      "volumes": {
        "/host/database": "/app/data/db",
        "/host/uploads": "/app/data/uploads",
        "/host/cache": "/app/cache"
      }
    }
  ]
}

Volume Features:

• Persistent data across worker restarts
• Shared storage between updater and worker
• Host path must exist before starting
• Automatic permission handling

⁠Logging System

The Worker Updater provides structured logging for both the updater service and managed workers:

Updater Logs (/app/logs/updater/):

• YYYY-MM-DD.log - All updater activity (JSON format)
• YYYY-MM-DD-error.log - Error-only logs
• Daily rotation with automatic cleanup

Worker Logs (/app/logs/worker/):

• YYYY-MM-DD.log - Worker application logs (JSON format)
• YYYY-MM-DD-error.log - Worker error logs
• Captured from worker stdout/stderr

Log Levels: error, warn, info, debug

Configuration:

{
  "logging": {
    "level": "info",
    "directory": "/app/logs"
  }
}

View logs:

# Updater logs
tail -f /app/logs/updater/2026-03-04.log

# Worker logs
tail -f /app/logs/worker/2026-03-04.log

# Follow container logs
docker logs -f rec-worker-updater

⁠Features

• ✅ Automated Git repository monitoring and updates
• ✅ Package lifecycle management (install, build, start)
• ✅ Zero-downtime worker restarts
• ✅ Structured logging with daily rotation
• ✅ Self-healing worker monitoring: Automatically restart crashed workers
• ✅ Configurable health checks: Monitor worker health at custom intervals
• ✅ Health monitoring and process management
• ✅ GitHub webhook support with HMAC signature validation
• ✅ RSA/Ed25519 signature authentication for manual updates
• ✅ Auto-generation of missing secrets (self-healing secrets)
• ✅ Docker container orchestration (optional)
• ✅ Multi-package support in monorepos
• ✅ Dynamic health endpoint resolution (environment-driven)
• ✅ Automatic version injection from git tags
• ✅ Environment-based configuration (no hardcoded ports/hosts)

⁠SSH Authentication

Mount your SSH private key for Git operations:

# Create SSH key directory
mkdir -p ./ssh
cp ~/.ssh/id_rsa ./ssh/

# Set correct permissions
chmod 600 ./ssh/id_rsa

# Mount in Docker
-v $(pwd)/ssh:/root/.ssh:ro

⁠Health Monitoring

The Worker Updater monitors worker health and automatically restarts failed processes:

• Self-healing: Automatically restarts crashed or unhealthy workers
• Configurable intervals:
  • Update check interval (default: 300s) - monitors for new code deployments
  • Health check interval (optional) - monitors worker health status
• HTTP health check endpoints: Polls worker /health endpoint
• Automatic restart on failure: Detects crashed or unhealthy workers and restarts them
• Structured health status reporting: Detailed status via /health and /health/detail endpoints

⁠Health Check Configuration

Enable self-healing health monitoring by adding healthCheckInterval to your config.json:

{
  "updater": {
    "healthCheckInterval": 60000
  }
}

How it works:

1. Worker-updater polls the worker's health endpoint every healthCheckInterval milliseconds
2. If the worker is not running or health status is not 'ok', it automatically restarts the worker
3. Health checks are skipped during update operations to prevent interference
4. Initial health check starts 30 seconds after updater startup

Recommended intervals:

• Production: 60000 (60 seconds) - Quick recovery from crashes
• Development: 120000 (120 seconds) - Less aggressive checking
• Disabled: Omit healthCheckInterval to disable self-healing

When to use:

• ✅ Production environments where uptime is critical
• ✅ Workers that may crash due to external dependencies
• ✅ Long-running workers that need automatic recovery
• ❌ Development environments where manual restarts are preferred

⁠Versioning Strategy

Uses semantic versioning: <major>.<minor>.<patch>[-alpha.<increment>]

Available Tags:

• latest - Latest stable production release (recommended for production)
• preview - Preview/staging builds from main branch (for testing)
• <version> - Specific version pin (e.g., 0.0.15-alpha.1) for reproducible deployments

Check version:

curl http://localhost:3003/version

⁠Docker Compose Example

services:
  rec-worker-updater:
    image: bionicburn/rec-worker-updater:latest
    container_name: rec-worker-updater
    restart: unless-stopped
    ports:
      - "3003:3003"  # Updater API
      - "3002:3002"  # Worker API (if worker exposes port)
    volumes:
      - ./config:/app/config
      - ./repos:/app/repos
      - ./logs:/app/logs
      - ./secrets:/run/secrets:ro  # REQUIRED for authentication
      - ./ssh:/root/.ssh:ro
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - NODE_ENV=production

Before starting, create required secrets:

# Create secrets directory
mkdir -p ./secrets

# Generate update secret (required)
openssl rand -base64 32 > ./secrets/update_secret
chmod 600 ./secrets/update_secret

# Generate webhook secret (if using GitHub webhooks)
openssl rand -base64 32 > ./secrets/github_webhook_secret
chmod 600 ./secrets/github_webhook_secret

# Start the container
docker-compose up -d

⁠Use Cases

• CI/CD Automation: Deploy workers on Git push
• Self-Healing: Auto-restart on failure
• Multi-Environment: Dev/staging/production workers
• Monorepo: Deploy multiple workers from one repo

⁠Security Setup

⚠️ CRITICAL: Mount /run/secrets for authentication.

⁠Create Secrets

mkdir -p /path/to/secrets && chmod 700 /path/to/secrets
openssl rand -base64 32 > /path/to/secrets/update_secret
chmod 600 /path/to/secrets/update_secret

⁠Configure

{
  "security": {
    "updateSecretPath": "/run/secrets/update_secret"
  }
}

⁠Mount

docker run -d -v /path/to/secrets:/run/secrets:ro ... bionicburn/rec-worker-updater:latest

⁠GitHub Webhook (Optional)

openssl rand -base64 32 > /path/to/secrets/github_webhook_secret
chmod 600 /path/to/secrets/github_webhook_secret

⁠Security

🔒 See Security Setup⁠ above for required secrets configuration.

⁠Authentication Methods

The Worker Updater supports two authentication methods for the POST /update endpoint:

1. Signature Authentication (Advanced - Recommended for Production)

• Uses RSA or Ed25519 asymmetric cryptography
• Clients sign requests with their private key
• Worker-updater verifies with registered public keys
• Each client has unique credentials
• Private keys never transmitted
• Prevents replay attacks with timestamp validation

Setup:

# 1. Create client-keys directory inside secrets
mkdir -p /path/to/secrets/client-keys
chmod 700 /path/to/secrets/client-keys

# 2. Generate client key pair
openssl genrsa -out client-private.pem 4096
openssl rsa -in client-private.pem -pubout -out /path/to/secrets/client-keys/github-ci.pem
chmod 600 /path/to/secrets/client-keys/github-ci.pem

# 3. Keep private key secure (for CI/CD use)
chmod 600 client-private.pem
# Store client-private.pem in GitHub Secrets or secure vault

# 4. Secrets directory structure:
# /path/to/secrets/
# ├── update_secret              # Shared secret (still needed as fallback)
# ├── github_webhook_secret      # Optional webhook secret
# └── client-keys/               # Client public keys directory
#     └── github-ci.pem          # Client public key

Configuration (config.json):

{
  "security": {
    "updateSecretPath": "/run/secrets/update_secret",
    "signatureAuthEnabled": true,
    "clientPublicKeysPath": "/run/secrets/client-keys"
  }
}

Volume Mount (already done if you followed Security Setup):

volumes:
  - ./secrets:/run/secrets:ro  # Mounts update_secret AND client-keys/

Note: The client-keys directory is inside the secrets directory, so the same volume mount provides access to both shared secrets and client public keys.

2. Shared Secret Authentication (Basic)

• Uses X-Update-Secret header
• Good for single-client scenarios
• See Security Setup⁠ above

Best Practices:

• Mount SSH keys read-only (:ro)
• Use repository-specific deploy keys
• Rotate secrets every 90 days
• Set file permissions to 600
• Never commit secrets to git

Full Security Documentation⁠

⁠Support

• Website: https://redeyedcrows.com⁠
• Docker Hub: https://hub.docker.com/r/bionicburn/rec-worker-updater⁠

⁠License

MIT License - Built and maintained by Red Eyed Crows.