mukovoz/xtjobs repository overview

⁠xtjobs

A job queue with fair load across projects, stored in Redis, with a live web dashboard built in.

• Add jobs over REST, WebSocket or a TCP socket. Each job belongs to an environment, a project and a queue.
• Workers connect over TCP or WebSocket and get jobs pushed to them. Projects are served round-robin, so no project starves the others.
• Retries with a delay, pause/resume, transfer and delete by scope.
• Finished jobs move out of Redis into a SQLite archive on disk.
• One shared token protects the REST API, WebSocket and TCP socket.

⁠Quick start

xtjobs needs Redis 8+. Redis 8 includes the Query Engine that job listing uses.

docker network create xtjobs
docker run -d --name redis --network xtjobs redis:8-alpine redis-server --appendonly yes
docker run -d --name xtjobs --network xtjobs \
  -p 8080:8080 -p 8081:8081 -p 9090:9090 \
  -e XTJOBS_TOKEN=change-me \
  -v xtjobs-data:/var/lib/xtjobs \
  mukovoz/xtjobs

Open http://localhost:8080/⁠: the dashboard asks for the token, and the REST API is under /api/v1.

curl -H "Authorization: Bearer change-me" -H "Content-Type: application/json" \
  -d '{"queue":"emails","project":"shop","environment":"production","data":{"to":"[email protected]"}}' \
  http://localhost:8080/api/v1/jobs

⁠Docker Compose

services:
  redis:
    image: redis:8-alpine
    command: ["redis-server", "--appendonly", "yes"]
    volumes: [redis-data:/data]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s

  xtjobs:
    image: mukovoz/xtjobs:latest
    environment:
      XTJOBS_TOKEN: change-me
    ports:
      - "8080:8080"   # dashboard + REST API
      - "8081:8081"   # WebSocket (dashboard, workers)
      - "9090:9090"   # TCP socket (workers, producers)
    volumes:
      - xtjobs-data:/var/lib/xtjobs
    depends_on:
      redis: { condition: service_healthy }
    restart: unless-stopped

volumes:
  redis-data:
  xtjobs-data:

⁠Configuration

The image ships a default config at /etc/xtjobs/config.yaml: Redis at redis:6379, no auth, and the archive in /var/lib/xtjobs. Override it with environment variables, or mount your own file (-v ./config.yaml:/etc/xtjobs/config.yaml:ro).

Always set XTJOBS_TOKEN when the ports can be reached from outside. Without it, anyone who can reach them has full access.

Port mapping: the dashboard connects to the WebSocket on port 8081 of the same host. If you change a port, change it on both sides (e.g. -e XTJOBS_WEBSOCKET_ADDR=:9081 -p 9081:9081). Behind a reverse proxy, route /ws to port 8081 and the rest to 8080, then set XTJOBS_UI_WEBSOCKET_URL=/ws.

⁠Volumes and health

• /var/lib/xtjobs holds archive.db, the SQLite archive of finished jobs. Mount a volume to keep it.
• Queued jobs live in Redis. Run Redis with persistence (--appendonly yes) and maxmemory-policy noeviction.
• GET /health (no token) returns {"status":"ok","redis":"ok","version":"…"}, or HTTP 503 when Redis is down.
• Startup summary: it's printed to stderr and shows the URLs, the Redis checks (version, search module, persistence, eviction) and the job counts. docker logs xtjobs shows it.

⁠Tags

• latest: the newest release.
• X.Y.Z (e.g. 0.3.1): a specific release. A released tag is never overwritten.

Each tag covers linux/amd64 and linux/arm64. The image is about 32 MB (Alpine, static Go binary), runs as a non-root user, and has the dashboard built in.