chiefj/forecast-worker repository overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                         YOUR APPLICATION STACK                              │
│                                                                             │
│  ┌──────────────────────────────┐    ┌──────────────────────────────────┐  │
│  │   chiefj/forecast-worker     │    │  chiefj/instrumentation-sdk-api  │  │
│  │                              │    │                                  │  │
│  │  ┌──────────────────────┐    │    │  ┌────────────────────────────┐  │  │
│  │  │ Temporal Cron Worker │    │    │  │ FastAPI (port 8000)        │  │  │
│  │  │ Runs every 5 minutes │    │    │  │  POST /v1/metrics/forecast │  │  │
│  │  └──────────┬───────────┘    │    │  │  POST /v1/metrics/record   │  │  │
│  │             │                │    │  │  GET  /v1/metrics/health   │  │  │
│  │  ┌──────────▼───────────┐    │    │  └────────────┬───────────────┘  │  │
│  │  │  TimesFM Inference   │    │    │               │                  │  │
│  │  │  (google/timesfm-    │    │    │  ┌────────────▼───────────────┐  │  │
│  │  │   2.5-200m-pytorch)  │    │    │  │ OTel Prometheus Adapter   │  │  │
│  │  └──────────┬───────────┘    │    │  │ Observable Gauges (p/s)   │  │  │
│  │             │                │    │  └────────────┬───────────────┘  │  │
│  │  ┌──────────▼───────────┐    │    │               │                  │  │
│  │  │  Quantile Output     │    │    │  ┌────────────▼───────────────┐  │  │
│  │  │  mean / p10 / p90    ├────┼────►  │ Prometheus Exporter :9464 │  │  │
│  │  └──────────────────────┘    │    │  └────────────┬───────────────┘  │  │
│  │                              │    │               │                  │  │
│  │  Your app also calls SDK     │    │  ┌────────────▼───────────────┐  │  │
│  │  directly for span metrics   │    │  │ Prometheus  :9090          │  │  │
│  └──────────────────────────────┘    │  └────────────┬───────────────┘  │  │
│                                      │               │                  │  │
│  ┌───────────────────────────────────┼───────────────▼───────────────┐  │  │
│  │               Grafana :3000       │   Dashboards auto-provisioned │  │  │
│  │  ┌────────────────────────────────┼──────────────────────────────┐│  │  │
│  │  │  LLM Cost Forecast Dashboard  │  Mean/p10/p90 time-series    ││  │  │
│  │  │  LLM Cost Dashboard           │  Actual vs Forecast overlay  ││  │  │
│  │  │  LLM Security Dashboard       │  PII / Injection events      ││  │  │
│  │  └────────────────────────────────┴──────────────────────────────┘│  │  │
│  └───────────────────────────────────────────────────────────────────┘  │  │
│                                      └──────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────────┘

docker login -u chiefj
# Enter your token when prompted

docker pull chiefj/instrumentation-sdk-api:latest
docker pull chiefj/forecast-worker:latest

docker run -d \
  --name observability-stack \
  -p 8000:8000 \
  -p 3000:3000 \
  -p 9090:9090 \
  -p 4317:4317 \
  chiefj/instrumentation-sdk-api:latest

curl http://localhost:8000/v1/metrics/health
# → {"initialized": true, "message": "# HELP ..."}

docker run -d \
  --name forecast-worker \
  -e CLICKHOUSE_HOST=your-clickhouse-host \
  -e CLICKHOUSE_PORT=8123 \
  -e CLICKHOUSE_DATABASE=llm_observability \
  -e CLICKHOUSE_USER=default \
  -e CLICKHOUSE_PASSWORD=your-password \
  -e INSTRUMENTATION_SDK_URL=http://observability-stack:8000 \
  -e TEMPORAL_HOST=your-temporal-host:7233 \
  --link observability-stack \
  chiefj/forecast-worker:latest

curl -X POST http://localhost:8000/v1/metrics/forecast \
  -H "Content-Type: application/json" \
  -d '{
    "mean": 9200,
    "p10": 3800,
    "p90": 17500,
    "model": "gpt-4o",
    "provider": "openai",
    "service_name": "my-chat-app"
  }'

import httpx

SDK_URL = "http://localhost:8000"

# Record a span (actual usage)
httpx.post(f"{SDK_URL}/v1/metrics/record", json={
    "model": "gpt-4o",
    "provider": "openai",
    "service_name": "my-chat-app",
    "prompt_tokens": 1500,
    "completion_tokens": 400,
    "latency_ms_total": 1800,
    "latency_ms_ttft": 320,
    "finish_reason": "stop",
    "status": "success",
    "cost_usd_micro": 7200,
    "pii_detected": False,
    "injection_attempt": False,
})

# Push a forecast (from your own model or from forecast-worker output)
httpx.post(f"{SDK_URL}/v1/metrics/forecast", json={
    "mean": 9200,
    "p10": 3800,
    "p90": 17500,
    "model": "gpt-4o",
    "provider": "openai",
    "service_name": "my-chat-app",
})

[Cron: 5 * * * *]
       │
       ▼
1. fetch_cost_series (Activity F-FM-01)
       │  Queries ClickHouse cost_by_dimension for last 168 hours
       ▼
2. ForecastService.build_dense_series (Domain Layer)
       │  Zero-pads missing hours, validates min_history_hours >= 48
       ▼
3. TimesFM Inference (Adapter Layer)
       │  google/timesfm-2.5-200m-pytorch
       │  context_len=168, patch_len=32, horizon=24
       ▼
4. Quantile Projection
       ├── forecast_mean → Expected cost trend
       ├── forecast_p10  → Optimistic lower bound
       └── forecast_p90  → Worst-case for budget breach alerting
       │
       ▼
5. POST /v1/metrics/forecast → instrumentation-sdk-api
       │
       ▼
6. OTel Observable Gauge → Prometheus :9464 → Grafana

docker run -d \
  --name forecast-worker \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  ... \
  chiefj/forecast-worker:latest

packages/python/forecast-worker/
├── build/
│   └── Dockerfile             # CPU-optimized PyTorch container
├── contracts/
│   └── workflows/
│       └── forecast_workflow.yaml
├── database/
│   ├── migrations/
│   │   ├── 0001_init.rollback.sql
│   │   └── 0001_init.sql
│   └── schema.lock
├── outcome/                   # Generated forecast graphs (PNG/SVG)
├── scripts/
│   └── test.sh
├── src/
│   ├── features/forecast/
│   │   └── service.py         # Dense series + business validation
│   ├── infra/adapters/clickhouse/
│   │   └── clickhouse_adapter.py
│   ├── shared/
│   │   ├── contracts/validator.py
│   │   ├── errors/base.py
│   │   └── ports/clickhouse_port.py
│   └── worker/
│       └── index.py           # Entry point
├── tests/unit/
├── pyproject.toml
├── worker-registry.yaml
└── README.md

python3 -m venv --without-pip .venv
.venv/bin/python3 get-pip.py
.venv/bin/pip install -e ".[dev]"

./scripts/test.sh

.venv/bin/pytest --cov=src --cov-report=term-missing

# Pull and run the full stack
docker pull chiefj/instrumentation-sdk-api:latest
docker pull chiefj/forecast-worker:latest