rustrak/llmtrack repository overview

⁠llmtrack

A self-hosted LLM gateway in one Rust binary. Point any OpenAI SDK at it, hand each team a virtual key, and see exactly what every one of them spent.

OpenAI- and Anthropic-compatible API in front of OpenAI, Azure OpenAI, Anthropic, Google Gemini, Groq, Mistral, DeepSeek, OpenRouter, Together, Ollama or any OpenAI-compatible server. The dashboard is inside the same image.

• 16 MB image, amd64 and arm64, distroless, runs as non-root
• Virtual keys and teams: budgets that reset daily, weekly or monthly, requests- and tokens-per-minute limits, model access, expiry
• Spend tracking per request, team, key, person, customer and tag, priced from a daily-synced price list (cache and reasoning tokens priced apart)
• Router: weighted deployments, retries, cooldowns, fallbacks
• LiteLLM-compatible: reads x-litellm-* headers, litellm_metadata, LITELLM_MASTER_KEY and router_settings
• Fast: +0.11 ms added p50 latency and under 90 MB of memory under load (benchmarks⁠)
• Prompts and completions are never stored

⁠Tags

Pin a version (v0.1.0) in production; latest moves with every release.

⁠Quick start

docker run -d --name llmtrack -p 4000:4000 -v llmtrack_data:/data \
  -e SECRET_KEY=$(openssl rand -hex 32) \
  -e [email protected]:changeme123 \
  rustrak/llmtrack:latest

Open http://localhost:4000⁠, sign in, add a model, create a team and a key. Then call it like OpenAI:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:4000/v1", api_key="sk-…")
client.chat.completions.create(
    model="gpt-5.6",
    messages=[{"role": "user", "content": "Hello"}],
)

⁠Docker Compose

services:
  llmtrack:
    image: rustrak/llmtrack:latest
    ports: ["4000:4000"]
    volumes: [llmtrack_data:/data]
    environment:
      - SECRET_KEY=${SECRET_KEY}
      - CREATE_SUPERUSER=${CREATE_SUPERUSER}
    restart: unless-stopped

volumes:
  llmtrack_data:

⁠PostgreSQL

services:
  llmtrack:
    image: rustrak/llmtrack:postgres
    ports: ["4000:4000"]
    environment:
      - DATABASE_URL=postgres://llmtrack:llmtrack@db:5432/llmtrack
      - SECRET_KEY=${SECRET_KEY}
      - CREATE_SUPERUSER=${CREATE_SUPERUSER}
    depends_on: [db]
  db:
    image: postgres:16-alpine
    environment:
      - POSTGRES_USER=llmtrack
      - POSTGRES_PASSWORD=llmtrack
      - POSTGRES_DB=llmtrack
    volumes: [pg_data:/var/lib/postgresql/data]

volumes:
  pg_data:

⁠Configuration

Data lives in the /data volume. Back it up; it holds your keys, settings and usage history.

⁠Links

• GitHub⁠: source, README, issues
• Releases⁠
• Rustrak⁠: self-hosted error tracking, from the same team

License: GPL-3.0.