mzdotai/gateway repository overview

⁠Why gateway?

gateway sits between your applications and LLM providers so you can control access, cost, and observability in one place.

• OpenAI-compatible endpoints (/v1/chat/completions, /v1/embeddings, /v1/models)
• Virtual API key management (/v1/keys) for safe client access
• User and budget controls (/v1/users, /v1/budgets)
• Usage and pricing tracking (/v1/messages, /v1/pricing)
• Health and metrics endpoints (/health, optional /metrics)

⁠Quickstart

⁠1) Install

uv venv
source .venv/bin/activate
uv sync --dev

⁠2) Configure

cp config.example.yml config.yml

Edit config.yml and set at least:

• master_key
• one provider credential in providers (for example openai.api_key)

⁠3) Run

uv run gateway serve --config config.yml

Open API docs at http://localhost:8000/docs.

⁠Start in platform mode

Platform mode is enabled automatically when OTARI_PLATFORM_TOKEN is set.

1. Export platform env vars:

export OTARI_PLATFORM_TOKEN=gw_xxx
export PLATFORM_BASE_URL=https://your-platform.example/api/v1

1. Start the gateway:

uv run gateway serve --config config.yml

Notes:

• GATEWAY_MODE is optional; effective mode is derived from OTARI_PLATFORM_TOKEN.
• If you explicitly set GATEWAY_MODE=platform, startup fails unless OTARI_PLATFORM_TOKEN is also set.
• In platform mode, local providers configuration is not used.

⁠First request (OpenAI SDK)

On startup, the gateway can bootstrap an API key in logs. Export it as GATEWAY_API_KEY, then call the gateway as an OpenAI-compatible server:

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GATEWAY_API_KEY"],
    base_url="http://localhost:8000/v1",
)

response = client.chat.completions.create(
    model="openai:gpt-4o",
    messages=[{"role": "user", "content": "Hello from gateway"}],
)

print(response.choices[0].message.content)

⁠Local development

Run with hot reload and .env:

cp .env.example .env
make dev

⁠Tests and checks

make test
make lint
make typecheck

Run a single test file:

uv run pytest tests/unit/test_gateway_cli.py -v

⁠Docker

The gateway image is published on Docker Hub⁠.

⁠Run with docker compose (gateway + PostgreSQL)

cp config.example.yml config.yml
docker compose up -d

⁠Run with docker only

docker run --rm \
  -p 8000:8000 \
  -v "$(pwd)/config.yml:/app/config.yml:ro" \
  mzdotai/gateway:latest \
  gateway serve --config /app/config.yml

Gateway will be available at http://localhost:8000.

⁠API surface

• GET /health
• POST /v1/chat/completions
• POST /v1/embeddings
• GET /v1/models
• POST/GET /v1/keys
• POST/GET /v1/users
• POST/GET /v1/budgets
• GET /v1/messages
• GET /v1/pricing

Full schema: docs/public/openapi.json

⁠Useful CLI commands

uv run gateway init-db --config config.yml
uv run gateway migrate --config config.yml
uv run gateway migrate --config config.yml --revision <rev>
uv run python scripts/generate_openapi.py --check

⁠License

Apache 2.0. See LICENSE.