HUB
Sign inSign up

inebrio/routerly

By inebrio

•Updated 17 days ago

Self-hosted LLM gateway with multi-provider routing, budgets, web dashboard, and CLI.

Image
Developer tools
Monitoring & observability
1

10K+

inebrio/routerly repository overview

⁠Routerly

One gateway. Any AI model. Total control.

Routerly is the self-hosted decision and control layer for your LLM providers. It sits between your applications and OpenAI, Anthropic, Google Gemini, Ollama and more. It decides which model serves each request through composable routing policies with explainable decisions, and it controls who may send what through roles, permissions, budgets, guardrails and audit.

Wire-format transparent. Change the base URL in your OpenAI or Anthropic client, nothing else. Routerly adds no custom headers and no non-standard fields, and returns the provider's response as received. Payloads change only for features you switch on yourself (PII scrubbing, guardrail steering, token optimizers).

Your App  ──────────────────────▶  Routerly  ──▶  OpenAI / Anthropic
                                      │       ──▶  AWS Bedrock
  POST /v1/chat/completions           │       ──▶  Azure OpenAI
  (same API, any SDK)                 │       ──▶  Google Vertex AI
                                      │       ──▶  Gemini / Mistral / Groq / Cohere
                                      │       ──▶  Ollama (local)
                              Auth · Route · Budget · Track

⁠Quick start

docker run -d \
  --name routerly \
  -p 3000:3000 \
  -v routerly_data:/data \
  -e ROUTERLY_HOME=/data \
  --restart unless-stopped \
  inebrio/routerly:latest

Open http://localhost:3000/dashboard⁠, create the admin account and add your first model.

⁠docker-compose
services:
  routerly:
    image: inebrio/routerly:latest
    ports:
      - "3000:3000"
    volumes:
      - routerly_data:/data
    environment:
      - ROUTERLY_HOME=/data
    restart: unless-stopped

volumes:
  routerly_data:

⁠What Routerly does

  • Routers, orchestrators, passthrough: a router scores models; an orchestrator forwards to a priority-ordered list of routers; a passthrough forwards the client's own credential to OpenAI or Anthropic.
  • 11 routing policies (llm, semantic-intent, model-preference, cheapest, health, performance, capability, context, budget-remaining, rate-limit, fairness) combined into one ranking, plus reusable routing profiles. Automatic fallback if a provider fails.
  • Explainable decisions: routing trace per request, Playground with trace and cost breakdown, per-call usage records.
  • Budgets and cost tracking: every call priced per token; hard limits from hourly to yearly at global, router and token level, enforced before the provider is called.
  • Guardrails and PII scrubbing: rule- and judge-based content guardrails and PII policies on requests, responses or both.
  • 8 token optimizers, off by default and enabled per router.
  • Experiments: A/B test whole routers against each other with no client change.
  • Roles and audit: custom roles over 26 granular permissions, two-factor authentication, audit log.
  • Notifications: in-app inbox, email (SMTP, SES, SendGrid, Azure, Google) and webhooks (generic, Slack, Teams, PagerDuty, Discord).
  • MCP server: personal MCP tokens let Claude Code, Cursor, Cline and other MCP clients inspect and adjust Routerly with your permissions.
  • Zero infrastructure: no database, no Redis. Configuration lives in JSON files.
  • Dashboard, CLI and API: every function is available in all three.

⁠Works with any OpenAI or Anthropic client

Cursor, Open WebUI, LibreChat, LangChain, LlamaIndex or plain curl: anything that speaks the OpenAI or Anthropic protocol works without modification (/v1/chat/completions, /v1/responses, /v1/messages).

client = OpenAI(
    base_url="http://localhost:3000/v1",
    api_key="your-router-token"
)

⁠Configuration

All data and configuration are stored in /data inside the container. Mount a volume there to persist settings across restarts.

VariableDefaultDescription
ROUTERLY_HOME/dataData directory
ROUTERLY_PORT3000HTTP port
NODE_ENVproductionNode environment

⁠Supported platforms

  • linux/amd64
  • linux/arm64 (Apple Silicon, AWS Graviton, Raspberry Pi)

⁠Tags

TagDescription
latestLatest stable release
vX.Y.ZSpecific version
developLatest develop build (pre-release)

Tag summary

Content type

Image

Digest

sha256:5ccc57655…

Size

537.2 MB

Last updated

17 days ago

docker pull inebrio/routerly