ykocaman/ilter repository overview

⁠ILTER — AI Gateway

Deploying AI models to production brings critical operational challenges such as unpredictable costs, data privacy risks, and API outages. ILTER is an independent gateway that sits between your application and AI providers, transforming your entire AI traffic into a secure, optimized, and fully controlled infrastructure without requiring any architectural changes in your code.

• 🧩 MCP Gateway & Marketplace: Connect AI to your APIs, CRM, or MCP servers — zero changes required.
• ⚡ Smart Router: Every request is scored for complexity in real time and routed to the right model tier.
• 🌐 Smart Fallback: Provider down or rate-limited? ILTER fails over to another key or provider, no changes.
• 💰 Budget Control: Hard daily/monthly spending limits per key — traffic cuts off the instant a limit is hit.
• 🪪 PII Guard: Emails, SSNs, and other sensitive data are masked before they ever leave your network.
• 🧠 Semantic Cache: Queries are served from cache — vector search with SHA-256 exact-match fallback.
• 🛡️ Prompt Guardrails: Blocks injection, toxic content, and off-topic requests before they reach the model.
• 🌀 Agent Loop Detector: Catches loops by rate, fingerprint, cost, or depth before burning budget.
• ⏱️ Cron Engine: Schedule recurring AI workflows with standard cron expressions — no external queue needed.
• 📡 Observability: Prometheus metrics and OpenTelemetry tracing, ready for Grafana, Datadog, or Honeycomb.
• 📊 Dashboard: Embedded web UI for KPIs, audit logs, API keys, and built-in chat playground — no server.

Ships as a zero-setup, single static binary — this image is a scratch-based build under 20MB.

⁠Quick start

Admin key + a provider key are enough to boot straight into serve — no ilter init step needed. The SQLite DB is created on first start, and the provider whose ILTER_PROVIDER_<NAME>_API_KEY is set is enabled automatically.

docker run -d \
  -p 8181:8181 -p 9191:9191 -p 9192:9192 \
  -v $(pwd)/data:/app/data \
  -e ILTER_ADMIN_API_KEY=<your-own-random-secret> \
  -e ILTER_PROVIDER_OPENAI_API_KEY=sk-... \
  ykocaman/ilter:latest

• 8181 — OpenAI-compatible proxy endpoint
• 9191 — Embedded dashboard
• 9192 — Prometheus metrics (/metrics)

Without ILTER_ADMIN_API_KEY and a provider key both set, serve refuses to start and exits with a message telling you to set both.

⁠Supported providers

OpenAI, Anthropic, Google Gemini, DeepSeek, OpenRouter, Ollama, Qwen (Alibaba), OpenCode, and a built-in Mock provider for local testing.

⁠Configuration

Zero configuration by default — compiled defaults + ILTER_* env vars. Runtime and per-key overrides (PII toggles, providers, routing strategies, budgets, rate limits) are available via the embedded dashboard or the Admin API.

⁠Links

• Source: https://github.com/ilter-ai/ilter⁠
• License: Apache 2.0 with Commons Clause