lemoelink/l3mcore repository overview

⁠l3mcore – Light Easy Mix Of Experts

l3mcore is a lightweight, open-source Mixture of Experts (MoE) intelligent routing proxy. It acts as a drop-in replacement for OpenAI and Ollama APIs, routing each request to the most suitable LLM expert based on semantic understanding of the input.

⁠Features

• OpenAI & Ollama compatible API – Any client that speaks either protocol works out of the box (Open WebUI, Continue, LangChain, LibreChat, OpenAI SDK, etc.)
• Real-time token streaming – Low-latency token-to-token streaming via Server-Sent Events (SSE) and ndjson
• vLLM & OpenAI-compatible support – Connect local or remote vLLM, TGI, LocalAI, or Ollama instances natively
• Semantic hybrid router – Multi-signal scoring (embeddings + keywords + fuzzy matching) with softmax normalization
• Multi-backend expert dispatch – Route to Ollama, OpenAI, Anthropic, Gemini (via LiteLLM), local ONNX, or local GGUF models
• Circuit Breaker & Silent self-correction – Automatic failover and fast circuit tripping for degraded backends
• Hot-reload – Edit experts.json or config.json and changes apply without restart
• Rate limiting – Sliding-window per-IP rate limiter (configurable)
• Security headers & Sanitization – XSS protection, nosniff, DENY framing, prompt injection and canary detection
• CORS support – Configurable cross-origin resource sharing
• Health endpoints – System status and expert backend health probes

⁠Available Image Tags

⁠Quick Start with Docker

⁠1. Standalone Routing Proxy (Default)

docker run -d \
  --name l3mcore \
  -p 11435:11435 \
  -v $(pwd)/config:/app/config \
  -v $(pwd)/models:/app/models \
  -v $(pwd)/data:/app/data \
  lemoelink/l3mcore:latest

⁠2. l3mcore + Open WebUI Bundle

docker run -d \
  --name l3mcore-bundle \
  -p 3000:8080 \
  -p 11435:11435 \
  -v $(pwd)/config:/app/lemoe/config \
  -v open-webui-data:/app/backend/data \
  lemoelink/l3mcore:bundle

Open WebUI will be available at http://localhost:3000.

⁠3. l3mcore + Ollama Bundle

docker run -d \
  --name l3mcore-ollama \
  -p 11434:11434 \
  -p 11435:11435 \
  -v $(pwd)/config:/app/config \
  -v ollama-models:/root/.ollama \
  lemoelink/l3mcore:ollama

⁠4. Complete All-in-One Stack (Open WebUI + l3mcore + Ollama)

docker run -d \
  --name l3mcore-all-in-one \
  -p 3000:8080 \
  -p 11435:11435 \
  -v $(pwd)/config:/app/lemoe/config \
  -v open-webui-data:/app/backend/data \
  -v ollama-models:/root/.ollama \
  lemoelink/l3mcore:all-in-one

Access the complete private AI ecosystem directly at http://localhost:3000.

⁠5. NVIDIA GPU Acceleration (CUDA)

docker run -d \
  --name l3mcore-cuda \
  --gpus all \
  -p 11435:11435 \
  -v $(pwd)/config:/app/config \
  -v $(pwd)/models:/app/models \
  -v $(pwd)/data:/app/data \
  lemoelink/l3mcore:cuda

⁠Docker Compose

Save the following as docker-compose.yml:

version: '3.8'

services:
  l3mcore:
    image: lemoelink/l3mcore:latest
    container_name: l3mcore
    ports:
      - "11435:11435"
    volumes:
      - ./config:/app/config
      - ./models:/app/models
      - ./data:/app/data
    restart: unless-stopped

Start the service:

docker compose up -d

⁠Environment Variables