n8500x/litellm-proxy repository overview

⁠LiteLLM Proxy — Air-Gapped Production Image

OpenAI-compatible API proxy for routing requests to multiple LLM providers. Fully self-contained: every runtime binary, tokeniser encoding, and data file is baked in at build time. Safe to run behind a corporate firewall with no egress except to your configured LLM API endpoints.

⁠Build

docker build -t litellm-proxy docker/litellm/

⁠Run

docker run -p 4000:4000 \
  -e LITELLM_MASTER_KEY=sk-your-key \
  -e OPENAI_API_KEY=sk-... \
  -e ANTHROPIC_API_KEY=sk-... \
  -v $(pwd)/litellm_config.yaml:/app/config.yaml \
  litellm-proxy:latest

⁠Environment Variables

⁠Endpoints

⁠Corporate CA Bundle

Mount your corporate root CA at /etc/ssl/certs/corporate-ca.crt or set:

-e SSL_CERT_FILE=/etc/ssl/certs/corporate-ca.crt
-e REQUESTS_CA_BUNDLE=/etc/ssl/certs/corporate-ca.crt
-e NODE_EXTRA_CA_CERTS=/etc/ssl/certs/corporate-ca.crt

⁠Configuration

The default config is at /app/config.yaml inside the image. Override it with a volume mount or Helm configmap. See litellm_config.yaml for the full example with all supported providers.

⁠Helm

Extract the bundled Helm chart:

docker cp $(docker create --name tmp litellm-proxy):/chart ./chart && docker rm tmp

⁠Embedded Artifacts (no runtime downloads)

• Prisma engine binaries
• tiktoken encodings (cl100k_base, o200k_base, p50k_base, p50k_edit, r50k_base, gpt2)
• HuggingFace tokenizer cache
• NLTK punkt tokenizer data