gokhalh/localaik repository overview

⁠localaik

Think LocalStack, but for AI APIs. One container that emulates Gemini, OpenAI, and Anthropic on the same port. Point your existing SDK at http://localhost:8090 and your code just works.

No API key. No internet. Deterministic enough for CI and local development.

⁠Quick start

docker run -d -p 8090:8090 gokhalh/localaik

Or in Docker Compose:

services:
  localaik:
    image: gokhalh/localaik
    ports:
      - "8090:8090"

The port accepts connections before the model has loaded, so wait for GET /health to return 200 rather than for the port to open.

⁠Point your SDK at it

Gemini (Python):

from google import genai
client = genai.Client(
    api_key="test",
    http_options=genai.types.HttpOptions(api_version="v1beta", base_url="http://localhost:8090"),
)

OpenAI (Python):

from openai import OpenAI
client = OpenAI(api_key="test", base_url="http://localhost:8090/v1")

Anthropic (Python):

from anthropic import Anthropic
# The Anthropic SDKs append v1/ themselves, so the base URL is the bare host.
client = Anthropic(api_key="test", base_url="http://localhost:8090")

Any language or SDK that can override the base URL will work. Text, image, and PDF inputs are all handled.

⁠Tags

Version-pinned: vX.Y.Z-gemma3-4b, vX.Y.Z-gemma3-12b, vX.Y.Z-proxy.

⁠Implemented routes

⁠Gemini (/v1beta/...)

• POST /v1beta/models/{model}:generateContent — Models.GenerateContent
• POST /v1beta/models/{model}:streamGenerateContent — Models.GenerateContentStream (SSE)
• POST /v1beta/models/{model}:countTokens — Models.CountTokens
• GET /v1beta/models — Models.List
• GET /v1beta/models/{model} — Models.Get

⁠OpenAI (/v1/...)

• POST /v1/chat/completions — Chat.Completions.New
• POST /v1/completions — legacy Completions.New
• GET /v1/models — Models.List
• GET /v1/models/{id} — Models.Retrieve

⁠Anthropic (/v1/messages)

• POST /v1/messages — Messages.Create (streaming supported)
• POST /v1/messages/count_tokens — Messages.CountTokens

⁠Health

• GET /health

⁠Bring your own model server (:proxy)

Already run llama.cpp, or any server that speaks the OpenAI chat-completions API? The proxy tag is the translation layer alone — 41 MB, no model, no inference engine.

docker run -d -p 127.0.0.1:8090:8090 \
  -e LK_UPSTREAM=http://your-server:8080/v1 \
  gokhalh/localaik:proxy

LK_UPSTREAM is required. LK_UPSTREAM_AUTH_HEADER="Authorization: Bearer ..." sends a credential to your upstream, and only there. :proxy authenticates none of its own callers, so bind to localhost and do not publish the port on a shared network.

⁠Use in GitHub Actions

services:
  localaik:
    image: gokhalh/localaik
    ports:
      - 8090:8090
    options: >-
      --health-cmd "curl -f http://localhost:8090/health"
      --health-interval 10s
      --health-timeout 5s
      --health-retries 30

⁠Logging

One access line per request, tagged with the client protocol:

localaik  [openai]    POST /v1/chat/completions  200  412ms
localaik  [anthropic] POST /v1/messages          200  318ms

Set LK_LOG=off to silence it. Headers and bodies are never logged.

⁠Tuning

Pass env vars to tune the underlying llama.cpp:

docker run -d -p 8090:8090 \
  -e LK_THREADS=8 -e LK_CTX_SIZE=4096 \
  -e LK_FLASH_ATTN=1 -e LK_CONT_BATCHING=1 \
  gokhalh/localaik

Full variable list, source code, and issues: https://github.com/harshaneel/localaik⁠

⁠Limitations

• For tests and dev, not production
• The model-bundled images are large; their size is dominated by model weights (the proxy tag ships none)
• Cold start ~10-30s while the model loads

⁠License

MIT