Offline embedding sidecar for the pattern MCP servers — Hugging Face TEI (CPU) with Qwen3-Embedding-0.6B (ONNX fp32, ORT backend) baked into the image. No downloads, no API keys at runtime.
┌────────────────────────────────────────────────────────────┐
│ architecture-pattern-mcp / agent-pattern-mcp │
│ ────── HTTP /v1/embeddings ──────► │
│ │
│ ┌────────────────────────────┐ │
│ │ pattern-tei-embed │ ◄── pattern-tei-infra │
│ │ Qwen3-Emb-0.6B (ONNX fp32)│ │
│ │ :8080 (TEI, CPU) │ │
│ └────────────────────────────┘ │
└────────────────────────────────────────────────────────────┘
This image is the embedder only. It contains no MCP server, no reranker,
and no LLM client. The MCP servers (Olk/architecture-pattern-mcp,
Olk/agent-pattern-mcp) reach it via
the shared Docker network pattern-tei-shared at the DNS name
pattern-tei-embed (see the pattern-tei-infra stack).
Runtime base is ghcr.io/huggingface/text-embeddings-inference:cpu-1.9
(shared libs, ONNX Runtime dylib), but the router binary is self-built
TEI v1.9.4 + unmerged PR #884 (5806bdc, ort,http features), not the
stock cpu-1.9 binary. The rebuild exists for exactly one reason:
--warmup-tokens (see below).
onnx-community/Qwen3-Embedding-0.6B-ONNX (float32 ONNX,
onnx/model.onnx + onnx/model.onnx_data, ~4.7 GB), served by the ORT backendCMD, not overridable via env):
--model-id /data/qwen3-embedding-0.6b --pooling last-token --dtype float32
— ORT requires float32; Qwen3 requires last-token poolingPOST /v1/embeddings (OpenAI-compat, what the MCP client uses) ·
POST /embed (raw) · GET /health · GET /infoHF_HUB_OFFLINE=1max_batch_size 8 for this model
(see GET /info); this matches the MCP client's EMBEDDER_BATCH_SIZE=8,
so no client-side batch splitting occurs--warmup-tokens problemStock TEI 1.9 has no way to disable the synthetic startup warmup pass.
On CPU it always warms min(max_input_length, max_batch_tokens) tokens
before serving. Measured on a 32-core host, that single pass costs ~131 s
of ~142 s total startup while adding zero accuracy — it only exercises the
compute graph.
The documented workaround (lower MAX_BATCH_TOKENS) is a bad trade: it
shrinks the warmup, but it also permanently throttles serving throughput
(upstream issue #819: 16384 → minutes of warmup, 1024 → ~3 s, plus a
known CPU warmup OOM at large batch sizes, PR #634/#725).
This image ships the PR #884
patch, which adds --warmup-tokens / WARMUP_TOKENS:
| Setting | Behaviour |
|---|---|
WARMUP_TOKENS=0 (this image's default) | Skip the synthetic warmup entirely. Startup is ~10 s weight load + ORT init |
WARMUP_TOKENS=N (positive) | Warm with exactly N tokens instead of the full min(max_input_length, max_batch_tokens) |
| Unset (stock TEI behaviour) | Full warmup — minutes on CPU. Only use to reproduce upstream timings |
Values larger than MAX_BATCH_TOKENS are rejected at startup. Clap
precedence applies: CLI flag > env var > built-in default. When rebasing to a
newer TEI tag, bump TEI_SOURCE_VERSION and TEI_PR884_SHA together — and
drop the patch once the PR merges upstream.
Eight tunables, all via environment (docker run -e or compose
environment:). Image defaults are the safe-CPU values:
| Variable | Default | Notes |
|---|---|---|
WARMUP_TOKENS | 0 | Skip synthetic warmup (see above) |
MAX_BATCH_TOKENS | 8192 | Upstream default is 16384; 8192 avoids the CPU warmup OOM and matches Qwen's own max_length=8192. config.json's max_position_embeddings is patched 32768 → 8192 to match |
MAX_CONCURRENT_REQUESTS | 32 | Safe CPU cap (upstream default 512) |
MAX_CLIENT_BATCH_SIZE | 32 | Must be ≥ client's EMBEDDER_BATCH_SIZE, else TEI returns HTTP 429 no permits available (one semaphore permit per input) |
AUTO_TRUNCATE | true | Keep on: client requirement strings can reach 100 000 chars |
PROMETHEUS_PORT | 0 | OS-assigned ephemeral port — /metrics never exposed outside the container |
HOSTNAME | 0.0.0.0 | Included for API consistency; Docker overwrites it at runtime, effective bind stays 0.0.0.0 |
PORT | 8080 | If changed, update EMBEDDER_BASE_URL and healthcheck URLs |
Keep the invariant EMBEDDER_BATCH_SIZE (client, 8) ≤ MAX_CLIENT_BATCH_SIZE ≤ MAX_CONCURRENT_REQUESTS.
Two log warnings are expected and harmless: Could not find a Sentence Transformers config (artifact lacks the ST file; TEI falls back to
config.json) and Backend does not support a batch size > 8 (ORT's
intrinsic cap for this model, not your flag setting).
docker compose version)Part of the pattern-tei-infra stack
(recommended — shared Docker network pattern-tei-shared, DNS name pattern-tei-embed):
git clone https://github.com/Olk/pattern-tei-infra.git && cd pattern-tei-infra
TEI_IMAGE=olkowa/pattern-tei-embed:latest docker compose up -d
Or standalone (no MCP consumer):
docker run -d --name pattern-tei-embed -p 8080:8080 olkowa/pattern-tei-embed:latest
curl -fsS http://localhost:8080/health
Restore the full warmup for comparison:
docker run -d --name pattern-tei-embed -p 8080:8080 \
-e WARMUP_TOKENS=8192 olkowa/pattern-tei-embed:latest
Mirrored on GHCR at
ghcr.io/olk/pattern-tei-embed.
# Update
docker compose -f pattern-tei-infra/docker-compose.yml pull
# Logs
docker compose -f pattern-tei-infra/docker-compose.yml logs -f pattern-tei-embed
# Health
docker inspect --format '{{.State.Health.Status}}' pattern-tei-embed
# Pin a version
TEI_IMAGE=olkowa/pattern-tei-embed:0.1.0 \
docker compose -f pattern-tei-infra/docker-compose.yml up -d
Content type
Image
Digest
sha256:18b19c223…
Size
1.2 GB
Last updated
10 days ago
docker pull olkowa/pattern-tei-embed