codecoradev/laya-service-rust repository overview

⁠laya-service-rust

The same decision API as codecoradev/laya-service⁠ — typed decisions over any state in one forward pass — served by a single Rust binary (axum + ONNX Runtime). No Python, no PyTorch, no HF download at boot.

~23 MB compressed image · 31 MB binary · CPU-only · stateless · Apache-2.0

Wire-compatible with the Python service: the same four endpoints (/v1/systemone, /api/v1/systemone, /api/v1/decide, /v1/decide), the same request/response contract, the same Jev-style errors. Verified live: 15/15 exact parity against the deployed Python service (10 ID+EN decisions byte-identical, plus 401/400/422 error shapes). Jev clients and existing laya-service integrations work by swapping image + bearer token only.

Built and maintained by CodeCora⁠ — pragmatic AI infrastructure.

⁠laya-service or laya-service-rust?

Pick this image for slim footprints, fast cold starts and no Python in your stack. Pick the Python image for the zero-config weight download and the API browser. Both are maintained together in the same project and share the contract.

⁠Quick start

This image is the binary-only build (~31 MB on distroless/cc, non-root): the ONNX checkpoint is not baked in — mount it at /model (read-only is fine). The checkpoint is the fused export layout: model.onnx, model.onnx.data, rl_agent_config.json, tokenizer/.

docker run -d --name laya-rust \
  -p 8080:8080 \
  -e BEARER_TOKEN=change-me \
  -v "$PWD/onnx/multilingual:/model:ro" \
  codecoradev/laya-service-rust:latest

BEARER_TOKEN is required — the service refuses to start without it (fail-closed). Check readiness (open endpoint, no auth):

curl http://localhost:8080/healthz
# {"status":"ok","checkpoint":"multilingual","device":"cpu"}

⁠Getting the checkpoint (one-time)

Export it from upstream Laya⁠ with tools/export_onnx.py (PyTorch 2.14.x, CPU wheels: --index-url https://download.pytorch.org/whl/cpu; the exporter upcasts the 644 MB fp16 checkpoint to a ~1.29 GB fp32 ONNX). Export takes ~5 min and is done once — reuse the resulting directory for every deployment (container mount, Modal Volume, etc.). Don't want that step? The Python image⁠ downloads weights from Hugging Face automatically.

On Modal: put the checkpoint directory on a Volume, mount it, and point LAYA_RUST_MODEL_DIR at it — the same binary serves unchanged.

⁠Ask a question

Four equivalent endpoints, one contract: /v1/systemone, /api/v1/systemone, /api/v1/decide, /v1/decide.

curl -s -X POST http://localhost:8080/v1/systemone \
  -H "Authorization: Bearer change-me" \
  -H 'Content-Type: application/json' \
  -d '{
    "state": {"body": "I was charged twice, refund now or I cancel!"},
    "questions": {
      "intent":   {"type": "choice", "instructions": "What does the customer want?",
                   "criteria": {"refund": "money back", "technical": "bug", "shipping": "delivery"}},
      "urgency":  {"type": "score",  "instructions": "How urgent?",
                   "criteria": ["routine", "soon", "critical"]},
      "escalate": {"type": "noul",   "instructions": "Needs a human now?"}
    }
  }'

model is optional — omit it (or send latest / jev-latest / jev-preview / multilingual) to use the loaded checkpoint. Response (same shape as the Python service, plus checkpoint / device / latency_ms / auto_threshold):

{
  "model": "multilingual",
  "answers": {
    "intent":   {"type": "choice", "choice": "refund",
                 "probabilities": {"refund": 1.0, "technical": 0.0, "shipping": 0.0},
                 "confidence": 1.0},
    "urgency":  {"type": "score", "score": 1.92,
                 "legend": {"0": "routine", "1": "soon", "2": "critical"},
                 "probabilities": {"0": 0.005, "1": 0.065, "2": 0.93},
                 "confidence": 0.75},
    "escalate": {"type": "noul", "noul": 0.46}
  },
  "usage": {"input_tokens": 123, "output_tokens": 0}
}

usage.input_tokens is the same chars÷4 estimate as Python; output_tokens is always 0 (non-autoregressive — no generated text). confidence is calibrated (RLCD): a common policy is auto-act at ≥ 0.85, escalate below (AUTO_THRESHOLD is echoed, policy stays in your caller).

⁠Environment variables

⁠Endpoints & errors

Errors are Jev-style: {"error": "…", "detail": …}. 401 wrong/missing bearer · 422 malformed body (pydantic-v2-style shape: type/loc/msg/input) · 400 semantically invalid (unknown model, wrong criteria shape) · 500 prediction failure · 503 server busy (all run slots taken, queue full) with a Retry-After header: retry the same request. Request bodies are capped at 2 MB. No /docs · /redoc · /openapi.json (that's a Python-service feature — the contract itself is identical).

⁠Measured performance

Per-call latency and throughput converge to the same model's compute — see the Python service page⁠ for the measured baseline (235–600 ms/call on CPU). Every response carries its own latency_ms.

⁠Tags

⁠Links

• Sibling image (Python) — same API, auto-downloads weights: https://hub.docker.com/r/codecoradev/laya-service⁠
• Upstream Laya (Apache-2.0 by Convai Innovations): https://github.com/NandhaKishorM/laya⁠
• CodeCora — the team behind this: https://codecora.dev⁠

⁠More from CodeCora

• Uteke⁠ — the remember-side of the same architecture: local-first semantic memory for AI agents (this service is the decide-side). Single Rust binary, ~45 ms recall, no cloud required.
• Cora Code⁠ — CLI-first AI code review (BYOK, pre-commit hooks, CI).

Apache-2.0. Laya is Apache-2.0 by Convai Innovations — this service only wraps it.