hellohal2064/cca-functionary-router repository overview

⁠CCA Functionary Router

A fast request classification router for CCA (Confucius Code Agent)⁠. Uses the Functionary v3.2 small model to classify incoming requests and route them to the appropriate CCA expert.

⁠What it does

• Receives incoming user requests via OpenAI-compatible API
• Classifies them using tool-calling to determine which CCA expert should handle the request (Coder, Search, Infrastructure, etc.)
• Runs on minimal GPU memory (~4GB) alongside other services

⁠Built for ARM64 + CUDA

This image is built from source for ARM64 + CUDA (NVIDIA DGX Spark GB10 SM120/121). The upstream ghcr.io/ggml-org/llama.cpp:server-cuda is x86-only, so this custom build is required.

Built on top of llama.cpp⁠ server.

⁠Quick Start

docker run -d \
  --gpus all \
  --name functionary-router \
  -v /data/models/functionary:/models:ro \
  -p 8001:8001 \
  hellohal2064/cca-functionary-router:latest

⁠Model Required

• functionary-small-v3.2.Q4_0.gguf (~4GB) from meetkai/functionary-small-v3.2-GGUF⁠
• Place in your models directory under functionary/

⁠API

• Port: 8001
• Endpoint: POST /v1/chat/completions (OpenAI-compatible with tool_calls)
• Health: GET /health

⁠Part of CCA Stack

This is one of 11 containers in the CCA full stack. See:

• CCA agent⁠
• Full deployment guide⁠