docdbrown/trinity-nano repository overview

Arcee trinity-nano

6B MoE model with 1B active parameters. Best for CPU inference

Endpoint: http://localhost:8085/v1/chat/completions⁠

Repo: https://huggingface.co/arcee-ai/Trinity-Nano-Preview-GGUF⁠

For CPU inference, the more RAM the better.

Q6KL_AMD allows Linux (min 8gb ram context=16384 approx 12288 words)

Q6KL_ARM and MacOS (8gb ram macbook pro context=16384 approx 12288 words)

Q2 is Linux only - made to test CPU with limited RAM (eg oracle always free tier, context=512 approx 390 words)

curl -sS http://127.0.0.1:8085/v1/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "trinity-nano",
    "prompt": "Write one sentence explaining what a transformer is.",
    "temperature": 0.2,
    "max_tokens": 80
  }'