docdbrown/granite-4-tiny repository overview

Non-GPU Version for fast CPU inference (Linux with > 8GB Ram, nore ram is better)

7B parameters total, 1B active parameters. Designed for low latency and local applications, particularly where the task has long prefill or other scenarios where a MoE model is desired. e.g. edge inference.

Endpoint: http://localhost:8084/v1/chat/completions⁠

Repo: https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF⁠