dumitruiuga/bonsai-llama repository overview

--18.09.2026-- These files need our llama.cpp build The ternary hybrid-attention kernels live in the PrismML-Eng/llama.cpp fork. Stock llama.cpp will not run these files. It rejects PQ2_0 and PTQ1_0 as unknown types, and it loads Q2_0 without any warning and produces garbage, because it has no Hadamard activation runtime. Use a binary from the fork.

⁠prebuilt, pick the archive for your platform

⁠https://github.com/PrismML-Eng/llama.cpp/releases/latest⁠

tar -xzf llama--bin-.tar.gz -C bin --strip-components=1

⁠or build it

git clone https://github.com/PrismML-Eng/llama.cpp⁠ && cd llama.cpp cmake -B build -DGGML_CUDA=ON && cmake --build build -j # drop -DGGML_CUDA=ON on macOS, Metal is default

hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir .

./bin/llama-cli -m Ternary-Bonsai-2-27B-PQ2_0.gguf
-ngl 99 -fa on -c 32768
--temp 1.0 --top-p 0.95 --top-k 20
-p "Explain quantum computing in simple terms." -n 256

The binary is ./bin/llama-cli from an extracted release archive, or ./build/bin/llama-cli if you built the fork yourself. -ngl 99 offloads every layer, 0 is CPU-only; -c sets the context, up to 262144.