notheisz57/llamacpp-lfm25-8b-a1b repository overview

⁠llamacpp-lfm25-8b-a1b

docker run -it --rm --gpus all -p 8080:8080 -v /var/cache/llama-models:/models notheisz57/llamacpp-lfm25-8b-a1b:latest

Preset on notheisz57/llamacpp⁠ serving the ud-q8_k_xl quantization of unsloth/LFM2.5-8B-A1B-GGUF⁠.

Weights download on first run into the /models mount (resumable + namespaced) and load from cache thereafter.

OpenAI-compatible API on port 8080 (/v1/..., /health).

⁠Tags

• ud-q8_k_xl-b10472_cuda13 -- full pin
• ud-q8_k_xl -- recommended config at this quant
• q8 -- shorthand
• latest -- recommended overall

⁠Notes

• Extra llama-server args append to the preset: append them to the run command, e.g. --ctx-size 8192
• Replace the preset's baked args wholesale: -e LLAMA_EXTRA_ARGS="..."