notheisz57/llamacpp-gemma4-12b-it repository overview

⁠llamacpp-gemma4-12b-it

docker run -it --rm --gpus all -p 8080:8080 -v /var/cache/llama-models:/models notheisz57/llamacpp-gemma4-12b-it:latest

Preset on notheisz57/llamacpp⁠ serving the ud-q8_k_xl-mtp quantization of unsloth/gemma-4-12b-it-GGUF⁠.

Weights download on first run into the /models mount (resumable + namespaced) and load from cache thereafter.

OpenAI-compatible API on port 8080 (/v1/..., /health).

⁠Tags

• ud-q8_k_xl-mtp-b10472_cuda13 -- full pin
• ud-q8_k_xl-mtp -- this config, rolling across llama.cpp releases
• ud-q8_k_xl -- recommended config at this quant
• q8 -- shorthand
• latest -- recommended overall

⁠Notes

• Extra llama-server args append to the preset: append them to the run command, e.g. --ctx-size 8192
• Replace the preset's baked args wholesale: -e LLAMA_EXTRA_ARGS="..."