sapbot/llamacpp repository overview

⁠llama.cpp

llama.cpp is lightweight inference framework, supporting latest LLM.

⁠What this container is?

This is prebuilt llama.cpp bundled with:

• LFM2.5 1.2B
• Gemma 3 4B Why them? Because both is lightweight but one for speed (LFM), one for quality (Gemma).

This container includes CPU and Vulkan backend, so you can use it with your GPU.

⁠USAGE

sudo docker run -p 8080:8080 sapbot/llamacpp

With GPUs:

sudo docker run -p 8080:8080 --gpus all sapbot/llamacpp

It's recommended to tweak this command for your setup.

⁠Exposed ports

The only exposed port is 8080, where is webui and api. Just add -p 8080:8080 in running command before container name.

⁠WARNING

Credit for LFM2.5, llama.cpp and Gemma 3 goes to their authors, I only builded container.