samueltallet/alpine-llama-cpp-server repository overview

Alpine LLaMA is an ultra-compact Docker image (less than 14 MB), providing a LLaMA.cpp⁠ HTTP server for language model inference.

‍

⁠Use cases

This Docker image is particularly suited for:

• Environments with limited disk space or low bandwidth.
• Servers that cannot do GPU-accelerated inference, e.g. a CPU-only VPS or a Raspberry Pi.

⁠Quick start

You can host an HTTP inference server who leverages the LFM2.5 350M⁠ language model with:

docker run --name alpine-llama -p 80:8080 -e LLAMA_API_KEY=sk-xxxx -e LLAMA_ARG_MODEL_URL=https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF/resolve/main/LFM2.5-350M-Q4_K_M.gguf samueltallet/alpine-llama-cpp-server

Once the GGUF model file is downloaded (and cached in the Docker container filesystem), you can query the OpenAI-compatible Chat Completions API endpoint exposed.

For example, you can classify the sentiment of a feedback with:

curl -s http://127.0.0.1/v1/chat/completions \
  -H 'Authorization: Bearer sk-xxxx' \
  -d '{
    "messages": [
      { "role": "user", "content": "Classify as exactly one word (Positive, Neutral, or Negative) the sentiment of this feedback: This application doesn''t work for all scenarios, but I think it has potential." }
    ],
    "temperature": 0,
    "max_tokens": 2
  }' | jq '.choices[0].message.content'
# > "Neutral"

Notes for the above script:

• If you run docker remotely, replace 127.0.0.1 with your server IP.
• In production, use your own strong secret key instead of sk-xxxx.
• Install jq⁠ with sudo apt install jq on Debian-based systems.

⁠More examples

See the GitHub repository README⁠ for more examples (structured output and summarization).

⁠Configuration

You can pass environment variables to the Docker container to configure the Alpine LLaMA server:

An exhaustive list of these variables can be found in the official LLaMA.cpp server documentation⁠.

⁠License

Project licensed under MIT. See the LICENSE⁠ file for details.

⁠Copyright

© 2026 Samuel Tallet