gencore/llama-cli repository overview

⁠Overview

This project will provide the llama-cli utility as a Docker image that has built in support for multiple CPU architectures. Both Linux amd64 and MacOS arm64 are supported.

⁠Why does this project exist?

Users who want llama.cpp can just download it from the github releases page⁠.

The main thing this project adds is a convenient way to use the llama-cli binary from within Docker in a multi-arch format. This allows projects that want to package LLMs inside Docker containers to also be multi-arch.

⁠Building

To build this project, use the provided Makefile:

make build

⁠Run

Then once the image is build, we can run it as follows:

docker run --rm -ti gencore/llama-cpp-builder:latest --help