iverly/llamafile-docker repository overview

⁠llamafile-docker

⁠Introduction

This repository, llamafile-docker, automates the process of checking for new releases of Mozilla-Ocho/llamafile, building a Docker image with the latest version, and pushing it to Docker Hub.

⁠Usage

You will have to download a pre-trained model using the gguf format. You can find some on hugging face⁠. Please refer to the llamafile⁠ documentation for more information or report an issue if you need help.

⁠Server version (default)

⁠Prerequisites

• Docker
• A gguf pre-trained model

⁠Using the image

docker run -it --rm \
-p 8080:8080 \
-v /path/to/gguf/model:/model \
iverly/llamafile-docker:latest

The server will be listening on port 8080 and expose an ui to interact with the model.

Please refer to the llamafile⁠ documentation the available endpoints.

⁠CLI version

⁠Prerequisites

• Docker
• A gguf pre-trained model

⁠Using the image

docker run -it --rm \
-v /path/to/gguf/model:/model \
iverly/llamafile-docker:latest --cli -m /model -p {prompt}

You will see the output of the model in the terminal.

⁠Contributing

Contributions are welcome. Please follow the standard Git workflow - fork, branch, and pull request.

⁠License

This project is licensed under the Apache 2.0 - see the LICENSE file for details.