hidakakombu/llama-cpp-python repository overview

⁠llama-cpp-python

This image enables the use of LLM in python with GPUs.
To use GPUs, NVIDIA Driver and NVIDIA Container Toolkit must be installed in advance.

Dockerfile : https://github.com/hidakakubo/Dockerfiles/blob/main/llama-cpp-python/Dockerfile⁠

Pull the image.

docker pull hidakakombu/llama-cpp-python

Run and enter the container.

docker container run -it --rm --gpus all -v /your/local/path/to/working/directory:/tmp hidakakombu/llama-cpp-python