mnccouk/ollama-gpu-rx580 repository overview

Before running make sure your docker host has the Rocm(5.7.1) libraries installed and are working with your Radeon RX580. see - https://rocm.docs.amd.com/en/docs-5.7.0/deploy/linux/os-native/install.html⁠ to install Rocm libraries on your docker host machine

To verify, use the following command. You should see details about your graphics card.

rocminfo

To run

docker run -e HIP_PATH=/opt/rocm/lib/ -e LD_LIBRARY_PATH=/opt/rocm/lib --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama_gpu mnccouk/ollama-gpu-rx580:latest

Once the container is running to start a chat session using the llama3.1 model

docker exec -it ollama_gpu ollama run llama3.1

To stop the running container

docker container stop ollama_gpu

To start the container

docker container start ollama_gpu