install ollama on jetson agx orin. CUDA 12.2, nvidia-smi 540.3.0, jetpack: 6.0, L4T: 36.3.0
377
This is temp image for ollama to run on jetson agx orin with CUDA 12.2 and L4T 36.3.0 as there is a bug with missing /data/logs
to run this image run command:
docker run -d --runtime nvidia -v [mount]/home/ollama:/root/.ollama -e OLLAMA_MODELS=/root/.ollama -p 11434:11434 --name ollama vndr/ollama:r36.3.0
To update container, you have to build it by yourself as at this time L4T 36.2.0 version does not work for me:
- git clone https://github.com/dusty-nv/jetson-containers.git
- follow instructions to install jetson containers
- cd jetson-containers/packages/llm/ollama
- modify config.py with version needed
- do not forget to update Dockerfile, as there is a bug with missing log directories
- run: jetson-containers build ollama
- when image done, test it: docker run -d --runtime nvidia -v /home/ollama:/root/.ollama -e OLLAMA_MODELS=/root/.ollama -p 11434:11434 --name ollama ollama:r36.3.0
Content type
Image
Digest
sha256:799c089e5…
Size
5.4 GB
Last updated
over 1 year ago
docker pull vndr/ollama:r36.3.0-cu122