An image for running GPULlama3.java with TornadoVM NVIDIA GPUs with PTX.
767
docker pull beehivelab/gpullama3.java-nvidia-openjdk-ptx:latest
git clone https://github.com/beehive-lab/docker-gpullama3.java
cd docker-gpullama3.java
wget https://huggingface.co/beehive-lab/Llama-3.2-1B-Instruct-GGUF-FP16/resolve/main/beehive-llama-3.2-1b-instruct-fp16.gguf
./dockerized-llama-tornado-nvidia-ptx \
--gpu --ptx --verbose-init \
--model beehive-llama-3.2-1b-instruct-fp16.gguf \
--prompt "tell me a joke"
Sample Output:
Here's one: What do you call a fake noodle? An impasta!
Content type
Image
Digest
sha256:3f8730d16…
Size
5.7 GB
Last updated
12 months ago
docker pull beehivelab/gpullama3.java-nvidia-openjdk-ptx