dryearn/llama-cpp repository overview

⁠dryearn/llama-cpp

CUDA-enabled llama.cpp container built and validated on Jetson Orin Nano Super.

⁠Target platform

• Jetson Orin Nano Super
• JetPack 6.2
• L4T R36.4.3
• Ubuntu 22.04
• CUDA 12.6
• linux/arm64

⁠llama.cpp build

• b8235-ff52ee964
• Python bindings: llama-cpp-python 0.3.16

⁠Source

• Built from: https://github.com/dusty-nv/jetson-containers⁠
• Benchmarks and comparison repo: https://github.com/Horschig/Jetson_Orin_Nano_Super_Benchmark_Overview⁠

⁠Available tags

• b8235
• b8235-ff52ee964-jp6.2-arm64
• b8235-ff52ee964-l4t-r36.4.3-cu126-22.04-arm64
• b8235-jp6.2-arm64
• b8235-l4t-r36.4.3-cu126-22.04-arm64
• jetson-jp6.2-arm64
• l4t-r36.4.3-cu126-22.04-arm64

⁠Run

docker run --runtime nvidia --rm -it --network host   -v /path/to/models:/data   dryearn/llama-cpp:b8235-jp6.2-arm64 /bin/bash

⁠Notes

• Built natively on Jetson Orin Nano Super.
• Intended for Jetson linux/arm64 systems with NVIDIA container runtime configured.
• For performance comparisons, see the benchmark overview repository linked above.