toolboc/nv-jarvis
NVIDIA Jetson Embedded Device Support for https://github.com/microsoft/JARVIS
103
This image provides accelerated ffmpeg, pytorch, torchaudio, and torchvision dependencies.
Due to to memory requirements, JARVIS is required to run on Jetson AGX Orin family devices (64G on-board RAM device preferred) with config options set to:
inference_mode: local
local_deployment: standard
Models and configs are recommended to be provided through a volume mount from the host to the container as shown in the docker run
step below. It is possible to uncomment the # Download local models
section of the Dockerfile to build a container with models included.
# run the container which will automatically start the model server
docker run --name jarvis --net=host --gpus all -v ~/jarvis/configs:/app/server/configs -v ~/src/JARVIS/server/models:/app/server/models toolboc/nv-jarvis:r35.2.1
# (wait for model server to complete initialization)
# start awesome_chat.py
docker exec jarvis python3 awesome_chat.py --config configs/config.default.yaml --mode server
#start the web application (application will be acessible at http://localhost:9999)
docker exec jarvis npm run dev --prefix=/app/web
docker pull toolboc/nv-jarvis