NVIDIA Jetson Embedded Device Support for https://github.com/microsoft/JARVIS⁠

This image provides accelerated ffmpeg, pytorch, torchaudio, and torchvision dependencies.

Due to to memory requirements, JARVIS is required to run on Jetson AGX Orin family devices (64G on-board RAM device preferred) with config options set to:

• inference_mode: local
• local_deployment: standard

Models and configs are recommended to be provided through a volume mount from the host to the container as shown in the docker run step below. It is possible to uncomment the # Download local models section of the Dockerfile to build a container with models included.

Start the model server, awesomechat, and web app on Jetson Orin AGX

# run the container which will automatically start the model server
docker run --name jarvis --net=host --gpus all -v ~/jarvis/configs:/app/server/configs -v ~/src/JARVIS/server/models:/app/server/models toolboc/nv-jarvis:r35.2.1

# (wait for model server to complete initialization)

# start awesome_chat.py 
docker exec jarvis python3 awesome_chat.py --config configs/config.default.yaml --mode server

#start the web application (application will be acessible at http://localhost:9999)
docker exec jarvis npm run dev --prefix=/app/web