make a working directory:
mkdir nv_ft_inference
cd nv_ft_inference
launch the image:
docker run -it --rm --gpus=all --shm-size=8G --network host -v $(pwd):/ft_workspace -p 8888:8888 smellslikeml/triton_with_ft:22.03-main-new bash
Notes:
May be beneficial to fiddle with the -shm-size arg
Run the following notebook with Jupyter lab:
cd /ft_workspace
git clone https://github.com/triton-inference-server/fastertransformer_backend.git
git checkout -b t5_gptj_blog remotes/origin/dev/t5_gptj_blog
jupyter lab --ip 0.0.0.0 --allow-root
navigate to ./fastertransformer_backend/notebooks/GPT-J_and_T5_inference.ipynb
Content type
Image
Digest
sha256:a1d9e7c6c…
Size
9.8 GB
Last updated
almost 4 years ago
docker pull smellslikeml/triton_with_ft:22.03