This image is based on pytorch 2.1.0-cuda12.1-cudnn8-runtime.
Requirement: Nvidia video card with 13G vram at least.
expose port 8000 for api.
git lfs install
git clone https://huggingface.co/THUDM/chatglm3-6b
sudo docker run -it -d --name chatglm3_6b_api \
-v /path/to/models/chatglm3-6b:/workspace/THUDM/chatglm3-6b \
--gpus all \
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility \
-e NVIDIA_VISIBLE_DEVICES=all \
-p 8000:8000 \
blodely/chatglm3:1.0.0
replace /path/to/models/chatglm3-6b with actual path to file(ChatGLM3 model project).
Rick
December 18, 2023
Content type
Image
Digest
sha256:7c4b30468…
Size
5.4 GB
Last updated
over 2 years ago
docker pull blodely/chatglm3:1.0.0