Run text-generation-webui compiled for CPU
618
*** THIS IS NOT OFFICIAL *** I'm not associated with oobabooga, if you have any problems with it feel free to message me at https://sh.itjust.works/u/noneabove1182, do not bother them on their own github please :)
Runs text-generation-webui for CPU ONLY hosted on port 7860, and sets the server.py file as the entrypoint so you can pass any args to it you'd like, I have my default ones below, please edit to fit your setup!
I use the following setup:
docker-compose.yml:
version: '3.8'
services:
text-generation-webui:
container_name: text-generation-webui
restart: always
volumes:
- '${MODEL_DIR}:/app/models'
ports:
- '7860:7860'
image: 'noneabove1182/text-gen-ui-cpu:latest'
ulimits:
memlock: -1
mem_limit: 30gb
command:
[
"--model",
"${MODEL}",
"--chat",
"--cpu",
"--threads",
"6",
"--listen",
"--listen-port",
"7860"
]
.env:
MODEL_DIR=/home/user/models/
MODEL=Manticore-13B.ggmlv3.q4_1.bin
Content type
Image
Digest
sha256:bed6972c6…
Size
1.4 GB
Last updated
over 3 years ago
docker pull noneabove1182/text-gen-ui-cpu