The REST API server for ScaleLLM, the high-performance inference system for LLM.
10K+
The REST API Server for ScaleLLM. You can start the REST API gateway using the following command:
docker run -it --net=host \
docker.io/vectorchai/scalellm-gateway:latest --logtostderr
The REST API Server is available on localhost:8080. You can use REST API requests to interact with the system. Check out the Usage Examples for more details.
Content type
Image
Digest
sha256:fc35cb94a…
Size
75.6 MB
Last updated
over 1 year ago
docker pull vectorchai/scalellm-gateway