Sign inSign up

mmoskal/llgtrt

By mmoskal

•Updated almost 2 years ago

HTTP server with constrained output and OpenAI-compatible REST APIs using TensorRT-LLM

Image
Machine learning & AI
0

457

mmoskal/llgtrt repository overview

Follow instructions at https://github.com/guidance-ai/llgtrt/blob/main/README.md⁠

Tag summary

Content type

Image

Digest

sha256:7e78d5d44…

Size

15.4 GB

Last updated

almost 2 years ago

docker pull mmoskal/llgtrt:prod-0.14.2