asusnw/llm repository overview

⁠OpenAI-Compatible API Endpoint (Dockerized)

This Docker container provides an OpenAI-compatible API endpoint.
It includes the following components:

⁠LLM Model

• SmolVLM-500M-Instruct-GGUF
  https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF⁠

⁠Inference Framework

• llama.cpp.python
  https://github.com/abetlen/llama-cpp-python⁠