ai/llama3.2

Docker

Solid LLaMA 3 update, reliable for coding, chat, and Q&A tasks

ai/llama3.1

Docker

Meta’s LLama 3.1: Chat-focused, benchmark-strong, multilingual-ready.

ai/gemma3-vllm

Docker

Google’s latest Gemma, small yet strong for chat and generation

ai/smollm3

Docker

SmolLM3 is a 3.1B model for efficient on-device use, with strong performance in chat

aistaging/gemma3

aistaging

Google’s latest Gemma, small yet strong for chat and generation

Q4_K_M GGUF of a MedGemma health-chat fine-tune for local, private clinical AI prototyping.

Step-3.5-Flash Q3_K_M GGUF for Docker Model Runner (chat)

GLM-4.5-Air Q4_K_M GGUF for Docker Model Runner (chat)

MiniMax-M2.5 UD-Q2_K_XL GGUF for Docker Model Runner (chat)

DeepSeek-V4-Flash UD-IQ3_XXS GGUF for Docker Model Runner (chat)

EuroLLM-22B-Instruct Q4_K_M GGUF for Docker Model Runner (chat)

Qwen2.5-32B-Instruct Q4_K_M GGUF for Docker Model Runner (chat)

Olmo 3.1 32B Think Q4_K_M GGUF for Docker Model Runner (chat)

Hermes 4.3 36B Q4_K_M GGUF for Docker Model Runner (chat)

KAT-Coder-V2.5-Dev Q4_K_M GGUF for Docker Model Runner (chat)

Qwen-AgentWorld-35B-A3B UD-Q4_K_M GGUF for Docker Model Runner (chat)

Huihui Qwen3.8-27B abliterated UD-DW-Q4_K_M GGUF for Docker Model Runner (chat)

GLM-Z1-32B-0414 Q4_K_M GGUF for Docker Model Runner (chat)

DeepSeek-R1-Distill-Qwen-32B Q4_K_M GGUF for Docker Model Runner (chat)

LFM2.5-24B-A2B Q4_K_M GGUF for Docker Model Runner (chat)

GLM-4-32B-0414 Q4_K_M GGUF for Docker Model Runner (chat)

Qwen 3.6 35B A3B GGUF quantizations from Unsloth Studio with the Nail chat template.

Mellum2-12B-A2.5B-Instruct Q8_0 GGUF for Docker Model Runner (chat)

Olmo 3.1 32B Instruct Q4_K_M GGUF for Docker Model Runner (chat)

GLM-4-9B-0414 Q4_K_M GGUF for Docker Model Runner (chat)

North Mini Code 1.0 UD-Q4_K_M GGUF for Docker Model Runner (chat)

ERNIE-4.5-21B-A3B-Thinking Q4_K_M GGUF for Docker Model Runner (chat)

LFM2.5-8B-A1B Q4_K_M GGUF for Docker Model Runner (chat)

DeepSeek-Coder-V2-Lite-Instruct Q4_K_M GGUF for Docker Model Runner (chat)