finalend4/ollama-converter repository overview

Contained scripts:

1. Download a LLM model (safetensors) from Hugging Face (HF)
2. Convert HF models to GGUF format with llama.cpp
3. Quantize GGUF model with desired type (e.g. Q4_K_M)
4. Import quantized GGUF model into Ollama
5. Push imported model to Ollama registry

When the environment variables of the docker container are set, another script will instead automatically execute the whole pipeline for optionally multiple quantization types on one HF model.

Environment:

• HF_SOURCE: Hugging Face repository, e.g. "google/gemma-2-2b"
• QUANTS: List of quantization types to perform, e.g. "Q4_K_M,Q6_K,Q8_0"
• REGISTRY_USER: Username for the Ollama registry