migrationimages/huggingface_hub repository overview

Docker image with Python 3.12 and huggingface_hub python package pre installed.

This is useful if you want to separate downloading models from your code and centralize how you store and cache them.

The image is pretty small because huggingface_hub doesn't rely on transformers.

⁠Use case

Say you have a backend application that depends on a small model to be present to run (if it's something bigger you are better off using something like vllm, but that added complexity is not always needed).

Downloading the models as part of server startup is a problem, as it makes the backend responsible for locating the files and managing their cache.
It also makes scaling harder. What if you want to launch multiple replicas of the backend and the model's weren't previously downloaded? All the instances would have to pay for the downloading cost, and even worse it might corrupt the data because of the concurrent downloads.

⁠What to do instead

Just run the download command from the image in separate container and make the backend depend on it before running, something akin to k8s init containers.
You can achieve this in docker compose with the following:

services:
  download-models:
    image: migrationimages/huggingface_hub
    command: [ "hf", "download", "stabilityai/stable-diffusion-xl-base-1.0", "--cache-dir", "/some/path/you/want" ]
    volumes:
      - models:/some/path/you/want

  backend:
    build:
      context: .
    environment:
      - MODEL_SAVE_DIR=/backend/models
    volumes:
      - models:/backend/models
    depends_on:
      download-models:
        condition: service_completed_successfully

volumes:
  models: