More Docker. Easy Access. New Streamlined Plans. Learn more.
localai logo

localai/localai

Sponsored OSS

By localai

Updated 34 minutes ago

LocalAI is the free, Open Source OpenAI alternative

Image
Machine Learning & AI
Gen AI
61

500K+



LocalAI

LocalAI forksLocalAI starsLocalAI pull-requests

LocalAI Docker hubLocalAI Quay.io

Follow LocalAI_APIJoin LocalAI Discord Community

Star us at Github! https://github.com/mudler/LocalAI

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by Ettore Di Giacinto.

Run LocalAI with docker:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu
# Alternative images:
# - if you have an Nvidia GPU:
# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12
# - without preconfigured models
# docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
# - without preconfigured models for Nvidia GPUs
# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12 

To load models:

# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-cpu run llama-3.2-1b-instruct:q4_k_m

# Start LocalAI with the phi-2 model directly from huggingface
... run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# Install and run a model from the Ollama OCI registry
... run ollama://gemma:2b
# Run a model from a configuration file
... run https://gist.githubusercontent.com/.../phi-2.yaml
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
... run oci://localai/phi-2:latest

💻 Getting started

Docker Pull Command

docker pull localai/localai