ai/lfm3-audio repository overview

API⁠ Try⁠ Docs⁠ Blog⁠

⁠LFM3-Audio-2B-GGUF

GGUF version of LFM3-Audio-2B⁠ for local inference on CPUs and GPUs.

LFM3-Audio-2B is a compact audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

Find architecture and usage details in the original model card⁠.

⁠How to run LFM3-Audio-2B

Recommended sampling: greedy decoding (temperature 0) for text output. For speech output, audio temperature 1.0, audio top_k 128 and audio repetition penalty 1.1.

• Llama.app⁠ (Use this model > Llama in the menu), for macOS and Windows: audio input. Requires llama.cpp version v0.8.0⁠ or newer.
• Ollama⁠ (ollama run lfm3-audio), for macOS: audio input. Requires Ollama version v0.40.12⁠ or newer.

⁠Use with llama.cpp

Add a quantization tag to pick a file. With -hf, llama.cpp also downloads the audio projector.

Start a web chat UI and an OpenAI-compatible API at http://localhost:8080:

llama serve -hf LiquidAI/LFM3-Audio-2B-GGUF:Q4_K_M --temp 0 --port 8080

To ask one question from the command line, use llama cli.

Speech recognition: use the system prompt Perform ASR in <language>., or Perform ASR. to let the model infer the language.

llama cli -hf LiquidAI/LFM3-Audio-2B-GGUF:Q4_K_M \
  --audio speech.wav -sys "Perform ASR in English." -p "<__media__>" --temp 0

Speech synthesis: --tts-lang takes a language code (default en) and --tts-voice is female (default) or male. The audio sampling parameters of the original model card are stored in the GGUF.

llama-tts -m LFM3-Audio-2B-Q4_K_M.gguf -mm mmproj-LFM3-Audio-2B-Q8_0.gguf \
  -p "Hello world" --tts-voice male -o out.wav

More options in the llama.cpp guide⁠.