oleksiyml/groq-chat repository overview

⁠groq-chat: Lightweight, High-Performance Groq and OpenAI compatible AI models CLI Chat

groq-chat is a blazing-fast command-line interface (CLI) for interacting with Groq and other OpenAI compatible AI models. Built in Go for maximum efficiency, it provides a simple one-shot chat experience with incredibly low resource usage and high-speed performance, even on older hardware.

Packaged as tiny Docker images (~9–11MB), this tool is ideal for fast deployment and minimal runtime environments.

⁠🚀 Key Features (Docker Edition)

• Tiny Footprint: Runs in ~9–11MB Docker containers, consuming only 11–14MB memory during usage.
• High Performance: Leverages Groq's ultra-fast inference for snappy terminal interaction.
• One-Shot Prompts: Designed for simple, stateless question-answer usage.
• Persistent Config & History: Mount a local directory to save config.yaml and chat history (.md files).
• Secure: Uses API_KEY environment variable for authentication.
• Fast & lightweight — runs even on older hardware.
• Interactive CLI — switch models, view history, and more.
• Response statistics — see token count, completion time, and tokens per second.
• Support for multiple AI providers with custom configuration

⁠🐳 Running with Docker

⁠Prerequisite

Obtain your GROQ_API_KEY from Groq⁠ and export it:

export GROQ_API_KEY=gsk_...

⁠Run the Container (Debian-based image ~9MB)

⁠Option 1: Inline API key

docker run --rm -it \
  -e GROQ_API_KEY=your_api_key \
  -v $HOME/.groq-chat:/root/.groq-chat \
  oleksiyml/groq-chat

⁠Option 2: Use exported env variable

export GROQ_API_KEY=your_api_key

docker run --rm -it \
  -e GROQ_API_KEY=$GROQ_API_KEY \
  -v $HOME/.groq-chat:/root/.groq-chat \
  oleksiyml/groq-chat

⁠Mount Options Explained

• --rm: Automatically remove container after exit.
• -it: Interactive terminal with TTY.
• -e GROQ_API_KEY=...: Inject API key.
• -v $HOME/.groq-chat:/root/.groq-chat: Mount persistent config & chat history directory.

⁠💬 Usage Inside Container

Once inside, you'll see the prompt.

🍎 One-shot Groq CLI chat
[i]nfo | [m]odel | [u]pdate | [h]istory ([s]ave:On/Off) | [c]onfig | [q]uit

[model_name] >

Interact via:

• [i] — Show current model info
• [m] — Switch model (up to 20 shown)
• [u] — Update models list from the current provider. Excluded models (part of the names) can be configured in config_<provider>.yaml (see below)
• [h] — Show history - list of the saved answers in Markdown format
• [s] — Toggle history saving (On/Off)
• [c] — Change config (it should be previously saved in config_<provider>.yaml)
• [q] — Quit

⁠⚠️ Important Notes

• One-Shot Mode: The CLI does not retain any history or prior inputs between queries. Make sure your prompts are clear, complete, and self-contained.

• Initial Setup: On first launch with volume mount, groq-chat will create config.yaml by fetching model data from the Groq API. Edit this config to:

  • Change your default model
  • Trim down the model list for better usability in [m] selection

⁠📦 Source Code

Full source and more instructions available on the GitHub repository⁠.

Built with Go, Docker, and ❤️ for fast, reliable CLI interaction with Groq, OpenAI, xAI, OpenRouter, Mistral AI and any other OpenAI compatible API.