bogdan01m/laperf-cli repository overview

The goal of this project is to create an all-in-one source of information you need before buying your next laptop or PC for local AI tasks.

It’s designed for AI/ML engineers who prefer to run workloads locally — and for AI enthusiasts who want to understand real-world device performance.

⁠Table of Contents

• Overview⁠
• Philosophy⁠
• Benchmark Results⁠
• Quick Start⁠
• Contributing⁠

⁠Overview

⁠Tasks

La Perf is a collection of reproducible tests and community-submitted results for :

• ⁠Embeddings — ✅ Ready (sentence-transformers, IMDB dataset⁠)

  sts models:

  • modernbert-embed-base⁠

• ⁠LLM inference — ✅ Ready (LM Studio and Ollama, Awesome Prompts dataset⁠)

  llm models:

  • LM Studio: gpt-oss-20b⁠
    • macOS: mlx-community/gpt-oss-20b-MXFP4-Q8 (MLX MXFP4-Q8)
    • Other platforms: lmstudio-community/gpt-oss-20b-GGUF (GGUF)
  • Ollama: gpt-oss-20b⁠

• ⁠VLM inference — ✅ Ready (LM Studio and Ollama, Hallucination_COCO dataset⁠)

  vlm models:

  • LM Studio: Qwen3-VL-8B-Instruct⁠
    • macOS: lmstudio-community/Qwen3-VL-8B-Instruct-MLX-4bit (MLX 4-bit)
    • Other platforms: lmstudio-community/Qwen3-VL-8B-Instruct-GGUF-Q4_K_M (Q4_K_M)
  • Ollama: qwen3-vl:8b⁠
    • all platforms: qwen3-vl:8b (Q4_K_M)

• ⁠Diffusion image generation — 📋 Planned

• ⁠Speach to Text - 📋 Planned (whisper)

• ⁠Classic ML — 📋 Planned (scikit-learn, XGBoost, LightGBM, Catboost)

Note For mac-users: If it's possible prefer to use lmstudio with mlx backend, which gives 10-20% more performance then gguf. If you run ollama (by default benchmarks runs both lmstudio and ollama) then you'll see a difference between mlx and gguf formats.

The MLX backend makes the benchmark harder to maintain, but it provides a more realistic performance view, since it’s easy to convert a safetensors model into an mlx x-bit model.

⁠Requirements

La Perf is compatible with Linux, macOS, and Windows. For embedding tasks, 8 GB of RAM is usually sufficient. However for all tasks, it is recommended to have at least 16 GB, 18 GB is better, and 24 GB or more provides the best performance and reduces swap usage.

It’s designed to run anywhere the uv package manager is installed.

It’s recommended to use a GPU from NVIDIA, AMD, Intel, or Apple, since AI workloads run significantly faster on GPUs. Make sure to enable full GPU offload in tools like LM Studio or Ollama for optimal performance.

For embedding tasks, La Perf automatically detects your available device and runs computations accordingly.

⁠Benchmark Results

RPS - Requests Per Second (embeddings throughput)

TPS - Tokens Per Second (generation speed)

W - Watts (power consumption)

Efficiency metrics (RPS/W, TPS/W) are calculated using GPU power consumption

⁠⚡ Quick Start

For a full quickstart and setup instructions, please visit the La Perf documentation: Quickstart⁠.

⁠1. Clone the repository

git clone https://github.com/bogdanminko/laperf.git
cd laperf

⁠2. (Optional) Configure environment variables

La Perf works out of the box with default settings, but you can customize it for different providers:

cp .env.example .env
# Edit .env to change URLs, models, dataset sizes, etc.

See .env.example⁠ for all available options, including how to use custom OpenAI-compatible providers like vLLM, TGI, or LocalAI.

⁠3. Install dependencies (optional)

uv sync

This will:

• Create a virtual environment
• Install all required dependencies
• Set up the project for immediate use

⁠Running Your First Benchmark

⁠Run all benchmarks

Using make

make bench

Using uv

uv run python main.py

This will:

1. Auto-detect your hardware (CUDA / MPS / CPU)
2. Run all available benchmarks (all are pre-selected — you can toggle individual ones in the TUI using Space)
3. Save the results to results/report_{your_device}.json

⁠Running on GPU Servers (Docker)

For production deployments on cloud GPU instances or dedicated servers, you can use our Docker image:

⁠Pull the image

docker pull bogdan01m/laperf-cli:latest

⁠Run with NVIDIA GPU

docker run --gpus all -it --rm \
  -v $(pwd)/results:/app/results \
  bogdan01m/laperf-cli:latest

⁠Run with AMD ROCm

docker run --device=/dev/kfd --device=/dev/dri -it --rm \
  -v $(pwd)/results:/app/results \
  bogdan01m/laperf-cli:latest

⁠Run CPU-only

docker run -it --rm \
  -v $(pwd)/results:/app/results \
  bogdan01m/laperf-cli:latest

Note: Results will be saved to the mounted ./results directory on your host machine.

⁠Citation

If you use LaPerf in your research or reports, please cite it as follows:

BibTeX:

@software{laperf,
  author       = {Bogdan Minko},
  title        = {LaPerf: Local AI Performance Benchmark Suite},
  year         = {2025},
  url          = {https://github.com/bogdan01m/laperf},
  license      = {Apache-2.0},
  note         = {GitHub repository}
}