aminediro/ferrules-api-gpu repository overview

Ferrules API Server - GPU Edition

High-performance document parsing API server with NVIDIA GPU acceleration. Convert PDFs to LLM-ready documents at blazing speed using Rust and CUDA.

⁠Quick Start

docker run -p 3002:3002 --gpus all aminediro/ferrules-api-gpu

⁠Overview

Ferrules is a modern, opinionated document parsing library written in Rust that generates LLM-ready documents efficiently. This Docker image provides the HTTP API server with NVIDIA GPU acceleration for maximum performance.

⁠Key Features

• 🚀 High Performance: Built with Rust for maximum speed and efficiency
• 🎯 GPU Accelerated: Leverages NVIDIA CUDA for accelerated ML inference
• 📄 Advanced PDF Parsing: Extracts text, layouts, and structure from PDFs
• 🧠 ML-Powered: Uses SoA ML models for layout detection and document understanding
• 🔄 Multiple Formats: Outputs to JSON, HTML, and Markdown
• 🛠️ Single binary - Zero Dependencies: No Python runtime required
• 🌐 RESTful API: Easy integration into existing systems

⁠Requirements

• Docker with NVIDIA Container Toolkit installed
• NVIDIA GPU with CUDA support (12.3+ recommended)
• Minimum 4GB GPU memory recommended

⁠Usage

⁠Basic Usage

Start the API server on default port 3002:

docker run -p 3002:3002 --gpus all aminediro/ferrules-api-gpu

⁠Custom Port

Run on a different port:

docker run -p 8080:3002 --gpus all aminediro/ferrules-api-gpu

⁠With Environment Variables

Configure the server with environment variables:

docker run -p 3002:3002 --gpus all \
  -e FERRULES_DEBUG=true \
  aminediro/ferrules-api-gpu

⁠Docker Compose

version: '3.8'
services:
  ferrules-api:
    image: aminediro/ferrules-api-gpu
    ports:
      - "3002:3002"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

⁠API Endpoints

Once running, the API server provides the following endpoints:

• POST /parse - Parse a PDF document
• GET /health - Health check endpoint
• GET /version - Get API version

For detailed API documentation, visit the API Documentation⁠.

⁠Configuration

The container runs with the following default configuration:

• CUDA Acceleration: Enabled (--cuda)
• Intra-threads: 2 (-j2)
• Optimization Level: 3 (-O3)

⁠Performance

This GPU-accelerated version provides:

• Up to 10x faster processing compared to CPU-only solutions
• Parallel processing of multiple documents
• Hardware-accelerated ML inference for layout detection

⁠Tags

• latest - Latest stable release with GPU support
• x.y.z - Specific version releases

⁠Support

• GitHub: https://github.com/aminediro/ferrules⁠
• Issues: https://github.com/aminediro/ferrules/issues⁠
• Documentation: https://github.com/aminediro/ferrules/blob/main/README.md⁠

⁠License

See the LICENSE⁠ file in the GitHub repository.

Built with 🦀 Rust for maximum performance and reliability.