goshops/bge repository overview

⁠BGE-M3 Embedding API

A FastAPI application that exposes the BGE-M3 (BAAI General Embedding - Multi-lingual, Multi-functionality, Multi-granularity) model for generating three types of embeddings.

⁠Features

BGE-M3 is a versatile embedding model that supports:

1. Dense Embeddings (1024-dimensional vectors)

   • Traditional semantic embeddings for similarity search
   • Best for general-purpose semantic matching

2. Sparse Embeddings (Lexical weights)

   • Token-level weights similar to BM25
   • Good for keyword-based matching and hybrid search

3. ColBERT Multi-Vectors

   • Multiple vectors per text for fine-grained matching
   • Enables token-level similarity with higher precision

⁠Installation

⁠Option 1: Local Installation

pip install -r requirements.txt

⁠Option 2: Docker (Recommended for Production)

# Using Docker Compose
docker-compose up -d

# Or using Docker CLI
docker build -t bge-m3-api .
docker run -d -p 8001:8001 --name bge-m3-api bge-m3-api

See DOCKER.md⁠ for detailed Docker deployment instructions, GPU support, and production configurations.

⁠Running the Server

⁠Local

python3 main.py

⁠Docker

docker-compose up -d

The server will start on http://localhost:8001 and automatically download the BGE-M3 model on first run.

⁠Automatic Device Detection

The API automatically detects and optimizes for your hardware:

• CUDA GPU: Uses GPU acceleration with FP16 for maximum performance
• Apple Silicon (MPS): Uses Metal Performance Shaders with FP16 optimization
• CPU: Falls back to CPU with FP32 for compatibility

Check the startup logs or /health endpoint to see which device is being used.

⁠API Endpoints

⁠Health Check

GET /health

Returns server and model status.

Response:

{
  "status": "healthy",
  "model_loaded": true,
  "model_name": "BAAI/bge-m3",
  "device": "mps",
  "use_fp16": true
}

⁠Dense Embeddings Only

POST /embed/dense

Generate only dense embeddings (most common use case).

Request:

{
  "texts": ["Your text here", "Another text"],
  "batch_size": 12,
  "max_length": 8192
}

Response:

{
  "embeddings": [[...1024 floats...], [...]],
  "num_texts": 2,
  "embedding_dim": 1024,
  "model_name": "BAAI/bge-m3"
}

⁠Sparse Embeddings Only

POST /embed/sparse

Generate only sparse embeddings for keyword-based matching.

Request:

{
  "texts": ["Your text here"],
  "batch_size": 12,
  "max_length": 8192
}

Response:

{
  "embeddings": [{"token_id": weight, ...}],
  "num_texts": 1,
  "model_name": "BAAI/bge-m3"
}

⁠ColBERT Multi-Vector Embeddings

POST /embed/colbert

Generate multi-vector embeddings for fine-grained matching.

Request:

{
  "texts": ["Your text here"],
  "batch_size": 12,
  "max_length": 8192
}

Response:

{
  "embeddings": [[[...], [...], ...]],
  "num_texts": 1,
  "num_vectors_per_text": [15],
  "model_name": "BAAI/bge-m3"
}

⁠All Embeddings Combined

POST /embed/all

Generate all three types of embeddings in a single call.

Request:

{
  "texts": ["Your text here"],
  "batch_size": 12,
  "max_length": 8192
}

Response:

{
  "dense_embeddings": [[...]],
  "sparse_embeddings": [{"token": weight}],
  "colbert_embeddings": [[[...]]],
  "num_texts": 1,
  "dense_dim": 1024,
  "num_colbert_vectors": [15],
  "model_name": "BAAI/bge-m3"
}

⁠Flexible Embedding Endpoint

POST /embed

Generate any combination of embeddings.

Request:

{
  "texts": ["Your text here"],
  "return_dense": true,
  "return_sparse": true,
  "return_colbert": false,
  "batch_size": 12,
  "max_length": 8192
}

Response:

{
  "dense_embeddings": [[...]] or null,
  "sparse_embeddings": [{...}] or null,
  "colbert_embeddings": [[...]] or null,
  "num_texts": 1,
  "model_name": "BAAI/bge-m3"
}

⁠Testing

Run the test suite to verify all endpoints:

python3 test_api.py

⁠Example Usage

⁠Python

import requests

# Dense embeddings for semantic search
response = requests.post(
    "http://localhost:8001/embed/dense",
    json={
        "texts": ["Machine learning is amazing", "Deep learning uses neural networks"],
        "max_length": 512
    }
)
embeddings = response.json()["embeddings"]

# All three embedding types
response = requests.post(
    "http://localhost:8001/embed/all",
    json={"texts": ["Multi-functional embeddings"]}
)
data = response.json()
dense = data["dense_embeddings"]
sparse = data["sparse_embeddings"]
colbert = data["colbert_embeddings"]

⁠cURL

# Health check
curl http://localhost:8001/health

# Dense embeddings
curl -X POST http://localhost:8001/embed/dense \
  -H "Content-Type: application/json" \
  -d '{"texts": ["Hello world"], "max_length": 512}'

⁠Model Information

• Model: BAAI/bge-m3
• Max Input Length: 8192 tokens
• Dense Embedding Dimension: 1024
• Supported Languages: 100+
• Use Cases: Semantic search, information retrieval, hybrid search, cross-lingual retrieval

⁠Performance Notes

• First request may be slower as the model is loaded into memory
• Using use_fp16=True for faster inference with minimal quality loss
• Batch processing is supported for multiple texts
• Model weights are cached after first download

⁠API Documentation

Once the server is running, visit:

• Interactive API docs: http://localhost:8001/docs⁠
• Alternative docs: http://localhost:8001/redoc⁠

⁠References

• BGE-M3 on Hugging Face⁠
• FlagEmbedding GitHub⁠