Sign inSign up

cabreraevil/nsfw-detector

By cabreraevil

•Updated 10 months ago

Image
0

658

cabreraevil/nsfw-detector repository overview

⁠NSFW Image Detector

A professional, production-ready NSFW (Not Safe For Work) image detection API using the Hugging Face Falconsai/nsfw_image_detection model. Built with FastAPI, containerized with Docker, and optimized for scalability.

⁠Features

  • REST API - FastAPI-based REST API with automatic documentation
  • Multiple Input Methods - Classify images from:
    • File uploads (multipart/form-data)
    • URLs
    • Batch processing (up to 10 URLs)
  • Production Ready
    • Structured logging with structlog
    • Health checks and metrics endpoints
    • Docker containerization with multi-stage builds
    • Proper error handling and validation
    • CORS support
  • Developer Friendly
    • Type hints throughout
    • CLI tool for testing
    • Comprehensive test suite
    • Code formatting and linting with Ruff
    • Managed with mise for reproducible environments

⁠Project Structure

nsfw-detector/
├── app/
│   ├── __init__.py
│   ├── main.py                 # FastAPI application
│   ├── api/
│   │   ├── __init__.py
│   │   └── routes.py          # API endpoints
│   ├── core/
│   │   ├── __init__.py
│   │   ├── config.py          # Configuration management
│   │   └── detector.py        # NSFW detection logic
│   ├── models/
│   │   ├── __init__.py
│   │   └── schemas.py         # Pydantic models
│   └── utils/
│       ├── __init__.py
│       └── logger.py          # Structured logging
├── tests/
│   ├── __init__.py
│   └── test_api.py            # API tests
├── scripts/
│   └── classify_cli.py        # CLI tool
├── config/                     # Configuration files
├── docker/                     # Docker-related files
├── logs/                       # Application logs
├── .mise.toml                  # Mise configuration
├── docker-compose.yml          # Docker Compose config
├── Dockerfile                  # Multi-stage Docker build
├── requirements.txt            # Python dependencies
├── requirements-dev.txt        # Development dependencies
├── pyproject.toml             # Project metadata and tools
├── Makefile                   # Convenience commands
└── README.md                  # This file

⁠Quick Start

  1. Clone and start the service:
git clone <repository-url>
cd nsfw-detector
docker-compose up -d
  1. Check health:
curl http://localhost:8000/api/v1/health
  1. Classify an image:
curl -X POST "http://localhost:8000/api/v1/classify/url" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/image.jpg"}'
⁠Using mise (Development)
  1. Install mise:

Follow instructions at mise.jdx.dev⁠

  1. Setup environment:
cd nsfw-detector
mise install          # Install Python 3.11
mise run install      # Install dependencies
  1. Run development server:
mise run dev
⁠Manual Setup
  1. Create virtual environment:
python -m venv .venv
source .venv/bin/activate  # Linux/Mac
# or
.venv\Scripts\activate     # Windows
  1. Install dependencies:
pip install -r requirements.txt
  1. Run the server:
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

⁠API Documentation

Once the server is running, visit:

⁠Endpoints
⁠POST /api/v1/classify

Classify a single image uploaded as multipart/form-data.

Request:

curl -X POST "http://localhost:8000/api/v1/classify" \
  -F "file=@/path/to/image.jpg"

Response:

{
  "label": "nsfw",
  "score": 0.9876,
  "raw": [
    {"label": "nsfw", "score": 0.9876},
    {"label": "normal", "score": 0.0124}
  ]
}
⁠POST /api/v1/classify/url

Classify an image from a URL.

Request:

curl -X POST "http://localhost:8000/api/v1/classify/url" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/image.jpg"}'

Response:

{
  "label": "safe",
  "score": 0.9912,
  "raw": [
    {"label": "normal", "score": 0.9912},
    {"label": "nsfw", "score": 0.0088}
  ]
}
⁠POST /api/v1/classify/batch

Classify multiple images from URLs (max 10).

Request:

curl -X POST "http://localhost:8000/api/v1/classify/batch" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://example.com/image1.jpg",
      "https://example.com/image2.jpg"
    ]
  }'

Response:

{
  "results": [
    {
      "image": "https://example.com/image1.jpg",
      "label": "safe",
      "score": 0.9912,
      "raw": [...],
      "error": null
    },
    {
      "image": "https://example.com/image2.jpg",
      "label": "nsfw",
      "score": 0.9234,
      "raw": [...],
      "error": null
    }
  ]
}
⁠GET /api/v1/health

Health check endpoint.

Response:

{
  "status": "healthy",
  "version": "1.0.0",
  "model_loaded": true
}
⁠GET /api/v1/metrics

Get service metrics.

Response:

{
  "total_requests": 42,
  "total_images": 45,
  "nsfw_count": 12,
  "safe_count": 33,
  "average_processing_time": 0.1234
}

⁠CLI Tool

Use the CLI tool for quick testing:

# Single image
python scripts/classify_cli.py /path/to/image.jpg

# Multiple images
python scripts/classify_cli.py image1.jpg image2.jpg image3.jpg

# JSON output
python scripts/classify_cli.py --json /path/to/image.jpg

⁠Configuration

Configuration is managed via environment variables or .env file. Copy .env.example to .env and adjust:

# Application Settings
APP_NAME=NSFW Image Detector
ENVIRONMENT=production
DEBUG=False

# API Settings
API_HOST=0.0.0.0
API_PORT=8000
MAX_UPLOAD_SIZE=10485760  # 10MB

# Model Settings
MODEL_NAME=Falconsai/nsfw_image_detection
MODEL_CACHE_DIR=./model_cache
DEVICE=cpu  # Use 'cuda' for GPU acceleration

# Logging
LOG_LEVEL=INFO
LOG_FORMAT=json

# Workers
WORKERS=1

⁠Development

⁠Install development dependencies:
pip install -r requirements-dev.txt
⁠Run tests:
pytest tests/ -v
# or
make test
⁠Code formatting:
ruff format app/ tests/
# or
make format
⁠Linting:
ruff check app/ tests/
# or
make lint

⁠Docker Commands

# Build image
make docker-build
# or
docker build -t nsfw-detector:latest .

# Start services
make docker-up
# or
docker-compose up -d

# View logs
make docker-logs
# or
docker-compose logs -f

# Stop services
make docker-down
# or
docker-compose down

⁠Scalability

⁠Horizontal Scaling

Use Docker Compose or Kubernetes to run multiple instances:

# docker-compose.scale.yml
services:
  nsfw-detector:
    # ... existing config ...
    deploy:
      replicas: 3

Run with:

docker-compose up -d --scale nsfw-detector=3
⁠Load Balancing

Add nginx or traefik for load balancing multiple instances.

⁠GPU Support

For GPU acceleration, update the Dockerfile to use a CUDA base image and set DEVICE=cuda:

FROM nvidia/cuda:11.8.0-runtime-ubuntu22.04
⁠Production Considerations
  • Use a reverse proxy (nginx, traefik)
  • Add rate limiting
  • Implement authentication/authorization
  • Use Redis for distributed caching
  • Monitor with Prometheus + Grafana
  • Use managed databases for metrics
  • Set up log aggregation (ELK, Loki)

⁠Performance

Typical processing times (CPU):

  • Single image: ~0.5-2 seconds (first request)
  • Subsequent images: ~0.1-0.5 seconds (model cached)

GPU acceleration can reduce processing time by 5-10x.

⁠License

MIT License - see LICENSE file for details.

⁠Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Run tests and linting
  5. Submit a pull request

⁠Support

For issues and questions:

  • Open an issue on GitHub
  • Check the API documentation at /docs

⁠Acknowledgments

Tag summary

Content type

Image

Digest

sha256:9cb90e80b…

Size

2.7 GB

Last updated

10 months ago

docker pull cabreraevil/nsfw-detector