Sign inSign up

mlachgar/rawlens-enterprise-api

By mlachgar

•Updated 6 months ago

Image
0

195

mlachgar/rawlens-enterprise-api repository overview

⁠RawLens Enterprise API

RawLens Enterprise API is the self-hosted backend for enterprise asset ingestion, search, preview generation, metadata processing, and image classification.

It is designed to run alongside PostgreSQL, OpenSearch, and a mounted source filesystem that contains the assets to scan.

⁠What It Provides

  • Read and write HTTP APIs for assets, search, and jobs.
  • Static bearer-token authentication with role-based access control.
  • A background worker that processes queued jobs.
  • PostgreSQL persistence for assets, jobs, metadata, previews, and job state.
  • OpenSearch indexing for searchable asset metadata.
  • Vision Forge integration for image classification.

⁠Deployment

Deployment assets and runtime packaging helpers live here.

Use docker-compose.yml as the only Compose entrypoint for the API stack with Postgres, OpenSearch, and Vision Forge as separate containers.

The current Compose file is:

services:
  postgres:
    image: postgres:16
    container_name: rawlens-enterprise-api-postgres
    environment:
      POSTGRES_DB: rawlens
      POSTGRES_USER: rawlens
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    ports:
      - "127.0.0.1:5434:5432"
    volumes:
      - postgres-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U rawlens -d rawlens"]
      interval: 5s
      timeout: 5s
      retries: 10
      start_period: 10s

  opensearch:
    image: opensearchproject/opensearch:2.16.0
    container_name: rawlens-enterprise-api-opensearch
    environment:
      discovery.type: single-node
      plugins.security.disabled: "true"
      OPENSEARCH_JAVA_OPTS: -Xms512m -Xmx512m
      OPENSEARCH_INITIAL_ADMIN_PASSWORD: ${OPENSEARCH_INITIAL_ADMIN_PASSWORD:-N7!qL4@p9Z#t2Xv8}
    ports:
      - "127.0.0.1:9201:9200"
    volumes:
      - opensearch-data:/usr/share/opensearch/data
    restart: unless-stopped

  vision-forge:
    image: ${VISION_FORGE_IMAGE:-mlachgar/vision-forge-api:cpu-full}
    container_name: rawlens-enterprise-api-vision-forge
    platform: ${VISION_FORGE_PLATFORM:-linux/amd64}
    env_file:
      - .env
    environment:
      VISION_FORGE_CONFIG_DIR: /config
      VISION_FORGE_DATA_DIR: /data
      VISION_FORGE_DEVICE: ${VISION_FORGE_DEVICE:-cpu}
    ports:
      - "127.0.0.1:8001:8000"
    volumes:
      - ./data:/data
    restart: unless-stopped

  api:
    build:
      context: ../..
      dockerfile: rawlens-enterprise-api/Dockerfile
    container_name: rawlens-enterprise-api
    env_file:
      - .env
    depends_on:
      postgres:
        condition: service_healthy
      opensearch:
        condition: service_started
      vision-forge:
        condition: service_started
    environment:
      RAWLENS_API__SERVER__HOST: 0.0.0.0
      RAWLENS_API__SERVER__PORT: 3000
      RAWLENS_API__POSTGRES__URL: postgres://rawlens:${POSTGRES_PASSWORD}@postgres:5432/rawlens
      RAWLENS_API__POSTGRES__MAX_CONNECTIONS: 10
      RAWLENS_API__OPENSEARCH__URL: http://opensearch:9200
      RAWLENS_API__OPENSEARCH__INDEX_PREFIX: rawlens
      RAWLENS_API__VISION_FORGE__URL: http://vision-forge:8000
      RAWLENS_API__LOGGING__LEVEL: info
    ports:
      - "127.0.0.1:3002:3000"
    volumes:
      - rawlens-enterprise-api-data:/tmp/rawlens-enterprise-api
    restart: unless-stopped

volumes:
  postgres-data:
  opensearch-data:
  rawlens-enterprise-api-data:

The Compose file publishes the API, Postgres, Vision Forge, and OpenSearch on loopback-bound host ports by default, which makes the stack safer to run on a VPS that already has other containers listening on standard ports.

The same Compose stack can optionally run Vision Forge for image classification. This stack keeps its own Vision Forge runtime data in ./data⁠. You can point VISION_FORGE_IMAGE at a published image and seed the matching API key in /data/api_keys.json. Vision Forge manages its own configuration internally; the compose stack only mounts ./data. The RawLens Enterprise API expects RAWLENS_API__VISION_FORGE__API_KEY to contain the plaintext predictor token from that file, which is vfk_predict_token_87654321fedcba in the seeded sample stack.

The API also expects RAWLENS_API__AUTH__TOKENS to contain a JSON array of bearer tokens with roles such as reader, writer, and admin.

By default, the API listens on 127.0.0.1:3002, Postgres on 127.0.0.1:5434, Vision Forge on 127.0.0.1:8001, and OpenSearch on 127.0.0.1:9201.

Set POSTGRES_PASSWORD in ./.env to control both the Postgres container password and the API's database connection string.

For Compose-based local development, keep the runtime variables in ./.env⁠ beside the compose file.

Public registration and upload classification now live in ../rawlens-access-api/deploy/docker-compose.yml⁠ with its own deployment .env file.

Tag summary

Content type

Image

Digest

sha256:068b17b31…

Size

34.6 MB

Last updated

6 months ago

docker pull mlachgar/rawlens-enterprise-api