Sign inSign up

fn0000/fn-data-server

By fn0000

•Updated 6 months ago

On-site S3-compatible ZARR data server for microscopy imaging.

Image
Data science
Web servers
Databases & storage
0

2.0K

fn0000/fn-data-server repository overview

⁠Find Nuclei Data Server

On-site S3-compatible HTTP server for streaming OME-ZARR⁠ microscopy data to the Find Nuclei Viewer⁠.

Deploy on your lab network, NAS, or HPC cluster. The browser fetches ZARR chunks directly from this server — your data never leaves your infrastructure.

šŸ”¬ Viewer: find-nuclei.github.io⁠

⁠Quick Start

mkdir -p ~/.findnuclei

docker run -d \
  -p 3333:3333 \
  -p 3334:3334 \
  -v /path/to/your/zarr/data:/data/storage \
  -v ~/.findnuclei:/data/config \
  -e FN_ADMIN_PASSWORD=changeme \
  fn0000/fn-data-server

Two mount points: /data/storage for your ZARR data (read-only is fine), /data/config for the token database and signing key. Tokens persist across container restarts.

⁠View Your Data

Open the viewer with a direct URL to your data:

https://find-nuclei.github.io?url=http://localhost:3333/my-bucket/sample.zarr

If auth is enabled, add your token:

https://find-nuclei.github.io?url=http://localhost:3333/my-bucket/sample.zarr&token=YOUR_JWT

⁠Architecture

ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”     ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”     ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│  Find Nuclei     │     │  User's Browser  │     │  Data Server     │
│  Platform        │◄───►│  (Viewer)        │◄───►│  (Your Network)  │
│  metadata only   │     │                  │     │  ZARR chunks     │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜     ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜     ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜

The browser is the bridge. The data server only needs to be reachable from the user's browser (corporate network, VPN). The cloud platform never connects to it directly.

⁠Split Ports
PortProcessPurpose
3333Data ServerPure S3-compatible HTTP
3334Admin ConsoleToken management UI, health check, bucket browser

⁠Operation Modes

⁠Secure Mode (default)

All data endpoints require a JWT Bearer token. Create tokens via the admin console and scope them to specific buckets.

docker run -d \
  -e FN_AUTH_ENABLED=true \
  -e FN_ADMIN_PASSWORD=mypass \
  ...
  fn0000/fn-data-server
⁠Open Mode

No authentication on data endpoints. Use only on trusted networks.

docker run -d \
  -e FN_AUTH_ENABLED=false \
  ...
  fn0000/fn-data-server

⁠Token Scoping

Each JWT token is scoped to a bucket prefix:

  • my-project — access only objects under /my-project/
  • * — access all buckets (wildcard)

Tokens also support:

  • TTL — automatic expiration
  • Host restriction — limit to specific domains (supports wildcards: *.lab.internal)
  • IP restriction — limit to specific client IPs (supports wildcards: 192.168.1.*)

⁠Environment Variables

VariableDefaultDescription
FN_AUTH_ENABLEDtruetrue = secure mode, false = open mode
FN_ADMIN_PASSWORDadminAdmin console password
FN_DATA_PORT3333Data server port
FN_ADMIN_PORT3334Admin console port
FN_DATA_WORKERS4Worker processes (increase for more concurrent users)
FN_CORS_ORIGINS*Comma-separated allowed CORS origins

⁠Data Setup

Place your OME-ZARR datasets in the storage directory. Each top-level directory becomes a bucket:

/your/zarr/data/
ā”œā”€ā”€ project-alpha/
│   ā”œā”€ā”€ sample1.zarr/
│   └── sample2.zarr/
└── project-beta/
    └── experiment.zarr/

The admin console's bucket browser shows all top-level directories. Click a bucket to pre-fill the prefix when creating a token.

⁠Scaling

  • 1-5 users: Default config (4 workers) handles this comfortably
  • 5-20 users: Increase workers: -e FN_DATA_WORKERS=8
  • 20-50 users: Multiple containers behind a load balancer
  • 50+ users: Contact us about Enterprise deployment

⁠API Endpoints

⁠Data Server (port 3333)
MethodEndpointDescription
GET/List buckets (S3 XML)
GET/{bucket}List objects in bucket
GET/{bucket}/{key}Download object (supports Range headers)
HEAD/{bucket}/{key}Object metadata
PUT/{bucket}Create bucket
PUT/{bucket}/{key}Upload object
DELETE/{bucket}/{key}Delete object
⁠Admin Console (port 3334)
MethodEndpointDescription
GET/healthHealth check + server info
GET/adminToken management UI
GET/api/bucketsList buckets with item count
POST/admin/api/tokensCreate new token
POST/admin/api/tokens/{id}/revokeRevoke a token

⁠Platform Integration

  1. Start the data server on your network
  2. In Find Nuclei Platform → Settings → Data Sources, add your server URL and token
  3. Click "Test Connection" to verify
  4. Add images using paths like: http://your-server:3333/project-alpha/sample1.zarr

⁠License

MIT

Tag summary

Content type

Image

Digest

sha256:8aac7911b…

Size

24.4 MB

Last updated

6 months ago

docker pull fn0000/fn-data-server