Sign inSign up

nacrecontextlayer/connector-s3

By nacrecontextlayer

•Updated about 14 hours ago

Keeps the objects of an S3 bucket (AWS, MinIO, any S3 store) in Nacre layers, files as files.

Image
Machine learning & AI
Data science
Databases & storage
0

60

nacrecontextlayer/connector-s3 repository overview

⁠Nacre connector — s3

Keeps the objects of a bucket in Nacre⁠ layers — AWS S3, MinIO, or any store that speaks S3. An object is a document, its key under the prefix is the external_id, a key rule picks its layer, and an object that left the bucket is a document that leaves the index. An object's ETag is its version, so a sweep over a large bucket downloads only what changed.

docker run -v s3-state:/state \
  -e NACRE_URL=https://nacre.example.com -e NACRE_TOKEN=... \
  -e S3_BUCKET=corp-documents -e S3_PREFIX=handbook/ -e S3_REGION=eu-central-1 \
  -e 'S3_LAYERS=policies/**=handbook;engineering/**=engineering' \
  nacrecontextlayer/connector-s3:0.1.1

Credentials come through the AWS SDK's own chain — a static key pair, an instance role, IRSA, SSO — so a connector in a cluster needs no secret at all; S3_ENDPOINT points it at MinIO or another compatible store. Files go up as files: a PDF, a Word document, a spreadsheet, an EPUB is sent with its declared type and the index extracts the text, so what is searchable is the original and not a conversion. The variables are in the connector's README⁠.

⁠Every connector

…same as git…

⁠The product

…same as git… Tags: {version}, amd64 and arm64. ghcr.io/nacre-work/connectors/s3 is the canonical address; this repository is a mirror pushed by the same build at the same version.

nacre.work⁠ · All connectors⁠ · This connector's README⁠ · Nacre quickstart⁠ · Releases⁠

Tag summary

Content type

Image

Digest

sha256:e126febf3…

Size

60.8 MB

Last updated

about 14 hours ago

docker pull nacrecontextlayer/connector-s3:0.1.1