openfintechlab/paytrace-file-ingest repository overview

⁠PayTrace File Ingest CSV

PayTrace File Ingest CSV is a worker service for file-based payment initiation. It watches a CSV inbox, claims stable files, validates payment rows against the PayTrace payment instruction schema, records processing state in PostgreSQL, and publishes valid payment instructions to RabbitMQ.

Docker Hub image:

docker pull openfintechlab/paytrace-file-ingest:latest

Docker Hub page:

https://hub.docker.com/r/openfintechlab/paytrace-file-ingest⁠

⁠What It Does

• Watches OFTL_FWCSV_ROOTDIR/inbox for .csv files
• Moves stable files into processing/ using claim semantics
• Streams CSV rows without loading full files into memory
• Validates rows using src/domain/payment_instruction.schema.json
• Publishes valid rows to domestic or cross-border RabbitMQ request queues
• Persists file, checkpoint, and row-dispatch state in PostgreSQL
• Publishes EV001 files.csv.loaded when a file completes
• Publishes EV002 files.csv.row.failed for source-row failures
• Archives successful files under archive/YYYY/MM/DD/
• Moves failed files to error/

⁠Tags

The build pipeline publishes:

• latest
• commit SHA tags
• branch-derived tags such as main, develop, or release-*

⁠Run

docker run --rm \
  --name paytrace-file-ingest \
  --env-file .env \
  -e OFTL_FWCSV_ROOTDIR=/app/fwcsv \
  -v /path/to/fwcsv:/app/fwcsv \
  openfintechlab/paytrace-file-ingest:latest

This is a worker process, not an HTTP API. It does not expose application routes or health endpoints from the container.

⁠Required Services

• PostgreSQL
• RabbitMQ
• A mounted CSV working directory

Expected CSV directory layout under OFTL_FWCSV_ROOTDIR:

fwcsv/
  inbox/
  processing/
  archive/
  error/

⁠Important Configuration

Database:

OFTL_POSTGRESDB_USERNAME=admin
OFTL_POSTGRESDB_PASSWORD=[CHANGE ME]
OFTL_POSTGRESDB_HOST=postgres
OFTL_POSTGRESDB_PORT=5432
OFTL_POSTGRESDB_NAME=public
OFTL_POSTGRESDB_SCHEMA=public

CSV watcher:

OFTL_FWCSV_ROOTDIR=/app/fwcsv
OFTL_FWCSV_SCAN_INTERVAL_SECONDS=10
OFTL_FWCSV_STABILITY_WINDOW_SECONDS=5
OFTL_FWCSV_WORKER_COUNT=2
OFTL_FWCSV_CHECKPOINT_EVERY_ROWS=1000
OFTL_FWCSV_FILE_ENCODING=utf-8

RabbitMQ:

OFTL_RABITMQ_HOST=rabbitmq
OFTL_RABITMQ_PORT=5672
OFTL_RABITMQ_USERNAME=guest
OFTL_RABITMQ_PASSWORD_SECRET=[CHANGE ME]
OFTL_RABITMQ_PUBEVENT_EXCHANGE=paytrace.events
OFTL_RABITMQ_PUBEVENT_EV001=files.csv.loaded
OFTL_RABITMQ_PUBEVENT_EV002=files.csv.row.failed
OFTL_RABITMQ_DOEMSTIC_REQUEST_QUEUE=CSV.PAYMENTS.DOMESTIC.REQ
OFTL_RABITMQ_CROSS_BORDER_REQUEST_QUEUE=CSV.PAYMENTS.CROSS_BORDER.REQ

Note: preserve the existing OFTL_RABITMQ_DOEMSTIC_REQUEST_QUEUE spelling for compatibility.

⁠Exit Behavior

• Exits with code 99 if RabbitMQ is unavailable after configured retries
• Exits with code 91 for unexpected startup/runtime errors
• Exits with code 0 on user-requested shutdown

⁠Build Locally

docker build -t openfintechlab/paytrace-file-ingest:latest .

Optional base image overrides:

docker build \
  --build-arg DOCKER_PYTHON_BUILDER_IMAGE=dhi.io/python:3-debian13-sfw-dev \
  --build-arg DOCKER_PYTHON_RUNTIME_IMAGE=dhi.io/python:3 \
  -t openfintechlab/paytrace-file-ingest:latest .

⁠License

Copyright 2026-2030 OpenFintechlab, Inc.