byzatic/tessera-data-flow-engine repository overview

⁠Tessera Data Flow Engine (Tessera-DFE)

⁠Official Docker Images

• Release 0.1.2: View on Docker Hub⁠
• Branch main: View on Docker Hub⁠
• Latest (from main): View on Docker Hub⁠

Version pinning can be done either:

• by pom.xml version number (e.g., 0.1.2), or
• by using latest / main.

All other tags are considered developer builds, including for example 0.1.2-SNAPSHOT.

A modular execution engine based on Directed Acyclic Graphs (DAGs) designed to build flexible and extensible data processing pipelines.

Project repository: Tessera-DFE on GitHub⁠
License: Apache-2.0⁠

⁠Quick Start

docker run -d --name tessera-dfe   -p 8080:8080   -e GRAPH_CALCULATION_CRON_CYCLE="*/10 * * * * *"   -e XMS=20m   -e XMX=8192m   -e DATA_DIR_WATCH_INTERVAL=5   -v $PWD/configurations/:/app/configurations/   -v $PWD/data/source/:/app/data/source_zip   -v $PWD/logs/:/app/logs/   byzatic/tessera-data-flow-engine:latest

⁠Environment Variables

• GRAPH_CALCULATION_CRON_CYCLE — cron schedule for graph recalculation (6 fields, first is seconds).
• XMS, XMX — JVM heap size parameters.
• DATA_DIR_WATCH_INTERVAL — polling interval (in seconds) for the data directory.

⁠Tag Policy

Automated via GitHub Actions:

• sha-<short> — image built from a specific commit (first 7 chars of SHA). Published for every push/PR.
• <branch> — image tagged with the branch name (slashes replaced with dashes). Published for every push/PR.
• latest, main, and <pom-version> — published only for the main branch. Version is taken from pom.xml. SNAPSHOT versions are blocked on main.

⁠Automatic Cleanup

• On branch deletion: Docker tag with the same name is deleted.
• Daily at 03:23 UTC: cleanup of orphaned feature-* tags if no corresponding branch exists.
• Daily at 03:17 UTC: cleanup of sha-* tags if the commit no longer exists.

⁠CI/CD Workflow

• Builds are run on ubuntu-latest using quay.io/buildah/stable.
• Tags created:
  • always: sha-<short>, <branch>
  • for main: latest, main, <pom-version> (from pom.xml)
• Images are pushed to: docker.io/byzatic/tessera-data-flow-engine

⁠Example docker-compose

version: '3.7'
services:
  tessera-dfe:
    image: byzatic/tessera-data-flow-engine:latest
    container_name: tessera-data-flow-engine
    ports:
      - "8080:8080"
    volumes:
      - ./configurations/:/app/configurations/
      - ./data/source/:/app/data/source_zip
      - ./logs/:/app/logs/
    environment:
      - GRAPH_CALCULATION_CRON_CYCLE=*/10 * * * * *
      - XMS=20m
      - XMX=8192m
      - DATA_DIR_WATCH_INTERVAL=5
    logging:
      driver: json-file
      options:
        max-file: "10"
        max-size: "10m"

⁠License

This project is licensed under the Apache License 2.0⁠.