Pipeline Copilot converts plain-English prompts into secure Apache Hop (.hpl) pipelines.
358
AI-powered natural-language → Apache Hop pipeline generator. Describe what you want ("read a CSV of customer orders and load the active ones into Postgres") and get back a working .hpl pipeline, streamed live and checked against a security allow-list before it's returned.
Source & full docs: https://github.com/Speedykom/pipeline-copilot
docker pull meddhiakassab/pipeline-copilot:0.1.0
Fastest way to try it is with the docker-compose.yml from the repo (it also runs Postgres and migrations for you):
git clone https://github.com/Speedykom/pipeline-copilot.git
cd pipeline-copilot
cp .env.example .env # fill in GROQ_API_KEY and KEYCLOAK_* at minimum
docker compose up
The app listens on http://localhost:8000 (/health for a liveness check, /docs for the OpenAPI UI).
Running the image standalone instead of via compose? Migrations don't run automatically — apply them once against your target Postgres first:
docker run --rm --env-file .env meddhiakassab/pipeline-copilot:0.1.0 /app/.venv/bin/alembic upgrade head
docker run --rm --env-file .env -p 8000:8000 meddhiakassab/pipeline-copilot:0.1.0
All configuration is via environment variables (--env-file .env or docker-compose.yml's environment:/env_file:).
| Variable | Required | Default | Purpose |
|---|---|---|---|
GROQ_API_KEY | yes (with default model) | — | API key for the LLM provider actually in use |
PIPELINE_GENERATOR_MODEL | no | groq/llama-3.3-70b-versatile | Any LiteLLM <provider>/<model> string — swap to openai/... or anthropic/... and set that provider's key instead |
OPENAI_API_KEY / ANTHROPIC_API_KEY | only if used | — | Set the one matching PIPELINE_GENERATOR_MODEL |
DATABASE_URL | yes | — | Postgres connection string, must use the asyncpg driver |
KEYCLOAK_SERVER_URL | yes | — | Base URL of the Keycloak server issuing bearer tokens |
KEYCLOAK_REALM | yes | — | Keycloak realm name |
KEYCLOAK_AUDIENCE | yes | — | Expected aud claim on verified tokens (needs an audience mapper on the Keycloak client) |
KEYCLOAK_JWKS_CACHE_SECONDS | no | 300 | How long fetched JWKS signing keys are cached |
LOG_LEVEL | no | info | debug / info / warning / error / critical |
CORS_ALLOW_ORIGINS | no | * | Comma-separated allowed origins, or * |
HOST_PORT | no (compose only) | 8000 | Host port the container's 8000 is published on |
POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DB | no (compose only) | pipeline_copilot | Credentials for the bundled postgres compose service — keep DATABASE_URL in sync if you change these |
CREWAI_DISABLE_TELEMETRY | no | unset | Set true to disable CrewAI's anonymous telemetry |
Session endpoints (/api/sessions*) require a valid Keycloak bearer token and return 503 without Keycloak configured — there is no unauthenticated fallback.
app, uid/gid 1000), overridable at build time with --build-arg APP_UID=... --build-arg APP_GID=....HEALTHCHECK (GET /health), so docker ps and orchestrators that respect container health reflect service status automatically.crewai[litellm] dependency tree (chromadb, onnxruntime, pyarrow, lancedb, etc.).Apache-2.0 — see the repo for full license text and source.
Content type
Image
Digest
sha256:ddff39404…
Size
346.2 MB
Last updated
3 months ago
docker pull meddhiakassab/pipeline-copilot