finwo/conductor repository overview

⁠conductor

A small CI server. Accepts triggers, schedules jobs, serves the worker API and hosts the application that drives it. One service, one image.

Source and issues: https://github.com/finwo/conductor⁠

⁠Tags

Built for linux/amd64, linux/arm64 and linux/riscv64.

0.10.0 closed several ways in. Workers are unaffected; the protocol they speak is unchanged.

• A project with no trigger secret no longer accepts anonymous triggers. Starting a build now takes a valid signature, or a session that may manage the project. Anything driving /trigger without either stops working, deliberately.
• A trigger secret starts builds and no longer reads them back. Reading a private job, task, log or artifact needs a session.
• visibility in .conductor.yml is refused, and a file still carrying it fails to compile. Visibility is a project setting: use PATCH /api/v1/projects/:id.
• repo_url must be https, ssh, git or an scp-style user@host:path. Other schemes select a git transport rather than a location and are rejected.
• Changing your own password now takes current_password alongside the new one. An administrator resetting someone else's does not.

0.9.0 changed how people sign in. Workers are unaffected; the protocol they speak is unchanged.

• The server rendered interface is gone. The image now serves a single page application that talks to /api/v1, and sessions are bearer tokens rather than a cookie. Everyone is signed out once on upgrade.
• CONDUCTOR_OIDC_CLIENT_SECRET no longer exists. An OIDC client must be reconfigured from confidential to public, with PKCE required, the conductor's origin in Web Origins, and that origin registered as a redirect URI. Leaving it confidential means nobody can sign in.
• CONDUCTOR_SESSION_TTL is replaced by CONDUCTOR_ACCESS_TTL and CONDUCTOR_REFRESH_TTL.

Upgrade the conductor and its workers together. Two releases have changed what a worker has to understand:

• 0.5.0 put every API response in an envelope: {"ok":true,"data":...} on success, {"ok":false,"error":{"name":...}} on failure. A worker older than 0.5.0 reads the claim response one level too high and will not run anything for a 0.5 or newer conductor.
• 0.3.0 made a job one run of a pipeline and a task the unit a worker runs. A 0.2.x worker cannot claim from a 0.3 or newer conductor.

⁠Quick start

docker run -d \n  --name conductor \n  -p 8080:8080 \n  -v conductor-data:/data \n  -e CONDUCTOR_PUBLIC_URL=http://localhost:8080 \n  finwo/conductor

The administrator password is printed once in the log on first start. Set CONDUCTOR_ADMIN_PASSWORD to choose it instead.

Tasks need a worker, which is a separate image: see finwo/conductor-worker⁠. A compose file running both is in the repository under deploy/.

⁠State

Everything lives in /data, so mount a volume there:

/data/conductor.db   sqlite, unless a database url is configured
/data/mirrors        one bare git mirror per project
/data/logs           the live log spool
/data/storage        artifacts and archived logs, unless S3 is configured

⁠Configuration

Every setting has a default and can come from the environment, so no configuration file is needed. Worth setting in production:

Generate the two secrets with openssl rand -hex 32.

⁠Signing in

The image serves the application and a login page for it.

localhost and 127.0.0.1 are different origins. An unregistered origin is refused, so list every one in use. * allows any, for local testing.

An external provider instead of the built-in page:

-e CONDUCTOR_OIDC_DISCOVERY_URL=https://idp.example.com/realms/ci/.well-known/openid-configuration \n-e CONDUCTOR_OIDC_CLIENT_ID=conductor \n-e CONDUCTOR_OIDC_ADMIN_ROLE=conductor-admin

Register conductor as a public client with PKCE. There is no client secret: the application redeems the code itself and the conductor never does. Only one provider is ever used, so configuring this turns the built-in login page off.

Postgres or MySQL instead of sqlite:

-e CONDUCTOR_DATABASE_URL=postgres://conductor:secret@postgres:5432/conductor

S3 compatible storage instead of the volume:

-e CONDUCTOR_S3_ENDPOINT=http://minio:9000 \n-e CONDUCTOR_S3_BUCKET=conductor \n-e CONDUCTOR_S3_ACCESS_KEY_ID=... \n-e CONDUCTOR_S3_SECRET_ACCESS_KEY=...

Artifacts and finished logs then go to the bucket, and downloads are handed over with a presigned redirect rather than proxied.

⁠Retention

Build output is deleted on a schedule, or a busy server fills its disk with logs nobody will read. The defaults keep artifacts for 30 days or the last 10 jobs, and logs for 14 days, and a project may override any of it. Upgrading an installation that has been running for a while will delete a lot on the first sweep, so set these before starting if that matters:

-e CONDUCTOR_RETENTION_ARTIFACT_DAYS=0 \n-e CONDUCTOR_RETENTION_LOG_DAYS=0

Zero means keep forever.

⁠Documentation

• Deployment⁠: images, state, production settings
• Authentication⁠: accounts, OIDC, who may see what
• Pipelines⁠: the .conductor.yml reference
• Workers⁠: running a worker
• HTTP API⁠: the JSON API and its credentials
• Contributing⁠: layout, testing, building

Overview and quick start: README⁠.