dbinvent/yt repository overview

⁠YT Cluster: PostgreSQL where every node takes writes

One YT Cluster node in one container: PostgreSQL 18 (the official postgres:18⁠ image) with the YT Cluster extension, and ytserv beside it, which reaches PostgreSQL over its unix socket. Start one, or three with Docker Compose, and every node takes writes: a plain COMMIT is a cluster commit. Same SQL, same drivers, no proxy in front.

• Guide: Running in Docker⁠
• Product: dbinvent.com/cluster⁠ - downloads⁠ (deb, EL9 rpm, static binaries)
• Tests: DBinvent/yt-cluster-tests⁠, run nightly; an image is published only from a build that passed every suite

⁠Tags

x86-64, PostgreSQL 18.

⁠A single node

docker run -d --name yt1 --hostname yt1 \
    -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=app \
    -p 5441:5432 -v yt1:/var/lib/postgresql dbinvent/yt:1.20

docker exec yt1 psql -U postgres -c "select * from yt_info('o')"   # Term: 1, ..., Leader

Everything the official image takes still works: POSTGRES_PASSWORD, POSTGRES_DB, the scripts in /docker-entrypoint-initdb.d, the volume at /var/lib/postgresql. On its first start the node sets itself up in seconds - yt.conf before PostgreSQL first starts for good, so nothing restarts; the extension in postgres and in POSTGRES_DB; the roster and the cluster's uuid; the tables to replicate - then ytserv starts, and starts again 5 seconds after it ever exits.

⁠A cluster of three

compose.yml:

x-yt: &yt
  image: dbinvent/yt:1.20
  restart: unless-stopped
  environment: &yt-env
    POSTGRES_PASSWORD: secret
    POSTGRES_DB: app
    YT_PEERS: yt1 yt2 yt3
    # the cluster's uuid, the same on every node - make one with: uuidgen
    YT_CLUSTER: 4f6c2b9e-8d1a-4c3e-9b7f-2a5d6e8c1f30

services:
  yt1:
    <<: *yt
    hostname: yt1
    ports: ["5441:5432"]
    volumes: [yt1:/var/lib/postgresql, ./schema.sql:/docker-entrypoint-initdb.d/schema.sql:ro]
  yt2:
    <<: *yt
    hostname: yt2
    ports: ["5442:5432"]
    volumes: [yt2:/var/lib/postgresql, ./schema.sql:/docker-entrypoint-initdb.d/schema.sql:ro]
  yt3:
    <<: *yt
    hostname: yt3
    ports: ["5443:5432"]
    volumes: [yt3:/var/lib/postgresql, ./schema.sql:/docker-entrypoint-initdb.d/schema.sql:ro]

volumes:
  yt1:
  yt2:
  yt3:

schema.sql, which each node runs on its first start:

create table orders (
    id uuid primary key default gen_random_uuid(),
    item text not null,
    node text not null default current_setting('yt.node'),
    created timestamptz not null default now()
);

docker compose up -d
docker compose exec yt1 psql -U postgres -d app -c "insert into orders (item) values ('from yt1')"
docker compose exec yt2 psql -U postgres -d app -c "insert into orders (item) values ('from yt2')"
docker compose exec yt3 psql -U postgres -d app -c "select item, node from orders order by created"

Each node has every row, whichever node it was written on, and a node going down stops nothing: the others go on taking writes, and it catches up when it is back.

• The same YT_PEERS and YT_CLUSTER on every node. A node's id is derived from its name in YT_PEERS, its hostname, so the name must not change.
• Every replicated table has a primary key, and keys written on different nodes do not collide (a serial counts on its own node only): use uuid keys.
• Port 7421 is ytserv's, between the nodes; applications use 5432.

⁠Settings

Read on a node's first start only; after that, the node is what its database says.

⁠Adding a node

Add a yt4 service whose own environment puts itself in the roster (environment: { <<: *yt-env, YT_PEERS: yt1 yt2 yt3 yt4 }), start it, and write its roster row on the leader - the other nodes refuse it, so the loop stops there:

docker compose up -d yt4
U4=$(docker compose exec -T yt4 yt-node uuid)
for n in yt1 yt2 yt3; do
  docker compose exec -T $n psql -U postgres -c \
    "insert into public.yt_config (name, module, value) values ('$U4', 'N', 'yt4:7421')" && break
done

The new node copies the replicated tables from a member, replays what was written since, and takes writes within seconds - while every node keeps writing.

⁠Data

The volume at /var/lib/postgresql holds the node: PostgreSQL's data directory and ytserv's working directory (the cluster WAL, the node's key). A restarted container, or a new one on the same volume, is the same node. ytctl, the setup helper of a server install, is in the image for anything else (docker exec yt1 ytctl --help); the image does not need it.

A schema change applies to the node it runs on: run it on each node in turn, for instance with dbinvent/sge⁠.

⁠License

YT Cluster is dual-licensed: free for noncommercial use under the PolyForm Noncommercial License 1.0.0⁠, and a paid license for commercial use - see Licensing⁠. The image carries the license in /usr/share/doc/yt-cluster/copyright. PostgreSQL and the base image are under their own licenses.