Sign inSign up

fourplayers/denokv

By fourplayers

•Updated 18 days ago

A self-hosted backend for Deno KV

Image
Databases & storage
0

484

fourplayers/denokv repository overview

⁠fourplayers/denokv

A self-hosted backend for Deno KV⁠, built from the 4Players fork of denoland/denokv⁠.

Compared to upstream, this fork adds two distributed storage backends (FoundationDB and ScyllaDB), a RocksDB backend, read coalescing, and a built-in migration path between all backends. All backends speak the same KV Connect protocol, so client code never changes.

⁠Quick start

docker run -it --init -p 4512:4512 -v ./data:/data \
  fourplayers/denokv --sqlite-path /data/denokv.sqlite serve --access-token <random-token>

Then connect from Deno, with the access token in DENO_KV_ACCESS_TOKEN:

const kv = await Deno.openKv("http://localhost:4512");

The entrypoint is the denokv binary; serve is the default command. Every flag can also be given as an environment variable (DENO_KV_*).

⁠Storage backends

Exactly one backend must be selected via its data-location flag:

BackendSelect withBest for
SQLite--sqlite-path <file>Single instance; S3 replication and point-in-time recovery
RocksDB--rocksdb-path <dir>Single instance, write-heavy workloads
ScyllaDB--scylladb-address <host:port>Horizontal scaling: many stateless servers on one keyspace
FoundationDB--fdb-cluster-file <file>Horizontal scaling with native transactions, watches and queues
⁠SQLite
docker run -it --init -p 4512:4512 -v ./data:/data \
  fourplayers/denokv --sqlite-path /data/denokv.sqlite serve --access-token <token>

Flags: --num-workers (DENO_KV_NUM_WORKERS, default 1), --atomic-write-batch-timeout-ms (DENO_KV_ATOMIC_WRITE_BATCH_TIMEOUT_MS).

⁠RocksDB
docker run -it --init -p 4512:4512 -v ./data:/data \
  fourplayers/denokv --rocksdb-path /data/denokv-rocksdb serve --access-token <token>

Flags: --rocksdb-write-buffer-mb (default 64), --rocksdb-block-cache-mb (default 128), --rocksdb-max-background-jobs (default 4).

⁠ScyllaDB

Any number of denokv containers can point at the same keyspace and act as one database. Writes serialize through LWT lock stripes in ScyllaDB, so check-and-set semantics and monotonic versionstamps hold across servers.

docker run -it --init -p 4512:4512 \
  fourplayers/denokv --scylladb-address 10.0.0.1:9042 serve --access-token <token>

Flags (DENO_KV_SCYLLADB_*):

  • --scylladb-keyspace (default denokv)
  • --scylladb-replication-factor (default 1; use 3 in production)
  • --scylladb-buckets (default 1024; frozen at keyspace creation)
  • --scylladb-locality-depth (default 2; frozen at keyspace creation)

Limitations: no queues, no S3 replication/PITR, only U64 sum/min/max.

⁠FoundationDB

Any number of denokv containers can attach to the same FoundationDB cluster. Every atomic write is one serializable FDB transaction and versionstamps are FDB commit versionstamps; there is no coordination point, watches are push-based, and queues work across servers.

docker run -it --init -p 4512:4512 -v ./fdb.cluster:/etc/foundationdb/fdb.cluster:ro \
  fourplayers/denokv --fdb-cluster-file /etc/foundationdb/fdb.cluster serve --access-token <token>

Flags: --fdb-prefix (DENO_KV_FDB_PREFIX, default denokv) — acts like a keyspace; several logical databases can share one cluster.

Limitations: no --read-only, no S3 replication/PITR.

⁠Read coalescing

--read-coalescing (DENO_KV_READ_COALESCING, off by default, any backend) merges identical concurrent snapshot reads into a single backend execution while preserving strong consistency. Useful when many clients issue the same query at the same time.

⁠Migrating between backends

serve can import an existing database on startup, before it opens its own store. SQLite is the hub: sqlite <-> rocksdb, sqlite <-> scylladb, sqlite <-> fdb, each in both directions.

# SQLite -> FoundationDB
docker run -it --init -p 4512:4512 -v ./data:/data -v ./fdb.cluster:/etc/foundationdb/fdb.cluster:ro \
  fourplayers/denokv --fdb-cluster-file /etc/foundationdb/fdb.cluster serve \
  --migrate-from-sqlite /data/denokv.sqlite --access-token <token>

Flags: --migrate-from-sqlite <file>, --migrate-from-rocksdb <dir>, --migrate-from-scylladb <addresses>, --migrate-from-fdb <cluster file>, --migrate-force (discards an existing target), --migrate-delete-source.

The migration preserves values, versionstamps, TTLs, queue messages and the version counter. It only runs when the target is empty, so the flag can stay in a deployment permanently.

⁠Supported features

All Deno KV operations over KV Connect: get / getMany / set / delete / list, atomic() with checks and sum / min / max, key expiration (expireIn) and watch. Queues (enqueue / listenQueue) are not part of the KV Connect protocol and therefore not reachable over the network.

⁠License

MIT, like upstream denokv.

Tag summary

Content type

Image

Digest

sha256:827e289d5…

Size

33.2 MB

Last updated

18 days ago

docker pull fourplayers/denokv