Sign inSign up

fourplayers/denokv

By fourplayers

Updated 29 days ago

A self-hosted backend for Deno KV

Image
Databases & storage
0

306

fourplayers/denokv repository overview

fourplayers/denokv

A self-hosted backend for Deno KV, built from the 4Players fork of denoland/denokv.

Compared to upstream, this fork adds two distributed storage backends (FoundationDB and ScyllaDB), a RocksDB backend, read coalescing, and a built-in migration path between all backends. All backends speak the same KV Connect protocol, so client code never changes.

Quick start

docker run -it --init -p 4512:4512 -v ./data:/data \
  fourplayers/denokv --sqlite-path /data/denokv.sqlite serve --access-token <random-token>

Then connect from Deno, with the access token in DENO_KV_ACCESS_TOKEN:

const kv = await Deno.openKv("http://localhost:4512");

The entrypoint is the denokv binary; serve is the default command. Every flag can also be given as an environment variable (DENO_KV_*).

Storage backends

Exactly one backend must be selected via its data-location flag:

BackendSelect withBest for
SQLite--sqlite-path <file>Single instance; S3 replication and point-in-time recovery
RocksDB--rocksdb-path <dir>Single instance, write-heavy workloads
ScyllaDB--scylladb-address <host:port>Horizontal scaling: many stateless servers on one keyspace
FoundationDB--fdb-cluster-file <file>Horizontal scaling with native transactions, watches and queues
SQLite
docker run -it --init -p 4512:4512 -v ./data:/data \
  fourplayers/denokv --sqlite-path /data/denokv.sqlite serve --access-token <token>

Flags: --num-workers (DENO_KV_NUM_WORKERS, default 1), --atomic-write-batch-timeout-ms (DENO_KV_ATOMIC_WRITE_BATCH_TIMEOUT_MS).

RocksDB
docker run -it --init -p 4512:4512 -v ./data:/data \
  fourplayers/denokv --rocksdb-path /data/denokv-rocksdb serve --access-token <token>

Flags: --rocksdb-write-buffer-mb (default 64), --rocksdb-block-cache-mb (default 128), --rocksdb-max-background-jobs (default 4).

ScyllaDB

Any number of denokv containers can point at the same keyspace and act as one database. Writes serialize through LWT lock stripes in ScyllaDB, so check-and-set semantics and monotonic versionstamps hold across servers.

docker run -it --init -p 4512:4512 \
  fourplayers/denokv --scylladb-address 10.0.0.1:9042 serve --access-token <token>

Flags (DENO_KV_SCYLLADB_*):

  • --scylladb-keyspace (default denokv)
  • --scylladb-replication-factor (default 1; use 3 in production)
  • --scylladb-buckets (default 1024; frozen at keyspace creation)
  • --scylladb-locality-depth (default 2; frozen at keyspace creation)

Limitations: no queues, no S3 replication/PITR, only U64 sum/min/max.

FoundationDB

Any number of denokv containers can attach to the same FoundationDB cluster. Every atomic write is one serializable FDB transaction and versionstamps are FDB commit versionstamps; there is no coordination point, watches are push-based, and queues work across servers.

docker run -it --init -p 4512:4512 -v ./fdb.cluster:/etc/foundationdb/fdb.cluster:ro \
  fourplayers/denokv --fdb-cluster-file /etc/foundationdb/fdb.cluster serve --access-token <token>

Flags: --fdb-prefix (DENO_KV_FDB_PREFIX, default denokv) — acts like a keyspace; several logical databases can share one cluster.

Limitations: no --read-only, no S3 replication/PITR.

Read coalescing

--read-coalescing (DENO_KV_READ_COALESCING, off by default, any backend) merges identical concurrent snapshot reads into a single backend execution while preserving strong consistency. Useful when many clients issue the same query at the same time.

Migrating between backends

serve can import an existing database on startup, before it opens its own store. SQLite is the hub: sqlite <-> rocksdb, sqlite <-> scylladb, sqlite <-> fdb, each in both directions.

# SQLite -> FoundationDB
docker run -it --init -p 4512:4512 -v ./data:/data -v ./fdb.cluster:/etc/foundationdb/fdb.cluster:ro \
  fourplayers/denokv --fdb-cluster-file /etc/foundationdb/fdb.cluster serve \
  --migrate-from-sqlite /data/denokv.sqlite --access-token <token>

Flags: --migrate-from-sqlite <file>, --migrate-from-rocksdb <dir>, --migrate-from-scylladb <addresses>, --migrate-from-fdb <cluster file>, --migrate-force (discards an existing target), --migrate-delete-source.

The migration preserves values, versionstamps, TTLs, queue messages and the version counter. It only runs when the target is empty, so the flag can stay in a deployment permanently.

Supported features

All Deno KV operations over KV Connect: get / getMany / set / delete / list, atomic() with checks and sum / min / max, key expiration (expireIn) and watch. Queues (enqueue / listenQueue) are not part of the KV Connect protocol and therefore not reachable over the network.

License

MIT, like upstream denokv.

Tag summary

Content type

Image

Digest

sha256:dd95bb5fa

Size

32.8 MB

Last updated

29 days ago

docker pull fourplayers/denokv