Self-seeding demo lake: stream raw blocks and query Parquet dumps as an Iceberg table
224
A self-seeding SeaweedFS S3 lake, preloaded with real sample data so you can try the Bitquery Data Lake locally with no setup and no credentials of your own.
The image is just SeaweedFS plus the sample files baked in. On start it boots an S3 endpoint on port 8333 and uploads the samples into a bucket named archive. Nothing is downloaded at runtime.
docker run -p 8333:8333 marketingbitquery/datalake-demo
S3 endpoint: http://localhost:8333 · access key admin · secret key secret · bucket archive
The lake serves two kinds of object. Copy the keys below verbatim — no need to list or search the bucket.
Raw blocks — Protobuf, LZ4-compressed. One file per block.
archive/base/blocks/000046600927_0x0133403c4fe53c434b1d2a1686d339eebd4e8e7f50ab52ab84cd68029e82e955_49e9339dd61bdb91320044378bff935efd925d868ca257ef8c3bc42177f9fd44.block.lz4
archive/tron/blocks/000071523078_0000000004435b06e7e8e836cc1738983b576d4b5ee7a98d1d7877d58c4e1aef_0c72a82a56df49720c455ba0d8e78c8c8a835f9b89bb42e263680397660551a1.block.lz4
Parquet dumps — five flattened Ethereum tables, one sample file each.
archive/ethereum/dex_trades/24053500_24053549.parquet 1,773 rows 53 cols DEX trades
archive/ethereum/transfers/24053500_24053549.parquet 26,563 rows 24 cols native + ERC-20 + internal transfers
archive/ethereum/calls/24053500_24053549.parquet 92,042 rows 41 cols contract call traces
archive/ethereum/events/24053500_24053549.parquet 33,277 rows 45 cols decoded log events
archive/ethereum/balances/2025-01-01.parquet 953,806 rows 11 cols daily balance snapshot (levels)
All cover blocks 24,053,500–24,053,549 except balances, which is a daily snapshot (2025-01-01). The Parquet schemas (per chain and table) are documented in the cloud-dump repo:
https://github.com/bitquery/blockchain-cloud-data-dump-sample
The Parquet files are standard. Any engine reads them straight off S3 — no Iceberg, no catalog:
aws --endpoint-url http://localhost:8333 s3 ls s3://archive/ --recursive
-- DuckDB
SELECT * FROM read_parquet('s3://archive/ethereum/dex_trades/*.parquet');
Parquet files in a bucket are files, not an Iceberg table. An Iceberg engine connects to a catalog, not to the files — so to get real Iceberg (snapshots, hidden partitioning, file pruning) you need a catalog holding the tables.
The scripts repo ships a one-command compose stack that does this for you — the lake, an Iceberg REST catalog (tabulario/iceberg-rest), and a one-shot job that registers all five dumps as Iceberg tables. You point your engine at the catalog; you never run PyIceberg yourself.
https://github.com/bitquery/blockchain-data-lake-sample
git clone https://github.com/bitquery/blockchain-data-lake-sample
cd blockchain-data-lake-sample
docker compose up -d
docker compose logs -f registrar # wait for "[registrar] done"
Then, from any Iceberg engine:
http://localhost:8181http://localhost:8333 (key admin, secret secret, path-style)-- Trino (config: trino/iceberg.properties in the repo)
SHOW TABLES FROM iceberg.ethereum; -- dex_trades, transfers, calls, events, balances
SELECT count(*) FROM iceberg.ethereum.calls;
SELECT Trade_Dex_ProtocolName, count(*) FROM iceberg.ethereum.dex_trades GROUP BY 1 ORDER BY 2 DESC;
This path is tested end to end: a stock trinodb/trino container reads all five tables through the REST catalog with no PyIceberg involved. The repo also has iceberg_query.py for a lighter single-user flow (PyIceberg + DuckDB, local SQLite catalog) and stream.py to stream and decode the raw blocks.
Content type
Image
Digest
sha256:2965e6172…
Size
184.6 MB
Last updated
about 1 month ago
docker pull marketingbitquery/datalake-demo