omab/cloudbox repository overview

⁠Cloudbox

A local emulator for Google Cloud Platform services — like LocalStack, but for GCP.

Run Cloud Storage, Pub/Sub, Firestore, Secret Manager, Cloud Tasks, BigQuery, Cloud Spanner, Cloud Logging, and Cloud Scheduler entirely on your machine, with no real GCP credentials or network access required.

⁠Table of Contents

• About this project⁠
• Services⁠
• Quick Start⁠
• Configuration⁠
• Using the GCP SDK⁠
• gsutillocal CLI⁠
• gcloudlocal CLI⁠
• BigQuery⁠
• Cloud Scheduler⁠
• Development⁠
• Architecture⁠
• Feature Matrix⁠
  • Cloud Storage⁠
  • Pub/Sub⁠
  • Firestore⁠
  • Secret Manager⁠
  • Cloud Tasks⁠
  • BigQuery⁠
  • Cloud Spanner⁠
  • Cloud Logging⁠
  • Cloud Scheduler⁠
• Roadmap⁠

⁠About this project

Cloudbox is an experiment in LLM-driven development. The entire codebase — services, tests, CLI tools, admin UI, and documentation — was written through an iterative conversation with Claude Code⁠, Anthropic's AI coding assistant, with no manual code authoring by the human developer.

The goal is twofold: to explore how far AI-assisted development can go on a non-trivial engineering project, and to produce something genuinely useful for engineers who want to run GCP-dependent services locally without real credentials or network access.

The feature matrix, test coverage, and architecture reflect real engineering tradeoffs, not generated boilerplate. Feedback, issues, and contributions are welcome.

⁠Services

⁠Quick Start

⁠Docker (recommended)

docker compose up

The container starts all services and exposes their ports automatically. The Admin UI is available at http://localhost:8888⁠.

⁠Local (Python)

Requires Python 3.12+ and uv⁠.

uv sync
uv run python -m cloudbox.main
# or
uv run cloudbox

⁠Configuration

All settings are controlled via environment variables:

To enable data persistence across restarts, set CLOUDBOX_DATA_DIR to a writable path (and mount it as a Docker volume if using containers).

⁠Using the GCP SDK

Point the official google-cloud-* SDK clients at Cloudbox using the helpers in sdk_compat/clients.py:

from sdk_compat.clients import (
    storage_client,
    pubsub_publisher,
    pubsub_subscriber,
    firestore_client,
    secret_manager_client,
    tasks_client,
    bigquery_client,
    scheduler_client,
)

# Cloud Storage
client = storage_client()
bucket = client.bucket("my-bucket")

# Pub/Sub (gRPC by default, or pass transport="rest")
publisher = pubsub_publisher()
subscriber = pubsub_subscriber()

# Firestore
db = firestore_client()

# Secret Manager
sm = secret_manager_client()

# Cloud Tasks
tasks = tasks_client()

# BigQuery
bq = bigquery_client()
result = bq.query("SELECT 1 AS n").result()

# Cloud Scheduler
sched = scheduler_client()

⁠Pub/Sub — connecting directly

For gRPC transport (the standard emulator pattern), set the environment variable before importing the SDK:

export PUBSUB_EMULATOR_HOST=localhost:8085

For REST transport, use api_endpoint="http://localhost:8086" in ClientOptions.

⁠gsutillocal CLI

gsutillocal mirrors the gsutil command syntax for Cloud Storage. After uv sync:

gsutillocal ls                              # list all buckets
gsutillocal ls gs://my-bucket              # list objects
gsutillocal ls -l gs://my-bucket           # long listing (size + timestamp)
gsutillocal ls -r gs://my-bucket           # recursive
gsutillocal mb gs://my-bucket              # create bucket
gsutillocal mb -l US-EAST1 gs://my-bucket  # with location
gsutillocal rb gs://my-bucket              # delete bucket
gsutillocal cp ./file.txt gs://bucket/     # upload
gsutillocal cp -r ./dir/ gs://bucket/dir/  # recursive upload
gsutillocal cp gs://bucket/file.txt ./     # download
gsutillocal cp gs://b1/obj gs://b2/obj     # GCS-to-GCS copy
gsutillocal mv gs://b/old gs://b/new       # move / rename
gsutillocal rm gs://bucket/file.txt        # delete object
gsutillocal rm gs://bucket/logs/*          # wildcard delete
gsutillocal rm -r gs://bucket              # delete all objects + bucket
gsutillocal cat gs://bucket/file.txt       # print to stdout
gsutillocal stat gs://bucket/file.txt      # object metadata
gsutillocal du gs://bucket                 # disk usage

Global flags -m (parallel) and -o (boto options) are accepted and silently ignored for drop-in compatibility with existing scripts.

⁠gcloudlocal CLI

gcloudlocal is a gcloud-compatible CLI installed as an entry point that targets the Cloudbox emulator. After uv sync it is available as:

uv run gcloudlocal [--project PROJECT] [--location LOCATION] [--format json] \
    SERVICE RESOURCE VERB [ARGS] [FLAGS]

Or run it directly via the shim:

python bin/gcloudlocal.py ...

⁠Examples

# Cloud Storage
gcloudlocal storage buckets list
gcloudlocal storage buckets create my-bucket
gcloudlocal storage cp ./file.txt gs://my-bucket/file.txt

# Pub/Sub
gcloudlocal pubsub topics create my-topic
gcloudlocal pubsub subscriptions create my-sub --topic my-topic
gcloudlocal pubsub topics publish my-topic --message "hello"

# Secrets
gcloudlocal secrets create my-secret
gcloudlocal secrets versions add my-secret --data "s3cr3t"

# Firestore
gcloudlocal firestore documents list my-collection

# Cloud Tasks
gcloudlocal tasks queues create my-queue
gcloudlocal tasks tasks create my-queue --url http://localhost:8000/task

# Cloud Scheduler
gcloudlocal scheduler jobs list
gcloudlocal scheduler jobs create my-job --schedule "* * * * *" --uri http://localhost:8000/handler

⁠BigQuery

BigQuery is backed by DuckDB⁠. Datasets map to DuckDB schemas; tables map to DuckDB tables. SQL is rewritten at query time to convert backtick identifiers (project.dataset.table) to double-quoted DuckDB form.

Supported operations:

• Dataset and table CRUD
• SELECT, WITH, VALUES queries
• DML: INSERT, UPDATE, DELETE — returns numDmlAffectedRows
• CREATE TABLE AS SELECT (CTAS)
• SHOW, DESCRIBE, EXPLAIN, PRAGMA

⁠Cloud Scheduler

Jobs are stored in memory (or persisted to CLOUDBOX_DATA_DIR). A background worker polls every 30 seconds and fires HTTP requests via httpTarget. Job state transitions (enable/pause/resume) and force-run are supported.

from sdk_compat.clients import scheduler_client
from google.cloud import scheduler_v1

client = scheduler_client()
parent = f"projects/local-project/locations/us-central1"

job = scheduler_v1.Job(
    name=f"{parent}/jobs/my-job",
    schedule="* * * * *",
    time_zone="UTC",
    http_target=scheduler_v1.HttpTarget(
        uri="http://localhost:8000/handler",
        http_method=scheduler_v1.HttpMethod.POST,
    ),
)
client.create_job(request={"parent": parent, "job": job})

⁠Development

⁠Running tests

uv run pytest tests/

Tests use pytest-asyncio with asyncio_mode = "auto". Each test file covers one service.

⁠Live SDK smoke tests

uv run python sdk_compat/test_with_sdk.py

This requires Cloudbox to be running and the dev dependencies installed (uv sync).

⁠Helper scripts

The bin/ directory contains shell scripts for common setup tasks:

• bin/setup_gcs_pubsub.sh — create a GCS bucket with a Pub/Sub notification
• bin/create_pubsub_topic_subscription.sh — create a topic and subscription
• bin/print_pubsub_messages.sh — pull and print messages from a subscription
• bin/upload_to_gcs.sh — upload a file to a GCS bucket

⁠Architecture

All state is stored in memory by default, using a NamespacedStore (cloudbox/core/store.py). Each service is a standalone FastAPI application started concurrently by cloudbox/main.py via uvicorn. Pub/Sub additionally runs a gRPC server on port 8085 (compatible with PUBSUB_EMULATOR_HOST). Cloud Scheduler runs a background asyncio worker that polls every 30 seconds.

Every service module follows the same layout:

cloudbox/services/<service>/
    app.py      — FastAPI routes
    models.py   — Pydantic v2 models
    store.py    — NamespacedStore wrapper
    worker.py   — (scheduler only) background dispatch loop

BigQuery uses an additional engine.py module wrapping DuckDB instead of a NamespacedStore.

⁠Feature Matrix

Legend: ✅ Supported · 🟡 Partial · ❌ Not supported

⁠Cloud Storage

⁠Pub/Sub

⁠Firestore

⁠Secret Manager

⁠Cloud Tasks

⁠BigQuery

⁠Cloud Scheduler

⁠Cloud Spanner

⁠Cloud Logging

⁠Roadmap

• Vertex AI