Kubernetes-native sandbox runtime stateless HTTP frontend, auth, API, and MCP server
878
Kubernetes-native sandbox runtime. Run isolated shell commands inside ephemeral Linux environments via a clean HTTP API or MCP — no VMs, no hypervisors, no hardware dependencies.
Each sandbox is an nsjail process jail: isolated filesystem, network namespace, and resource limits backed by a shared read-only Ubuntu 24.04 rootfs. No kernel modules, no container runtimes, no /dev/kvm — just a standard Linux node.
| Component | Language | Role |
|---|---|---|
| boxy-router | Go | Stateless HTTP frontend — auth, API, MCP server, sandbox CR management |
| boxy-operator | Go | Kubernetes controller — bin-packing, StatefulSet auto-scaling, TTL expiry, ControllerPool status |
| boxy-controller | Go | Per-node nsjail daemon — runs actual sandboxes, exposes mTLS HTTP API |
Client --(Bearer)--> [boxy-router Deployment × N]
| Sandbox CR create/read
| mTLS → ctrl-pod-dns:port/v1/...
v
[boxy-operator Deployment]
| Scales StatefulSet, assigns pods
v
[boxy-controller StatefulSet pod]
| nsjail --chroot /rootfs/ubuntu-24.04
v
[nsjail sandbox 1…N]
/workspace (per-sandbox, persistent)
/tmp (per-exec tmpfs, ephemeral)
/ (shared read-only Ubuntu 24.04)
For a deep dive into request flows, scaling algorithms, storage, failure modes, and security trade-offs, see docs/architecture.md.
Every sandbox gets:
/workspace — writable directory bind-mounted per sandbox. Persists across multiple execs within the same sandbox lifetime./tmp — fresh tmpfs per exec, discarded when the command exits.network.allowInternetAccess: true opts out.vm.memoryMb enforced via nsjail --cgroup_mem_max.as, core, cpu, fsize, nofile, nproc, stack configurable per sandbox.kubectl and helm ≥ 3.xImages are published to Docker Hub and the Helm chart to GHCR OCI on every release.
helm upgrade --install boxy oci://ghcr.io/niradler/charts/boxy \
--version 0.0.2 \
--namespace boxy --create-namespace \
--set router.auth.staticToken="$(openssl rand -hex 16)"
helm upgrade --install boxy ./deploy/helm/boxy \
--namespace boxy --create-namespace \
--set router.auth.staticToken="$(openssl rand -hex 16)"
The router accepts any valid Kubernetes ServiceAccount token. Get one for your client SA:
export TOKEN=$(kubectl create token <your-sa> -n <namespace> --duration=3600s)
export BASE=http://<router-service>:8080
# Create a sandbox
curl -sS -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"sessionId":"demo","sandboxId":"demo-1","owner":"you","ttlSeconds":3600}' \
$BASE/v1/sandboxes | jq .
# Execute a command
curl -sS -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"sessionId":"demo","sandboxId":"demo-1","command":"sh","args":["-c","echo hello from nsjail"],"timeoutSeconds":30}' \
$BASE/v1/exec | jq .
# Delete the sandbox
curl -sS -X DELETE -H "Authorization: Bearer $TOKEN" $BASE/v1/sandboxes/demo-1
GET /healthzReturns 200 OK with body ok.
POST /v1/sandboxesCreate a sandbox. Returns 201 when the sandbox is Running.
Required fields: sessionId, sandboxId, owner.
| Field | Type | Description |
|---|---|---|
ttlSeconds | int | Sandbox TTL (sliding window, refreshed on each exec). 0 = no expiry. |
env | map | Environment variables explicitly passed to the sandbox. KUBERNETES_* and BOXY_* prefixes are blocked. Max 64 keys. Sandboxes receive only these keys plus PATH and HOME=/workspace — no host environment leaks through. |
allowedBinaries | string[] | Binaries (e.g. "curl", "python3") bind-mounted read-only from BOXY_NSJAIL_BINARIES_DIR (/usr/local/bin) on the controller into the sandbox. Only listed binaries are accessible; an empty list mounts nothing. Binaries must exist on the controller image — use Dockerfile.controller.dev (or extend it) to pre-bake tools. See docs/architecture.md §4.3 for how to extend the dev image. |
vm | object | Resource and identity config — see below. |
network | object | Network policy — see below. |
volumes | array | Extra mounts inside the sandbox. |
vm fields:
| Field | Type | Description |
|---|---|---|
user | string | Run as this user. |
hostname | string | Sandbox hostname. Default: sandbox ID. |
workdir | string | Working directory inside the sandbox. |
memoryMb | int | Memory cap via cgroup. |
rlimits | array | [{ "resource": "nofile", "soft": 1024 }]. Supported: as, core, cpu, fsize, nofile, nproc, stack. |
image | string | Override rootfs path (must be pre-baked on the controller node). Default: /rootfs/ubuntu-24.04. |
seccompString | string | Kafel/seccomp policy string passed to nsjail. |
cloneNewTime | bool | Isolate the sandbox in a TIME namespace (Linux ≥ 5.6). |
network fields:
| Field | Type | Description |
|---|---|---|
enabled | bool | Default true. |
allowInternetAccess | bool | When true, disables network namespace isolation (sandbox shares the pod's network). |
macvlan | object | Clone a MACVLAN interface into the sandbox: { "interface": "eth0", "ip": "...", "netmask": "...", "gateway": "...", "mac": "..." }. |
usePasta | bool | Use pasta userland networking instead of a network namespace. |
Response: { "sandboxId", "sessionId", "owner", "runtime", "phase", "ready" }. runtime is always "nsjail".
POST /v1/execExecute a command inside an existing sandbox.
| Field | Type | Description |
|---|---|---|
sandboxId | string | Required. |
sessionId | string | Required. |
command | string | Required. Resolved against rootfs PATH if relative. |
args | string[] | Command arguments. |
env | map | Per-exec env overrides. |
timeoutSeconds | int | Required. Hard wall-clock timeout enforced by nsjail (SIGKILL). |
Response: { "exitCode", "stdout", "stderr", "timedOut" }.
GET /v1/sandboxes/{sandboxId}Returns sandbox status.
DELETE /v1/sandboxes/{sandboxId}Deletes the sandbox and removes its CR.
| Code | Meaning |
|---|---|
400 | Bad request (missing required field, invalid JSON, blocked env prefix, body exceeds 6 MB limit). |
401 | Missing, expired, or invalid bearer token (SA token rejected by TokenReview, or static token mismatch). |
404 | Sandbox not found. |
409 | Sandbox already exists. |
429 | Concurrency limit hit (router or controller semaphore full). |
502 | Controller pod unreachable or returned an error. |
503 | Controller pod not ready. |
Output truncation: exec responses that exceed
BOXY_MAX_OUTPUT_BYTES(default 6 MB) are truncated at the controller with a\n[output truncated]suffix. The response still returns200 OK.
POST /mcp — Model Context Protocol endpoint using Streamable HTTP transport (JSON-RPC 2.0), built with the official Go SDK. Stateless, no session management required.
Required headers:
Authorization: Bearer <BOXY_ROUTER_TOKEN>
Content-Type: application/json
Accept: application/json, text/event-stream
Sandbox routing: set X-Sandbox-Id to target a specific sandbox. Omit to use the default sandbox (when BOXY_DEFAULT_SANDBOX_ENABLED=true).
Available tools:
| Tool | Parameters |
|---|---|
bash | command (string, required), timeoutSeconds (int, default 60) |
# MCP initialize
curl -sS -H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","clientInfo":{"name":"cli","version":"1.0"},"capabilities":{}}}' \
$BASE/mcp | jq .
# MCP bash call
curl -sS -H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
-H 'X-Sandbox-Id: demo-1' \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"bash","arguments":{"command":"echo hello from nsjail"}}}' \
$BASE/mcp | jq .
Authorization: Bearer <token> required on every route. Two modes:
UserInfo (username + groups). No static secret to manage.BOXY_ROUTER_TOKEN; requests presenting this token are accepted as dev-token without a TokenReview call. Omit in production.
No per-caller RBAC — all authenticated callers have equal access.BOXY_MTLS_DISABLED=true for local dev. Generate production certs with bash local/gen-mtls-certs.sh [output-dir] [validity-days] (default: 3 years).SYS_ADMIN, SETUID, SETGID, NET_ADMIN, SYS_CHROOT, MKNOD, SETPCAP. All others dropped. allowPrivilegeEscalation: false.| Variable | Default | Description |
|---|---|---|
BOXY_ROUTER_TOKEN | — | Optional static dev/e2e bypass token. If set, accepted without TokenReview. Omit in production — use SA tokens instead. |
BOXY_SANDBOX_NAMESPACE | default | Namespace where Sandbox CRs and controller pods live. |
BOXY_LISTEN_ADDR | :8080 | |
BOXY_CONTROLLER_PORT | 8080 | Port the controller pods listen on. |
BOXY_MTLS_DISABLED | false | Set true for local dev. |
BOXY_TLS_CA_PATH | /tls/ca.crt | |
BOXY_TLS_CLIENT_CERT_PATH | /tls/tls.crt | |
BOXY_TLS_CLIENT_KEY_PATH | /tls/tls.key | |
BOXY_MAX_BODY_BYTES | 6291456 | Max inbound request body (6 MB). Requests over this limit return HTTP 400. |
BOXY_MAX_OUTPUT_BYTES | 6291456 | Max output per exec forwarded to the caller (6 MB). Oversized output is truncated at the router with a \n[output truncated] suffix. |
BOXY_MAX_TIMEOUT_SECONDS | 3600 | |
BOXY_MAX_CONCURRENCY | 100 | Concurrent execs per router replica. |
BOXY_MAX_ARGS | 256 | |
BOXY_MAX_ENV_KEYS | 64 | |
BOXY_CONTROLLER_TOKEN | — | Shared secret injected as X-Boxy-Controller-Token on every router→controller request. Must match the value set on the controller. Auto-generated by the Helm chart; override to rotate. |
BOXY_DEFAULT_SANDBOX_ENABLED | false | Enable default sandbox for stateless MCP clients. |
BOXY_DEFAULT_SANDBOX_CONFIG | — | Required when enabled. JSON sandbox create body. |
| Variable | Default | Description |
|---|---|---|
BOXY_NAMESPACE | — | Required. Namespace to watch. |
BOXY_CONTROLLER_STATEFULSET_NAME | — | Name of the controller StatefulSet. |
BOXY_CONTROLLER_HEADLESS_SERVICE | — | Headless service for pod DNS. |
BOXY_CONTROLLER_PORT | 8080 | |
BOXY_CONTROLLER_POOL_NAME | <statefulset-name> | Name of the ControllerPool CR to keep in sync. Defaults to the StatefulSet name. |
BOXY_MAX_SANDBOXES_PER_CONTROLLER | 20 | Sandboxes per controller pod (bin-packing cap). |
BOXY_MAX_CONTROLLER_REPLICAS | 50 | StatefulSet scale-out ceiling. |
BOXY_MIN_CONTROLLER_REPLICAS | 1 | StatefulSet scale-in floor. |
BOXY_TERMINATED_RETENTION_SECONDS | 3600 | How long to retain Terminated Sandbox CRs before deletion. |
BOXY_SCALE_DOWN_COOLDOWN_SECONDS | 300 | Minimum seconds between successive scale-down events. Prevents flapping when sandbox count fluctuates near the boundary. |
BOXY_MTLS_DISABLED | false |
| Variable | Default | Description |
|---|---|---|
BOXY_CONTROLLER_PORT | 8080 | |
BOXY_CONTROLLER_TOKEN | — | Token callers must send in X-Boxy-Controller-Token. Auto-generated and injected by the Helm chart. Enforced on all non-healthz endpoints. |
BOXY_MAX_SANDBOXES | 20 | Max concurrent sandboxes on this pod. |
BOXY_MTLS_DISABLED | false | |
BOXY_TLS_CERT_PATH | /tls/tls.crt | |
BOXY_TLS_KEY_PATH | /tls/tls.key | |
BOXY_TLS_CA_PATH | /tls/ca.crt | |
BOXY_NSJAIL_PATH | /usr/sbin/nsjail | Path to the nsjail binary. |
BOXY_NSJAIL_ROOTFS | /rootfs/ubuntu-24.04 | Default read-only rootfs. |
BOXY_NSJAIL_SANDBOX_ROOT | /var/lib/boxy/sandboxes | Host path for per-sandbox workspace directories. |
BOXY_NSJAIL_BINARIES_DIR | /usr/local/bin | Host directory for allowedBinaries. |
BOXY_MAX_EXEC_CONCURRENCY | 50 | Max parallel exec calls handled concurrently per controller pod. Returns HTTP 429 when full. |
BOXY_MAX_OUTPUT_BYTES | 6291456 | Max combined stdout/stderr per exec (6 MB). Output over this limit is truncated with a \n[output truncated] suffix. |
# Unit tests
make test
# Lint
make lint
# Format
make fmt
Requires Go ≥ 1.26.
Build and load images into a kind cluster, then run the test suites against a live deployment:
# Build and load into kind (production image)
make kind-load
# Build and load dev controller image (includes jq, yq, curl, git, python3, node in rootfs)
make kind-load-dev
# Go e2e suite
BOXY_E2E_BASE_URL=http://127.0.0.1:18080 \
BOXY_E2E_ROUTER_TOKEN=<token> \
make e2e-go
# Shell e2e suite (infra, security, config, api, isolation, operator, network, controllerpool, allowed-binaries)
BASE_URL=http://127.0.0.1:18080 ROUTER_TOKEN=<token> NAMESPACE=boxy \
make e2e-scripts
A kind config and full setup script is included for spinning up a local cluster.
cmd/boxy-router/ Router entry point (Go)
cmd/boxy-operator/ Operator entry point (Go)
cmd/boxy-controller/ Controller entry point (Go)
main.go Config, mTLS server setup, graceful shutdown
server.go HTTP handlers for /v1/sandboxes, /v1/exec, /healthz
local/
gen-mtls-certs.sh Generate CA + server + client mTLS certs for production deployment
setup-mcp-dev.sh Wire Claude Code (MCP) to a local boxy deployment
internal/
api/ Shared types and validation
nsjail/ nsjail adapter and proto-format config builder
adapter.go Sandbox create/exec/delete lifecycle
nsjail_config.go NsjailConfig struct + ToTextProto() serializer
kube/ Sandbox CR client, controller pod DNS
operator/ Reconciler, StatefulSet scaling, TTL expiry
router/ HTTP server, MCP server, mTLS client
deploy/
helm/boxy/ Helm chart (router, operator, controller, RBAC, mTLS, NetworkPolicy)
manifests/ Raw RBAC manifests
test/e2e/ Go and shell end-to-end test suites
docs/
architecture.md Full system design and architecture reference
Dockerfile.router
Dockerfile.controller Multi-stage: nsjail build + Ubuntu 24.04 rootfs + Go binary
Dockerfile.controller.dev Same as above but with jq, yq, curl, git, python3, node baked into rootfs and /usr/local/bin
Dockerfile.operator
Content type
Image
Digest
sha256:8718c451f…
Size
15.7 MB
Last updated
4 months ago
docker pull niradler/boxy-router