Sign inSign up

n8500x/atlas-ts

By n8500x

•Updated about 20 hours ago

Atlas agent (TypeScript): deep agent, service, on-prem test kit — docker run IMAGE kit | tar -x

Image
0

2.7K

n8500x/atlas-ts repository overview

⁠Atlas on-prem kit

Everything needed to stand up and test the two Atlas images on an on-prem Linux host, with nothing downloaded at run time.

ImageWhat it is
atlas-tsThe Atlas agent in TypeScript. It holds the deep agent (LangChain deepagents: planning, sub-agents, file tools, approval policy), the chapter 19 service (REST + SSE + async jobs + Prometheus), every chapter script, and the test runners this kit uses.
atlas-sandbox-devopsA Linux sandbox in a Node.js container. It is the full Debian userland plus a DevOps/programming/database toolbox, behind a token-protected TypeScript exec/file API. The agent never runs a command itself: every command and file operation goes to this container.

The kit is baked into the atlas-ts image, so the kit always matches the image version it came from (see VERSION).

docker pull <registry>/atlas-ts:<tag>
docker run --rm <registry>/atlas-ts:<tag> kit | tar -x      # → ./onprem
cd onprem && make help

⁠1. Requirements

HostLinux x86-64 (RHEL 8/9, Ubuntu 22.04+, …). arm64 works for the images built for it.
DockerEngine 24+ with the compose plugin v2.24+ (docker compose version). For Podman, set DOCKER=podman with podman-compose ≥ 1.1 or the docker-compose provider.
Toolsmake, bash, gzip, tar, df. There is no host Python/Node/jq: every check runs inside the containers.
Memorysmoke ≈ 4 GB, itest ≈ 5 GB, itest-all ≈ 8 GB (SQL Server and Oracle).
DiskAbout 6 GB for the two images, plus about 4 GB for all the test-database images.
NetworkOffline targets need none. gateway-check, e2e*, run and service-up need the corporate gateway (and the proxy, if there is one).

make doctor checks all of this.

⁠2. Quick start

# 1. images: from the internal registry …
make pull REGISTRY=nexus.internal:8443/n8500x          # or put REGISTRY in local.mk
#    … or from a tarball carried across the air gap
make load FILE=atlas-images-20261104-120000.tar.gz

# 2. preflight + offline tests (no gateway, no LLM)
make doctor
make smoke                 # sandbox + deep agent + service, ~3–5 min
make itest                 # sandbox ↔ Postgres / MariaDB / Redis, ~3 min (pull-itest first if needed)

# 3. the real model
make env                   # creates gateway.env from the template; edit it
make gateway-check         # token, chat, tool calling
make e2e                   # deep agent DevOps task + service, real model, ~5–20 min

Every test target exits non-zero on failure and writes JUnit XML to reports/.

⁠3. Targets

TargetNeeds gatewayWhat it does
help · info · doctor–Lists targets, shows the configuration, runs the preflight check
pull · pull-itest · pull-itest-all–Pulls the two Atlas images, then the default or all test-DB images
save · load–Writes or loads atlas-images-<tag>.tar.gz for air-gap transfer (make save DB=1 includes the test-DB images)
up · down · clean–Starts the sandbox (waits for healthy) · stops everything · stops it and deletes volumes and reports
shell · tools · logs S=…–Bash inside the sandbox · toolbox versions · follows logs
readme · version–Prints the READMEs baked into both images · shows image versions and labels
smoke–smoke-sandbox + smoke-agent + smoke-service
smoke-chapters–Every Atlas chapter (1–23), the deep agent and the kit load in the image: all modules are imported, so every runtime dependency resolves, native ones included
smoke-sandbox–Sandbox API, hardening, Linux userland, Python/Node/Java/Maven/Gradle/Spark, JDBC drivers, DB clients, file transfer, timeouts
smoke-agent–The deep agent drives the sandbox with a scripted fake model, once over each gateway protocol (Anthropic /v1/messages, the default, and OpenAI): the production model client is used, the toolset is offered, the file is written and the command run in the sandbox, git push is refused by the approval policy, and the verifier sub-agent completes a round trip
smoke-service–The chapter 19 API against the fake model over the Anthropic protocol (its OpenAI calls translated to /v1/messages): auth, sync, SSE, async job through the worker, metrics
smoke-mcp–MCP end to end, offline:
  • a fake OAuth-protected MCP server ← the mcp-hub (config and secret rendered from env, like Vault) ← the sandbox ← the agent and the mcp CLI;

  • token minted, tools discovered;

  • read runs, DELETE held;

  • the secret is invisible to commands. | | smoke-openshift | – | The smoke tests with every container as a random UID in group 0, the way OpenShift runs them | | itest | – | The sandbox's DB tooling against real databases, ITEST_DBS="postgres mariadb redis" by default | | itest-all · itest-<db> | – | Adds MongoDB, SQL Server and Oracle Free · one database (make itest-oracle) | | itest-env | – | Creates itest.env for your own database servers (ITEST_EXTERNAL=mssql …) | | mcp-env · mcp-check · mcp-restart | – | Your MCP servers:

  • mcp-env creates mcp.env;

  • mcp-check shows status, auth, every tool and its approval decision (SERVER= TOOL= ARGS= calls one), and exits 1 if a server is unreachable;

  • mcp-restart re-reads the configuration. | | gateway-check | ✓ | Config resolves, token minted, chat completion, tool calling through the deep agent's own client | | e2e | ✓ | gateway-check + e2e-deep + e2e-service | | e2e-deep | ✓ | The deep agent does a DevOps task unattended (Python module + pytest, Dockerfile + hadolint, Kubernetes manifest, git commit, git push). The harness then re-checks every result itself and never trusts the agent's report. | | e2e-full | ✓ | e2e-deep plus a Postgres table and a PySpark JDBC job | | e2e-service | ✓ | The chapter 19 API with the real model | | test · test-all | – / ✓ | smoke itest · everything (make -k test-all keeps going after a failure) | | run TASK="…" | ✓ | Uses the deep agent: asks on the terminal before risky commands | | run-unattended TASK="…" | ✓ | Same, never asks: risky commands are refused and listed in the report | | service-up · service-down | ✓ | Runs the chapter 19 API on 127.0.0.1:8000 (ATLAS_API_BIND, ATLAS_API_PORT) | | oc-deploy · oc-smoke · oc-e2e · oc-run · oc-status · oc-delete | – / ✓ | Deploy to OpenShift / Kubernetes and run the suites or the agent as in-cluster Jobs (section 7) |

KEEP=1 leaves the test databases running after itest / e2e-full, for debugging.

⁠4. What the tests prove

smoke-sandbox runs every check through the sandbox's HTTP API, which is the same path the agent uses:

  • /health answers and every required tool is present. Optional tools (Go, AWS CLI, SQLcl, mongosh) are reported if missing.
  • 401 without a token or with a wrong one. Commands run as a non-root user, the root filesystem is read-only, and the server's token is not visible to commands. /workspace and /tmp are writable.
  • The Linux userland works: grep, sed, gawk, find, xargs, sort, uniq, cut, tr in one pipeline.
  • Python: psycopg, SQLAlchemy, oracledb, PyMySQL, pymssql, pyodbc, pymongo, redis, pandas, pyarrow, duckdb, PyYAML, requests, boto3, kubernetes and Jinja2 import; DuckDB runs a query; pytest runs a test.
  • Node: pg, mysql2, oracledb, mssql, mongodb, redis, ioredis and knex resolve; better-sqlite3 runs a query.
  • Java: a single-file program runs, every JDBC driver loads (PostgreSQL, Oracle, MariaDB, SQL Server, SQLite, H2), and H2 and SQLite answer queries. Maven and Gradle start.
  • Spark: a local PySpark job computes a sum and reads a table over JDBC through the bundled drivers.
  • DB command-line clients start. A file uploads and downloads byte for byte. A timed-out command is killed.
  • The richer API works:
    • a background python -m http.server is started, curled, then stopped as a whole process group;
    • streaming exec and per-command env work;
    • a directory makes a tar.gz round trip, and list/move/delete work;
    • /metrics and /openapi.json are served.

itest: for every database the same matrix runs inside the sandbox.

DBCLIPythonNodeJDBC (Java)Spark JDBC
PostgreSQLpsqlpsycopg + SQLAlchemypgpostgresql✓
MariaDB / MySQLmariadbPyMySQLmysql2mariadb-java-client
Redisredis-cliredisioredis
MongoDBmongosh (optional)pymongomongodb
SQL Serversqlcmd (go-sqlcmd)pymssqlmssqlmssql-jdbc
OracleSQLcl (optional)oracledb (thin, no Instant Client)oracledb (thin)ojdbc11✓

Your own database servers instead of containers. Use this per database, e.g. when your registry has no SQL Server image (mcr.microsoft.com is not proxied).

  1. Run make itest-env and fill in itest.env.
  2. List those databases in ITEST_EXTERNAL: the others still run as throwaway containers, and the external ones are neither pulled nor started.
make itest ITEST_DBS="postgres mariadb redis mssql" ITEST_EXTERNAL=mssql
# or for good, in local.mk:   ITEST_EXTERNAL := mssql
MSSQL_HOST=sqlserver01.internal
MSSQL_PORT=1433
MSSQL_USER=atlas_itest
MSSQL_PASSWORD=…
MSSQL_DB=tempdb        # default: the scratch table never lands in a real database, and it is dropped at the end
# also PG_*, MARIADB_*, REDIS_HOST/PORT, MONGO_URI, ORACLE_* (see itest.env.example)

EXTERNAL=1 treats every database in ITEST_DBS as yours. The login only needs to create and drop a table in the target database.

Or put the SQL Server image into a hosted repository of your Nexus. Keep it private: the image is Microsoft's, under its EULA.

docker pull mcr.microsoft.com/mssql/server:2022-latest        # on a machine that can reach mcr.microsoft.com
docker tag  mcr.microsoft.com/mssql/server:2022-latest nexus.internal:8443/mssql/server:2022-latest
docker push nexus.internal:8443/mssql/server:2022-latest
# local.mk:  MSSQL_IMAGE := nexus.internal:8443/mssql/server:2022-latest

The sandbox reaches them over the egress network, so your firewall must allow that route.

e2e-deep checks the result, not the agent's claims. After the agent's report it runs:

  • pytest again;
  • divide(1, 0) (it must raise ValueError);
  • hadolint;
  • yq on the manifest (it must be a Deployment named calc with 2 replicas).

It then checks the git commit contents, that git push was held for approval and never run, and that no command hit the deny list. A small model can pass some checks and fail others; the per-check output shows which.

⁠5. Configuration

local.mk (site settings, read by the Makefile):

REGISTRY := nexus.internal:8443/n8500x    # where the Atlas images live
TAG      := 20261104-120000               # default: the kit's VERSION
MIRROR   := nexus.internal:8443/          # prefix for postgres/mariadb/redis/mongo/mssql/oracle test images
DOCKER   := docker                        # or podman
ITEST_DBS := postgres oracle

You can also override individual images: PG_IMAGE, MARIADB_IMAGE, REDIS_IMAGE, MONGO_IMAGE, MSSQL_IMAGE, ORACLE_IMAGE.

gateway.env (make env): Atlas calls the LLM exactly the way the n8500x/claude-code image does. It uses the same variables with the same defaults, so the same env file works for both:

claude-code imageAtlas
EndpointCLAUDE_GATEWAY_URL (or ANTHROPIC_BASE_URL), base URL only. A pasted /v1/messages or /v1/chat/completions is stripped and picks the protocol.same
ProtocolGATEWAY_PROTOCOL=anthropic (default) → POST <base>/v1/messagessame. The deep agent calls /v1/messages natively (LangChain ChatAnthropic); chapters and the service are translated in-process
GATEWAY_PROTOCOL=openai → POST <base>/v1/chat/completions (local bridge)same (native OpenAI client)
AuthOAuth2 client credentials (GATEWAY_TOKEN_URL or KEYCLOAK_URL/OIDC_ISSUER_URL + realm, GATEWAY_CLIENT_ID/_SECRET[_FILE], GATEWAY_SCOPE), via apiKeyHelper → Authorization: Bearer + X-Api-Key. Refresh every CLAUDE_CODE_API_KEY_HELPER_TTL_MS, re-mint on 401.same, natively (or CLAUDE_TOKEN_HELPER runs the image's helper program)
GATEWAY_API_KEY[_FILE] / GATEWAY_NO_AUTH=1 → Authorization: Bearer onlysame
RoutingGATEWAY_COOKIE[_FILE], ANTHROPIC_CUSTOM_HEADERSsame (+ GATEWAY_HEADERS)
TLSCLAUDE_CA_CERT, CLAUDE_TLS_INSECURE=1same (+ HTTPS_PROXY/NO_PROXY)
ModelANTHROPIC_MODEL (sent verbatim; ANTHROPIC_DEFAULT_SONNET_MODEL wins)same
Context windowCLAUDE_CONTEXT_LIMITsame (ATLAS_MODEL_CONTEXT also accepted). The deep agent summarises at 85 % of it.

The image's Bedrock mode (direct SigV4, no gateway) is not used on-prem and is not implemented in Atlas.

Corporate CA: put the PEM in certs/ and set CLAUDE_CA_CERT=/certs/ca.pem.

Proxy: HTTPS_PROXY / NO_PROXY in gateway.env (the gateway client honours them).

Giving the sandbox credentials (kubeconfig, cloud profile, …):

  1. Mount them in compose.yaml under sandbox.volumes. There is a commented example.
  2. Name the variables in SANDBOX_PASS_ENV (for example SANDBOX_PASS_ENV=KUBECONFIG,AWS_PROFILE in .env, through local.mk).

The sandbox scrubs every other variable from commands. The agent container never holds these credentials.

⁠6. MCP servers

The deep agent uses your MCP servers (SQL Server, Jira, Confluence, …) through the sandbox. The sandbox is the box with network access to your systems, and the agent container only talks to the LLM gateway.

agent ──tools mcp__<server>__<tool>──▶ sandbox ──▶ mcp-hub ──HTTP (+ OAuth token)──▶ your MCP server
                     sandbox commands: mcp servers · mcp tools <server> · mcp call <server> <tool> '{json}'

Separate hub container. mcp-hub is a separate container, started from the same sandbox image (docker run IMAGE mcp-hub). It is the only place the MCP configuration and secrets live. The agent's commands run in the sandbox container, so they cannot read the hub's environment. smoke-mcp proves this on every run.

Discovery. At the start of every run the agent asks the sandbox which servers and tools exist. Each tool becomes mcp__<server>__<tool>, with the server's own description and input schema. A server that is down is reported and skipped.

Configuration (mcp.env here; your Vault injector in production). Use any of three shapes; they merge:

ShapeExample
One variable per fieldATLAS_MCP_MSSQL_URL=http://mssql-mcp.internal:8000/mcp · ATLAS_MCP_JIRA_URL=… ATLAS_MCP_JIRA_TOKEN_URL=… ATLAS_MCP_JIRA_CLIENT_ID=… ATLAS_MCP_JIRA_CLIENT_SECRET=…
One JSON valueSANDBOX_MCP_CONFIG_JSON={"mcpServers":{"jira":{"url":"…","auth":{"type":"oauth2","token_url":"${SSO_TOKEN_URL}","client_id":"atlas","client_secret":"${JIRA_MCP_SECRET}"}}}}
A JSON filemcp/mcp.json (mounted at /etc/atlas/mcp.json), same format
  • ${VAR} / ${VAR:-default} is rendered from the hub's environment at start-up. A missing variable marks only that server as failed.
  • make mcp-restart re-reads the configuration.

Auth per server is optional:

  • none;
  • oauth2 client credentials: the hub mints the token, renews it before expiry and re-mints on 401;
  • bearer (a static token);
  • gateway (the same SSO client as the LLM gateway, from GATEWAY_TOKEN_URL/_CLIENT_ID/_CLIENT_SECRET in the hub's env).

Extra headers are allowed. Secrets never appear in an API reply, a log line or a command's environment.

Approval policy per MCP call, decided in the agent before the call leaves it. The hub also enforces deny itself, for every caller. In order:

  1. deny globs → never.
  2. Arguments the shell policy would deny (e.g. rm -rf /) → never.
  3. ask globs → approval.
  4. SQL that writes in any argument (INSERT/UPDATE/DELETE/DDL/GRANT/EXEC sp_…) → approval. This applies even to allowed tools; set "sql_writes": "allow" to permit it.
  5. allow globs → runs.
  6. Tools annotated readOnlyHint → run; destructiveHint → approval.
  7. Anything else → the server's default (default ask).

Unattended runs refuse "ask". Set allow globs for your read tools: ATLAS_MCP_MSSQL_POLICY_ALLOW=query,list_*,describe_*. make mcp-check prints the decision for every tool, so you can tune it before a run.

From scripts inside the sandbox:

  • mcp call mssql query '{"sql":"select top 5 * from dbo.orders"}', or POST http://127.0.0.1:8080/mcp-servers/<server>/tools/<tool> with {"arguments":{…}} (loopback needs no token);
  • the official SDKs are installed for direct use: Python mcp, Node @modelcontextprotocol/client.

OpenShift: the hub is a sidecar in the sandbox pod, listening on 127.0.0.1:8081. Point the Vault injector at the mcp-hub container, or let make oc-deploy create secret atlas-mcp from mcp.env. make oc-mcp-check runs the check in the cluster.

⁠7. OpenShift

Both images run unchanged under OpenShift's default restricted-v2 SCC.

Random UID. OpenShift starts containers as an assigned UID in group 0, not the image's user, and the images are built for that:

  • numeric USER (10001 / 1000) with primary group 0;
  • every writable path (/workspace, /home/sandbox, /conversation_history, /data, /home/node) is group 0 with group = owner permissions;
  • HOME is set explicitly.
  • The sandbox gives the unknown UID a passwd entry at start-up through nss_wrapper, so ssh, git and whoami work even with a read-only root.

Restrictions: no root, no capabilities, no privilege escalation, a read-only root filesystem, and ports 8080 / 8000.

Proven by the kit: make smoke-openshift runs every smoke test with all containers as UID 1000680000:0 (OCP_UID), read-only and capability-free.

Deploying (oc logged in to the target project; OC=kubectl works for plain Kubernetes):

make oc-deploy                 # secret from .sandbox-token, sandbox Deployment + Service + NetworkPolicy + PVC
make oc-smoke                  # the offline smoke suites as a Job in the cluster
make env && make oc-deploy SERVICE=1   # + gateway secret (from gateway.env), API Deployment + Service + Route
make oc-e2e SERVICE=1          # real-model e2e as a Job
make oc-run TASK="…"           # the deep agent, unattended, as a Job
make oc-status / oc-delete     # KEEP_PVC=1 keeps the volumes

The manifests in openshift/ are plain Kubernetes YAML: no runAsUser, RuntimeDefault seccomp, a read-only root, all capabilities dropped. A NetworkPolicy lets only pods labelled atlas.io/sandbox-client=true (the agent and test Jobs) reach the sandbox. Images come from $(REGISTRY); point it at the cluster's mirror. For a corporate CA, put it in certs/ca.pem: it becomes secret atlas-ca, mounted at /certs. On plain Kubernetes, add fsGroup: 10001 to the sandbox pod so its PVC is writable.

⁠8. Security model

  • Two containers:
    • The agent (model access, approval policy) and the sandbox (commands) only meet on an internal network.
    • The sandbox has no published port.
    • Both run read-only, with all capabilities dropped and no-new-privileges. The sandbox runs as uid 10001.
  • Approval policy (in the agent, before a command leaves it):
    • Denied always: wiping / or $HOME, disk writes, fork bombs, reverse shells, reading credential files, dumping the environment, deleting core namespaces.

    • Needs approval: terraform apply/destroy, kubectl apply/delete/…, helm install/upgrade, mutating AWS calls, git push, gh changes, registry push, curl | sh, ansible-playbook without --check, ssh/scp/rsync, systemctl stop, package publishing, and any database write:

      • SQL that writes (INSERT/UPDATE/DELETE/MERGE/COPY, DDL, GRANT, EXEC sp_…) through any client in the image, judged on the whole command line (heredocs and multi-statement strings included);
      • mongosh and redis-cli writes, and bcp in;
      • inline code (python -c, node -e) that uses a DB driver to write;
      • .sql files and scripts the command runs, which the agent fetches from the sandbox and checks before running (a DB driver plus a write, or a Spark JDBC write).

      The patterns catch mistakes, not a determined adversary. The real boundary for direct database access is a read-only login for the sandbox; let writes go through an MCP server, where the same SQL check and your MCP server's permissions apply.

    • Dry runs are allowed.

    • The sandbox enforces the never-allowed list itself for every caller, and the whole policy on its own MCP endpoint (/mcp), where no agent sits in front.

    • Each && / ; part of a command is judged separately.

  • Sandbox API:
    • bearer token (generated by make into .sandbox-token, mode 600);
    • per-command timeout with a process-group kill;
    • output, concurrency and timeout caps;
    • one JSON audit line per command (make logs S=sandbox).
  • The test databases use throwaway credentials and live only on the internal network, with their data in tmpfs.

⁠9. Reports and CI

Each suite writes reports/<suite>.xml (JUnit). A Jenkins stage example:

stage('Atlas on-prem tests') {
  steps {
    sh 'docker run --rm $REGISTRY/atlas-ts:$TAG kit | tar -x'
    dir('onprem') {
      sh 'make REGISTRY=$REGISTRY TAG=$TAG doctor smoke itest'
      withCredentials([file(credentialsId: 'atlas-gateway-env', variable: 'GW')]) {
        sh 'cp "$GW" gateway.env && make REGISTRY=$REGISTRY TAG=$TAG e2e'
      }
    }
  }
  post { always { junit 'onprem/reports/*.xml'; dir('onprem') { sh 'make clean || true' } } }
}

⁠10. Troubleshooting

SymptomCause / fix
sandbox not healthy after 180 smake logs S=sandbox. The first start probes every tool (the JVM tools take a few seconds). Check memory (SANDBOX_MEM, default 6g).
need docker compose v2.24+Upgrade the compose plugin. Older versions reject env_file: required:.
Permission denied on reports/ or certs/ (RHEL)SELinux: the mounts already carry :z. On rootless Podman, check podman unshare ls -ln reports.
gateway-check 401 / TLS errorWrong client id or secret, or scope; for TLS, the CA is missing (certs/ + CLAUDE_CA_CERT). For a proxy, set HTTPS_PROXY.
tool calling check failsThe gateway or model does not support tool use (Anthropic tools / OpenAI tools). The deep agent needs it.
404 on /v1/messagesThe gateway only speaks OpenAI: set GATEWAY_PROTOCOL=openai, the same switch as the claude-code image.
Agent stops mid-task / context length exceededSet CLAUDE_CONTEXT_LIMIT to the model's real window.
Oracle / SQL Server itest times outThe first start takes 1–3 min (Oracle allows up to 10). Re-run make itest-oracle KEEP=1 and watch make logs S=oracle.
A tool is missing in smoke-sandboxThe image was built without an optional component (e.g. WITH_GO=false). make tools shows what is installed.

⁠11. Files

File
Makefilethe targets above
compose.yamlsandbox, agent, fake model, services, test databases
compose.openshift.yamlOPENSHIFT=1 override: random UID, group 0
openshift/sandbox, API, Route and Job manifests for oc-*
gateway.env.examplegateway template (make env)
mcp.env.exampleMCP servers for the mcp-hub (make mcp-env)
itest.env.exampleyour own DB servers for itest (make itest-env)
VERSIONthe image tag this kit was extracted from
written at run time.env (compose variables, regenerated every run), .sandbox-token, gateway.env, reports/, certs/, optional local.mk / itest.env

Tag summary

Content type

Image

Digest

sha256:7e3af37f4…

Size

137.1 MB

Last updated

about 20 hours ago

docker pull n8500x/atlas-ts