Sign inSign up

psyb0t/auto-linux-desk

By psyb0t

•Updated about 1 hour ago

Full Linux desktop automation in Docker. An Xvfb + Openbox desktop you drive over a JSON HTTP API...

Image
0

872

psyb0t/auto-linux-desk repository overview

Source⁠ | Project page⁠

⁠auto-linux-desk

CI version license Docker Pulls

A virtual Linux desktop in Docker that you drive over HTTP or MCP. One container runs Xvfb, the Openbox window manager, a noVNC viewer and an API that moves the real mouse with human motion, types with human rhythm, presses window manager shortcuts, arranges windows, starts programs, runs shell commands, reads and writes the clipboard, takes screenshots and records the screen.

Use it as it is for Linux desktop automation, or build your own image on it: a plugin adds actions for the software you install, such as a browser.

⁠Quick start

docker run -d --name desk \
  -p 127.0.0.1:8080:8080 \
  -p 127.0.0.1:5900:5900 \
  psyb0t/auto-linux-desk

Open http://localhost:5900/ to watch the screen. Port 8080 serves the HTTP API and the MCP endpoint at /mcp/.

# Open a terminal with the window manager shortcut
curl -X POST http://localhost:8080 \
  -H "Content-Type: application/json" \
  -d '{"action":"send_key","key":"ctrl+alt+t"}'

# Type into it with human keystrokes
curl -X POST http://localhost:8080 \
  -H "Content-Type: application/json" \
  -d '{"action":"system_type","text":"uname -a\n"}'

# Or run a command directly
curl -X POST http://localhost:8080 \
  -H "Content-Type: application/json" \
  -d '{"action":"run_command","command":"ls -la /userdata"}'

# Look at the screen
curl "http://localhost:8080/screenshot/desktop?whLargest=512" -o desk.png

run_command runs any command it is given and launch_app starts any program. With no AUTH_TOKEN, anyone who can reach port 8080 can run commands in the container. Set AUTH_TOKEN before exposing the port beyond loopback, or set DESK_EXEC_ENABLED=false to turn both off. See installation and security⁠.

⁠What it can do

AreaActions
Mousesystem_click, system_double_click, system_right_click, mouse_click, mouse_move, system_hover, system_mouse_down, system_mouse_up, system_drag, scroll, human_pause
Keyboardpaste_text, system_type, send_key
Keyboard shortcutsdesktop_keybindings lists the Openbox bindings: terminal, half-screen tiling, maximize, fullscreen, minimize, close, window switching
Windowsdesktop_list_windows, desktop_focus_window, desktop_move_resize_window, desktop_set_window_state, desktop_close_window
Programs and shelllaunch_app, list_launched, kill_launched, run_command
Screendesktop_screenshot, get_resolution, start_recording, stop_recording, recording_status
Clipboardset_clipboard, get_clipboard (text, HTML, PNG, file lists)
Events and handoffget_events, wait_for_event, begin_handoff, handoff_status, end_handoff
Scripts and utilityrun_script, list_actions, ping, sleep, close

Input coordinates are screen pixels. Input is sent through XTest, so every X client sees ordinary input events, with mouse paths, click timing and typing rhythm shaped like a person's.

⁠MCP

claude mcp add --transport http desk http://localhost:8080/mcp/

Most actions have their own MCP tool, and desk_action runs any action by name. The server instructions tell agents to use window manager shortcuts from desktop_keybindings, to enter text with paste_text, and to keep system_type for a few characters.

⁠Script mode

docker run --rm -i psyb0t/auto-linux-desk --script < script.yaml

The container runs the YAML steps, prints one JSON document on stdout and exits with 0 when every step succeeded, 1 otherwise.

⁠Building on it

Derived images install their software and set AUTO_LINUX_DESK_PLUGINS to Python modules whose register(registry) adds or replaces actions, MCP tools, routes, screenshots, state, health checks and lifecycle hooks. See plugins and derived images⁠.

⁠Documentation

The full documentation is at psyb0t.github.io/docker-auto-linux-desk⁠.

PageCovers
Installation and security⁠Ports, environment variables, auth, users, volumes, hooks
Actions reference⁠Every endpoint, action and parameter
Humanized input⁠The input model, defaults and tuning options
Keyboard shortcuts⁠The window manager bindings
MCP and agents⁠Client setup and tools
Script mode⁠YAML format and output
Plugins and derived images⁠The extension interface
Development⁠Make targets and tests

⁠Development

make lint
make test-unit
make test

Everything runs in containers. make help lists all targets.

⁠Sponsor

Buy Me a Coffee

⁠License

WTFPL⁠. See the changelog⁠ for releases.

Tag summary

Content type

Image

Digest

sha256:b536493f0…

Size

378.6 MB

Last updated

about 1 hour ago

docker pull psyb0t/auto-linux-desk