tinyreasonlabs/ui-atlas repository overview

⁠UI-Atlas: CPU-Optimized UI Search & Indexing Engine

UI-Atlas is a lightweight, purely vision-driven inference engine designed for transforming raw UI screenshots into actionable data structures. Optimized for high-throughput search indexing and automated interaction, it runs entirely on CPU with a relatively compact footprint.

⁠Quick Start

Run the official UI-Atlas image with a single command:

docker run -d -p 8228:8000 --name ui-atlas tinyreasonlabs/ui-atlas:latest

Once running, access the interactive API documentation at: http://localhost:8228/docs

⁠Key Benefits

• Purely Vision-Based: No underlying metadata or accessibility tree required. works on any screenshot.
• CPU Optimized: Designed for economical, scalable cloud deployments without GPU requirements.
• Search & Indexing Ready: Optimized for large-scale UI crawling and context-aware indexing.
• Actionable Data: Extracts structured elements (buttons, inputs, text) with exact coordinates and hierarchical relationships.

⁠Usage Examples

⁠1. Initial Perception & Indexing

Upload a screenshot to generate a structured representation of the UI.

curl -X POST "http://localhost:8228/v1/perceive" \
     -H "Content-Type: multipart/form-data" \
     -F "[email protected]" \
     -F "session_id=optional_session_uuid"

Output: A structured JSON (UPG) containing detected elements, localized OCR labels, and spatial relationships.

⁠2. Follow-up Semantic Search

Query specific elements within an active session using natural language.

curl -X POST "http://localhost:8228/v1/search" \
     -H "Content-Type: application/json" \
     -d '{
       "session_id": "your_session_uuid",
       "query": "find the login button next to the password field"
     }'

⁠Session & State Features

UI-Atlas maintains persistent session states, allowing you to:

• Track UI Changes: Update screenshots within a session to monitor state transitions.
• Contextual Search: Query elements based on the history of the current interaction.
• State Snapshots: Efficiently store and retrieve UI perceptions for search indexing.

⁠Environment Variables

⁠License

UI-Atlas is distributed under the Polyform Internal Use License 1.0.0.

• Usage: Free for internal and non-commercial development.
• Redistribution: This Docker image is the official distribution channel.
• Full Terms: The complete license text is included within this image at /app/LICENSE.