AGENT 05 / 07
Browser
The digital computer operator. It drives web portals via ARIA accessibility trees, handles complex forms, and streams live viewports with 2FA takeover support.
browser · autonomous
01What it does
Governed web & desktop interaction
ARIA accessibility tree
Navigates complex web applications using semantic accessibility element IDs (@e1..@eN) rather than brittle pixels.
Live screen streaming
Streams browser viewports over WebSockets at 1080p for real-time human observation and debugging.
2FA & CAPTCHA takeover
Pauses safely when encountering SMS/2FA challenges and grants an exclusive human control mutex lease.
CDP browser driving
Direct Chrome DevTools Protocol automation for navigation, clicks, keystrokes, and structured DOM extraction.
Session state persistence
Maintains authenticated cookies and local storage across workflows inside isolated Playwright profiles.
Token-efficient parsing
Reduces visual browser loop token usage by 90% by processing text trees rather than raw screenshots.
02How it works
From URL to verified web action
Target navigation
Launches headless or headed Chrome session and navigates to the target web portal.
ARIA snapshot
Captures the accessibility tree and maps interactive elements to unambiguous token IDs.
Action execution
Executes clicks, form inputs, dropdown selections, and tab switches via CDP commands.
Verification & pause
Detects 2FA challenges; requests operator assistance if manual intervention is required.
Data extraction
Extracts clean structured tables and documents directly into the worker memory store.
03Under the hood
The runtime underneath
| protocol | Direct Chrome DevTools Protocol (CDP) and Playwright service layer |
| accessibility | Normalized ARIA snapshot tree with depth bounding and interactive element filtering |
| streaming | Binary WebSocket JPEG stream with adaptive quality and cursor overlay |
| takeover | Human-in-the-loop mutex lease with operator timeout and session recovery |
04Tools