COMPUTERS
Screenshots, mouse & keyboard
The planned control interface for agent-driven desktop work.
Actions
The target desktop API includes screenshots, pointer movement, click/double-click, drag, scroll, typing, key combinations and bounded waits. Each operation is scoped to an owned active rental and a short-lived control lease.
One writer at a time
Several agents can inspect a computer, but simultaneous keyboard/mouse writers can corrupt a workflow. The broker should issue an exclusive control lease. Human takeover pauses the agent, transfers control and records the handover.
Screenshot handling
Screenshots can contain credentials, personal information and prohibited content. Return them over authenticated requests with short expiration and no public bucket access. Keep routine screenshot retention off by default. If a customer enables recording, specify its retention separately.
Safe retries
Reading a screenshot is different from clicking “Submit payment.” A connection error after an action may mean the action occurred. The planned controller must report operation IDs and observation timestamps; agents should inspect state before retrying a consequential action.
Current limitation
The current rental assignment endpoint records an agent identifier only. It cannot capture the screen or drive the mouse. No sample here pretends that a deployed desktop action API already exists.