Live device screen and operator input in the console

Start with console issue #88 and ADR 0005, then trace the existing HTTP API, lease dataPlane handling, gateway path, and event stream. Watching is the first slice; use the listed completion conditions to verify local, gateway, leased, unleased, stopped, stale, and role behavior. Input remains a separately approved later slice, with docs/HTTP-API.md, docs/EVENTS.md, and docs/internal/agent-rules/safety.md included when implemented.

Request: #156

Problem

An operator can see that a device is leased and by whom, but not what is on its screen. To see what an agent is doing, or why it is stuck, they open the simulator window on the machine or take screenshots by hand. On a remote worker neither is possible: the worker has no inbound port and its address never reaches a client.

The console (#88) shows state but lists screen and input as a non-goal. dataPlane on the lease object is reserved for this and is always null. This feature builds on the console, so #88 ships first.

Who it is for

• An operator watching one machine or a fleet from the console, who wants to see a device without touching it.
• A developer debugging an agent run, who sometimes needs to nudge the device: dismiss a dialog, type a value, press a button.
• The agent holding the lease. It must not be disturbed by a watcher, and it must be able to tell when an operator did intervene.

Outcome

Watching. From the Devices and Leases screens an operator opens a live view of any running managed device: iOS or Android, leased or not, local or on any worker behind a gateway. The view follows the device closely enough to watch an agent work. If the stream stalls the console says so, so a frozen view is never mistaken for a frozen device. A stopped device shows "not running". Watching never boots, erases, or otherwise changes a device, and never touches the registry. Closing the view leaves nothing running on the worker. No lease is needed to watch.

Input. From the same view an operator can switch to sending input: tap, swipe, text, and hardware buttons. On a leased device that switch is an explicit step that names the lease's requester, once per input session, not once per tap. Every input session is recorded in the event stream: who, which device, which lease if any, when it started and when it ended. The lease holder can learn from its own lease's event stream that an operator sent input. Input never changes a device's lifecycle state, and Simlock never sends input on its own: no cleanup rule, reaper, or recovery path may.

Roles. The same tokens as the rest of the HTTP API. operator may watch and send input to any device. agent may watch and send input to the device of its own lease and to nothing else. worker tokens can do neither. Through a gateway the worker still needs no inbound port, and its address still never reaches the browser.

Order. Watching ships before input. Input is a later slice that is approved separately.

Non-goals

• Recording to a file, screenshots saved by the console, audio.
• Port forwarding, file transfer, an interactive shell. Each is its own request.
• Screen or input from the CLI or MCP. Agents keep using simlock simctl / simlock adb and exec.
• A new token role, or per-device permissions.
• Physical devices.
• Quality guarantees under load: the view may drop frames or lower its resolution.

Completion conditions

• Against a worker with http.enabled, an operator token opens a live view of a leased iOS device and of a leased Android device from the console. A change on the device shows in the view within two seconds on a local network.
• The same for an unleased running device. Its registry state and the lease table are unchanged after the view closes.
• A stopped device shows "not running" and is not booted.
• Through a gateway, the same view opens for a device on a worker that accepts no inbound connections, and the browser never receives the worker's address.
• When frames stop arriving, the console marks the view stale within five seconds.
• An agent token opens the view of its own leased device and gets 403 for any other device.
• An operator sends a tap, a swipe, text, and a hardware button to a leased iOS device and to a leased Android device, and the device reacts.
• Entering input mode on a leased device asks for confirmation naming the requester. Further taps in that session do not.
• The event stream shows one event when the operator started sending input to a leased device and one when they stopped, each naming the operator's token id, the device, the lease, and the requester. simlock events shows them; docs/EVENTS.md lists them.
• The lease holder's own event stream carries the fact that an operator sent input.
• After an input session the device's registry state and its lease are unchanged.
• No view or input session outlives the daemon that served it: stopping the daemon ends them, and a restart starts none.
• docs/internal/agent-rules/safety.md states the rule for operator input to a leased device. docs/HTTP-API.md documents the routes and roles. dataPlane on the lease object is no longer reserved, and its documented meaning matches what the daemon returns.

Open questions

Each carries a recommended answer. The technical section waits on these.

1. One feature or two? Recommended: one feature, split into tasks with watching first. The input tasks stay task:draft until watching has shipped.
2. Event granularity for input. Recommended: one event per input session (start and end), never one per tap, so the ring buffer stays useful. Per-tap detail is not kept anywhere.
3. Should the lease holder be told? Recommended: yes, on the lease's own event stream and in renew notices, the way device health facts arrive today.
4. Which hardware buttons? Recommended: home, lock, volume up and down on both platforms; back on Android only.
5. May an agent token watch and send input in v1, or is it operator-only? Recommended: agent may, on its own lease only. It can already send input through exec.
6. A limit on concurrent viewers per daemon. Recommended: a config key with a small default; beyond it the view is refused with a closed error code and the console says why.
7. Latency target. Recommended: two seconds on a local network, as above. Nothing is promised through a tunnel.
8. The transport needs an ADR: how screen bytes and input reach the browser, how they cross a gateway to a worker behind NAT, and what dataPlane carries. ADR 0005 left that decision open on purpose. Recommended: draft it in the technical session, after these answers.

Written by an agent.