iOS Simulator bridge and web content: follow-ups to the #2484 fence (content-class gate, probe circuit, WebContent spike)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- ios, node.js, objective-c, typescript
- Domain
- cli, developer-experience, mobile-dev, testing
Research direction
Start with test/integration/ios-simulator-e2e/live-webview-remote-content.ts and read snapshot-observability.ts, adapter.ts, and snapshot-route.ts to understand the existing route and fallback assertions. For the bridge spike, inspect SnapshotBridgeRuntime.m, SnapshotBridgeCapture.m, RunnerAXSnapshotBridge.m, and snapshot-source/limits.ts, using the existing local-page scenario and control reads. Done requires the ranked tests and help behavior plus a timeboxed report with the specified AX evidence and exit outcome.
Written by the indexing model from the issue text.
Description
Follow-up to #2484 / #2486. That PR is a fence: the Simulator AX bridge refuses a tree that ends at a web view's AXRemoteElement leaf (remote-content-boundary) and the route serves XCTest for the rest of the app generation. This issue scopes what makes the class of defect impossible or cheaper, in the order the design consultation ranked them (two adversarial reviews plus a codex exec architecture pass; verdicts below are the agreed ones).
Why we regressed
- #2279 judged the bridge on node/depth parity, ownership, and latency against native apps. The one fact that mattered — the reader snapshots one process (
apple/snapshot-bridge/README.md) — was never a declared capability or a test. - The failure was silent: a well-formed tree with a viewport and no truncation passes every protocol check, and the quality verdict knows sparse roots, not an on-screen web view with nothing inside.
- No gate drove web content through the default route. The WebView lab (#1397) was a manual lab.
http(s)URLs open Safari by design, but Safari was never in the acquisition validation set.
Tasks, ranked
1. Content-class gate (high value, moderate cost)
Extend the fixture-backed iOS E2E lane with a fixed screen set captured through the default route, asserting coarse content presence, backend-independent: native, WKWebView (exists: smoke:webview-remote-content), Safari on a harness-served local page (Node http server in the scenario; no network), a system sheet. Each row opens a fresh app generation so an earlier fallback cannot mask a broken bridge path. Add one deterministic page action with an observable result (a page ref press/fill) to the WebView row — today it checks content and warnings only (test/integration/ios-simulator-e2e/live-webview-remote-content.ts). Keep fallback-policy assertions (remote-content-boundary / circuit-disabled) separate from content assertions.
2. Launch-observation probe opens the circuit (high value, low cost)
snapshot-observability.ts reads the full tree (its depth: 1 regular hint resolves to full depth in adapter.ts), answers unobservable on a web-first app, and cannot record the refusal: only fallbackAfterFailure in snapshot-route.ts opens the circuit. So a Safari open pays the 300 ms fixed settle and the first snapshot pays one more refused bridge round trip. Pass a route-owned failure-recording hook into the probe and open the circuit for remote-content-boundary. Keep the verdict unobservable; mapping it to observable contradicts the contract in snapshot-observability.ts ("the host AX bridge published its tree").
3. Public help text (modest value, very low cost)
snapshot --help prints the internal backend capability table (src/commands/capture/snapshot.ts, snapshotBackendCapabilityHelp). The #2484 reporter read known gap deep-extension as this bug. Drop the internal table from ordinary help; keep the user-facing depth and fallback semantics.
4. Spike: bounded WebContent acquisition through the bridge's remote-access client (high information value, timeboxed)
Problem. The bridge cannot establish page content. Before any cross-process traversal is designed, prove whether its XCTAccessibilityFramework initForRemoteAccess client (SnapshotBridgeRuntime.m) can read an associated WebContent document while keeping foreground ownership, bounded capture, and actionable node identity.
Blocker to clear first. snapshotForProcess: checks the read pid against primaryApp before capture, during continuations, and after (SnapshotBridgeRuntime.m:319/326/337). A describe request with the WebContent pid is rejected by our own gate, which proves nothing about Apple's AX. The spike must separate the foreground-owner pid from the read pid in disposable code and keep owner checks against Safari or the fixture app.
Prove, in order.
- The experiment reaches the AX call (owner/read pid split; verified WebContent pid supplied manually the first time).
- Remote access returns page semantics: a deterministic local page with unique body text, a link, and an input; require roles and content, not a non-null root or the document title. Record native error domain/code, elapsed time, automation state, bounds. Keep a successful app-pid read as the control.
- The boundary supplies a generic target: inspect
UIAccessibilitySnapshotKeyElementfor theAXRemoteElementbefore JSON conversion; establish whether its pid denotes the host proxy or WebContent; try direct element-rooted resolution (SnapshotBridgeCapture.mcontinuation seam; note the runner re-roots retained elements via a different request API inRunnerAXSnapshotBridge.m). A SafariTabDocument … WebViewProcessID=string is discovery evidence only, never a production association contract. - The target identifies the correct document: two WKWebViews, a second Safari tab, and a navigation that replaces the WebContent process.
- Composition preserves identity and geometry: acquisition stamps every node with
target.pid(adapter.ts); prove node attribution, index/parent normalization, coordinate space, clipping, and a page-ref interaction through the existing runner path. - Recovery stays bounded: share the existing node/request/deadline budgets across all reads; keep the fence when resolution fails; cover duplicate boundaries, process disappearance, and navigation during capture.
Non-goals. Shipping grafting; changing default routing or the circuit; removing the fence; Safari-specific pid parsing in production; new quality vocabulary; physical devices.
Exit criteria.
- No-go: controlled reads reach AX but cannot obtain page semantics within bounds → record the exact failure, keep XCTest fallback.
- Partial: manually selected WebContent reads succeed but generic association or safe composition fails → report transport feasibility only.
- Go: both apps yield correctly associated page content, composition survives a document replacement, a page ref acts within shared bounds → open a separate implementation issue.
- Timebox: two engineering days; unresolved prerequisites end as inconclusive with the missing proof named.
Report: runtime/Xcode versions, native errors, call counts, elapsed time, process identities.
5. Truncation completeness policy (decision, then small change)
A tree cut by the bridge's 1500-node cap (snapshot-source/limits.ts) above the web view carries no remote leaf: the page is missing and only truncated residue discloses it. Decide when an involuntarily capped regular acquisition is publishable versus requiring fallback, for native controls beyond the cap as well as web views. Use the existing truncation residue; do not infer missing content from surviving shapes.
Rejected
- A producer-independent "on-screen WebView with no descendants" degraded-output verdict: empty web views are legitimate, depth/filtering can remove descendants, and truncation above the web view leaves nothing to classify. At most a qualified "content unverified" observation, not degradation or fallback.
- A dedicated
SnapshotSourceFailureKindfor per-screen refusals: changes no policy today; introduce it only when a consumer needs distinct recovery behavior, and migrate the producers that already useunsupportedfor transient states together. - Re-asking the bridge per capture instead of the per-generation circuit: charges a refused round trip to every
waitpoll on a web screen. Revisit only with measurements of native-screen cost in hybrid apps.
Related
open <http url>on a simulator does not bind Safari to the session (src/platform-runtime-open-target.tsgates the fallback to physical devices); tracked separately.
- Dominant language
- TypeScript
- Stars
- 4.7k
- Forks
- 304
- Avg merge
- 11h 19m
- Merged PRs (30d)
- 539
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from callstack/agent-device
-
Proxy lease refuses every iOS install/open: connection platform 'ios' compared with internal 'apple'Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
callstack/agent-device#2962 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
callstack/agent-device#1869 ·
Maintainers usually reply within 1 day
-
needs-triage refactor
Difficulty 5/5 Over a week Newbie friendliness 28/100
callstack/agent-device#2976 ·
Maintainers usually reply within 1 day
-
ready-for-agent refactor
Difficulty 4/5 3-5 days Newbie friendliness 48/100
callstack/agent-device#2974 ·
Maintainers usually reply within 1 day
-
needs-triage refactor
Difficulty 4/5 3-5 days Newbie friendliness 35/100
callstack/agent-device#2973 ·
Maintainers usually reply within 1 day
All issues in callstack/agent-device
Similar issues
-
check:passed streams:add
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Fission-AI/OpenSpec#1986 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
nestjs/docs.nestjs.com#3554 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
publicodes/publicodes#868 ·
-
namespace operations
Difficulty 1/5 Under an hour Newbie friendliness 78/100
EclipseFdn/open-vsx.org#13488 ·
Maintainers usually reply within 2 days