refactor(ios-runner): acquire first, present after — one geometry normalization pass instead of per-walker threading
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 28/100
- Issue type
- Refactor
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- ios, swift
- Domain
- mobile-dev, performance, testing-qa
Research direction
Start with captureWithBackend, the recursive acquisition code in apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift, and SnapshotCoordinateSpace.swift. Run the deep-tree benchmark and existing rotated-output tests before changing the acquisition flow. Done means one post-acquisition normalized pass, no geometry threading in walkers, updated ADR 0004, passing package and runner suites, and benchmark results within the stated limit.
Written by the indexing model from the issue text.
Description
Why
In the XCTest runner, acquisition and presentation are interleaved in the recursive tier, so every geometric fact has to be threaded through every walker by hand.
Evidence from PR #2653 (fixes #2612), which added one fact, "which coordinate space is this node in":
- It had to touch three walkers (
recursiveTreeSnapshotAcquisition,rawTreeSnapshotAcquisition, the private-AXappendPrivateAXNode) plus the collapsed-tab side path (collapsedTabFallbackNodes/collapsedTabCandidateNode) inapple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swiftandRunnerTests+PrivateAXPresentation.swift, addinggeometrySpace/parentIsWindowparameters to each frame of each walk. - The query-sweep tier (
querySweepSnapshotAcquisition) got nothing, because it has no window ancestry inside its walk, so that tier still publishes unnormalized geometry (ADR 0004, "What stays unnormalized"). - The collapsed-tab expansion needed a special guard (
guard geometrySpace == .appOrientation else { return [] }) because it reads live element frames that are already in the app's space.
The reason a post-acquisition pass could not do this in one place is at RunnerTests+Snapshot.swift around line 244: the recursive walk calls SnapshotPresentation.regularTraversalTransition(for: node, …) mid-walk, consulting the visibility fold to decide the presented depth and whether to descend. The fold reads node geometry, so geometry must be final before the fold sees it, so it must be computed inside the walk. The private-AX tier does not have this coupling: it acquires the whole raw tree and presents afterwards through SnapshotPresentation.present(acquisition, options:) (RunnerTests+SnapshotCapturePlan.swift, captureWithBackend).
Wider counts on main (same files): 9 production RawAXNode( construction sites across 5 files; 5 call sites reading RunnerSynthesizedGesture.interfaceOrientation(forApplication:); 13 sites reading the app viewport (app.frame, safeSnapshotViewport, app.windows.firstMatch.frame). The 2026-08-31 presenter-convergence review already flagged "presented-depth frontier coupling (acquisition consults fold decisions mid-traversal) must be redesigned before any runner demotion".
Task
Make the recursive tier acquire first and present after, like the private-AX tier already does, so that geometry normalization becomes one pass over the flat [RawAXNode] array.
- Establish what the mid-walk fold buys. The recursive tier walks
XCUIElementSnapshot.children, which XCTest materialized in one bulk snapshot; the frontier does not save AX round trips, it saves walk time and node construction on subtrees the fold would clip. Measure on the deep-tree benchmark (scripts/bench harness used for #2424 / the deep-tree work; seedocs/for the recipe) and on the test app: raw node count walked with and without the frontier, and wall time. Report the numbers before changing anything. If the frontier is load-bearing for a known hostile screen (Bluesky feed, #1105/#1156), keep a bound that does not consult presentation: a raw depth / node cap is acceptable, a fold decision is not. - Acquire raw, then normalize, then present. Walkers produce
RawAXNodewith reported frames only. AddSnapshotGeometrySpace.normalized(nodes:viewport:interfaceOrientation:) -> [RawAXNode]next to the existing rule inapple/snapshot-presentation/Sources/AgentDeviceSnapshotPresentation/SnapshotCoordinateSpace.swift: walk the array once, derive each node's space fromtype,parentIndexandrectwith the existingspace(reportedBySurfaceHost:…)rule, rewriterectthroughorientedFrame(of:), and recomputehittablewithSnapshotGeometry.isGeometricallyActionable. Run it once incaptureWithBackendbetween acquisition andSnapshotPresentation.present. Delete thegeometrySpace/parentIsWindowthreading from every walker and the collapsed-tab guard (collapsed tab containers sit under the app's own window, so a post-pass keyed on ancestry never touches them). - Bring the query-sweep tier under the same pass. Its flat nodes hang off the application root; with no window ancestry they declare no space and the pass leaves them alone, which is the current behaviour, but now for a structural reason rather than an omission. Say so in ADR 0004.
- Keep the disclosure.
unplacedSurfaceHostCount(added after #2653) already runs post-acquisition; it should now run after normalization and be zero whenever the orientation was readable.
Acceptance criteria
-
SnapshotGeometrySpaceis referenced from exactly one production site inapple/runner/**(the normalization call incaptureWithBackend); no walker takes ageometrySpaceorparentIsWindowparameter. - Every existing runner unit test that asserts rotated output still passes unchanged:
testPrivateAXAcquisitionPublishesATurnedSurfaceHostInAppOrientationSpace, theRunnerTests+Snapshot.swifttests added in #2653,testStampedPayloadDisclosesUnplacedSurfaceHosts. The package suite (swift test --package-path apple/snapshot-presentation) gains a test fornormalized(…)that replays therotationCasesofcontracts/fixtures/window-coordinate-space.jsonthrough a two-window tree. -
contracts/fixtures/snapshot-presentation-conformance.json,ios-snapshot-backend-conformance.jsonandios-ax-recovery-conformance.jsonare unchanged and green in both languages. - Live:
snapshot -i --jsonon the test app in portrait is byte-identical tomainon the runner path (confirm the producer isapple-runner, not the bridge: the bridge fingerprint is the "does not provide hittability evidence" warning). In landscape with the keyboard up, the band and key rects match the #2653 numbers ((75,198,724,202)band, keyqat(77,203,72,45)). - Benchmark: capture p50 and p95 on the deep-tree harness and on the Bluesky feed not regressed by more than 5 % against
main, with the numbers in the PR. If the frontier had to be replaced by a raw cap, the cap and the screens it was calibrated on are named in ADR 0004. -
pnpm check:affected --rungreen; runner units green underAGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1;pnpm test:ios-snapshot-differentialgreen.
Non-goals
Runner demotion or moving presentation to the host: that is a separate, measurement-gated decision (ADR 0004 host-side ownership boundary; two presenters was the measured optimum on 2026-08-31). Touching the Android or bridge producers.
Related: #2612, #2653, #1797 (acquire/present split), #2188 / #2199 (iOS snapshot convergence), ADR 0004.
- Dominant language
- TypeScript
- Stars
- 4.8k
- Forks
- 315
- Avg merge
- 11h 17m
- Merged PRs (30d)
- 536
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from callstack/agent-device
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
callstack/agent-device#1869 ·
Maintainers usually reply within 1 day
-
needs-triage refactor
Difficulty 5/5 Over a week Newbie friendliness 25/100
callstack/agent-device#3116 · 4 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
callstack/agent-device#3106 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
callstack/agent-device#3105 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
callstack/agent-device#3104 ·
Maintainers usually reply within 1 day
All issues in callstack/agent-device
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
external-issue to-triage
Difficulty 1/5 Under an hour Newbie friendliness 90/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
diegosouzapw/OmniRoute#15401 ·
Maintainers usually reply within 2 days
-
Sign the pledgeOpen
Difficulty 1/5 Under an hour Newbie friendliness 95/100
input-output-hk/devx-updates#163 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
code-yeongyu/oh-my-openagent#9454 ·
Maintainers usually reply within 1 day