refactor(apple-runner): classify AX snapshot failure once at acquisition
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 42/100
- Issue-Typ
- Refactoring
- Klarheit
- Klar beschrieben
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- swift
- Bereich
- mobile-dev, testing-qa
Rechercherichtung
Start by inventorying SnapshotCaptureFailure constructors and isAxSnapshotFailure consumers in apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/, especially RunnerTests+SnapshotAcquisition.swift, RunnerTests+SnapshotCapturePlan.swift, and RunnerTests+SnapshotPresentationInvariant.swift. Extend the named UnitTests snapshot, capture-plan, and presentation-invariant suites; done means downstream classification is code-only, unrelated AX-like prose stays unrelated, and recognized native failures preserve recovery behavior.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Classify native AX snapshot failure once at acquisition
Deletion-first acceptance
This is part of #2803's simplify-and-shrink initiative. Before production simplification, name the production mechanism, duplicated decision, state, fallback, forwarding layer, or public/internal interface this change removes. Compare deletion and inlining against extraction. Moving the same state into another file is not a deletion case.
Every PR reports baseline/final SHAs; added, deleted and net production lines; test/fixture/documentation changes separately; and the exact mechanisms removed. Use a rename-aware diff (git diff --numstat -M BASE...HEAD) and account for all affected production paths, including destination packages. Moving tests out of production files, generated output, formatting churn, and moving code elsewhere are not production-code deletion. Preserve readable code; no compressed formatting to hit a number.
Default for simplification work: net production reduction. A required behavior-preserving size split may be line-neutral. Growth needs an explicit, quantified explanation of the correctness or type guarantee it buys, why deletion/inlining cannot achieve it, and what old mechanism is removed; it is not automatically acceptable because ownership looks cleaner. If the only benefit is relocating fields or adding a wrapper, defer the change. Required regression tests and a minimal correctness fix are allowed to grow without inventing unrelated deletion to offset them.
Priority and scope gate
Early small deletion: remove downstream AX message classification while retaining necessary native-edge normalization. Prefer deleting the redundant branch/helper over introducing a new error hierarchy or classifier abstraction. Report its production delta separately from added negative regression cases. Serialize shared runner Swift edits; #2911 is already merged.
Problem and evidence
The snapshot acquisition edge already turns a recognized AX serialization failure into SnapshotCaptureFailure(code: axSnapshotErrorCode, ...). Downstream policy nevertheless classifies it again from the human message. That allows wording on an unrelated failure to affect recovery, the quality reason, and terminal target invalidation.
Verified against origin/main at 5712a3552e209c9c853bc1f42cb3e007f1213139: apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotAcquisition.swift:116–134 performs native classification, :159–181 constructs the typed failure and then permits message-based reclassification. RunnerTests+SnapshotCapturePlan.swift:361 retains AX failure for terminal policy, and RunnerTests+SnapshotPresentationInvariant.swift:17 derives its quality reason. The latest placeholder changes do not change these sites.
Intended design
Keep unavoidable private-API text interpretation at the native acquisition edge. Above that edge, a snapshot failure's machine code is the sole AX classification authority. Preserve the current recognized native messages, public code, diagnostic message, hint, raw-snapshot failure behavior, and interactive fail-closed behavior.
The intended interface is deliberately small:
static func isAxSnapshotFailure(_ failure: SnapshotCaptureFailure) -> Bool {
failure.code == axSnapshotErrorCode
}
Use the existing failure carrier; do not add a second isAXFailure flag that can disagree with its code. Keep or name a private acquisition-edge classifier for actual native error/exception evidence. If NSError or NSException exposes reliable structured evidence, characterize it first and prefer it only where proven. This issue does not depend on discovering a new native code or eliminating all native-edge text parsing.
Implementation and acceptance
- Inventory every constructor of
SnapshotCaptureFailureand every consumer ofisAxSnapshotFailure, including private-AX and presentation error mapping. Prove each genuine AX path reaches the same machine code before tightening consumers. - Add paired behavioral regressions through existing product seams: a typed AX failure with neutral wording remains AX; an unrelated typed failure containing identical AX-looking prose remains unrelated. Assert both quality classification and the terminal capture-plan consequence.
- Make downstream classification code-only. If a producer currently relies on downstream text classification, move that normalization to its actual acquisition edge while retaining its observed classification and public error details.
- Retain native exception handling, source diagnostic text, and the capture-plan ordering. Keep tests with the owning modules; do not add test-only production exports.
Before/after measures: there is no downstream AX message sniff after normalization; all previously recognized native AX failures keep their code and recovery result; changing diagnostic prose cannot alter quality or terminal policy. Plant the old downstream text fallback and observe the unrelated-code negative fail. A native-edge fixture still recognizes each currently supported signature, including the closest negative that resembles an AX message but lacks the full recognized signature.
Validation
Extend the owning snapshot acquisition/quality/terminal tests, including UnitTests/RunnerTests+SnapshotTests.swift, RunnerTests+SnapshotCapturePlanTests.swift, and RunnerTests+SnapshotPresentationInvariantTests.swift as appropriate to the actual call sites. Pure classification belongs in the macOS host XCTest lane; native exception capture semantics belong in the iOS simulator lane. Preserve RunnerTests+SnapshotCapturePlanOccupancyTests.swift as integration proof of recovery while abandoned native work remains outstanding.
Run pnpm check:xctest-selection, pnpm build:xcuitest:ios, and pnpm build:xcuitest:macos; run the selected host/simulator XCTest cases through the repository's existing harness and record the selected cases. Finish with pnpm check:affected --run and all reported exact-head obligations. Do not substitute compilation for XCTest evidence or claim a real-device AX failure was reproduced unless measured.
Scope and readiness
Ready for implementation after the constructor inventory; unexpected producer classification must be characterized before changing it. Independent child of umbrella #2803, with no dependency on broader capture-plan extraction. No retry, timeout, fallback order, error-message cleanup, or wire-schema changes. Expected effort: S–M; risk: medium because AX classification controls fail-closed behavior.
Source paths in this issue are rooted at apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/; extension names abbreviate the RunnerTests+ prefix where stated. Tests live in its UnitTests/ directory. For a fresh worktree, run pnpm install --frozen-lockfile && pnpm build before the focused checks. Read docs/agents/testing.md and docs/agents/device-verification.md for native lane selection and build freshness.
- Vorherrschende Sprache
- TypeScript
- Sterne
- 5k
- Forks
- 334
- Ø Merge
- 12 Std. 2 Min.
- Gemergte PRs (30 T.)
- 678
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Keine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus callstack/agent-device
-
`devices` reports a phantom Android emulator named after the emulator's crashdata INFO bannerEvtl. vergeben @airowe hat das vor 1 Tag übernommen. Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
callstack/agent-device#3716 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 68/100
callstack/agent-device#1869 ·
Maintainer antworten meist innerhalb von 1 Tag
-
backlog refactor
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 8/100
callstack/agent-device#3759 ·
Maintainer antworten meist innerhalb von 1 Tag
-
backlog enhancement needs-triage
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 8/100
callstack/agent-device#3758 ·
Maintainer antworten meist innerhalb von 1 Tag
-
enhancement ready-for-agent
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 15/100
callstack/agent-device#3757 ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in callstack/agent-device
Ähnliche Issues
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
polarsource/polar#15580 ·
Maintainer antworten meist innerhalb von 1 Tag
-
accepted bug via-triage
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 69/100
pingdotgg/t3code#20931 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug/1-unconfirmed kind/bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 83/100
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
ranxianglei/billion-context#3583 ·
Maintainer antworten meist innerhalb von 1 Tag