refactor(apple-runner): classify AX snapshot failure once at acquisition
Maintainer thường phản hồi trong vòng 1 ngày
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức phù hợp với người mới
- 42/100
- Loại issue
- Tái cấu trúc
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Công nghệ
- swift
- Lĩnh vực
- mobile-dev, testing-qa
Hướng nghiên cứu
Start by inventorying SnapshotCaptureFailure constructors and isAxSnapshotFailure consumers in apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/, especially RunnerTests+SnapshotAcquisition.swift, RunnerTests+SnapshotCapturePlan.swift, and RunnerTests+SnapshotPresentationInvariant.swift. Extend the named UnitTests snapshot, capture-plan, and presentation-invariant suites; done means downstream classification is code-only, unrelated AX-like prose stays unrelated, and recognized native failures preserve recovery behavior.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Classify native AX snapshot failure once at acquisition
Deletion-first acceptance
This is part of #2803's simplify-and-shrink initiative. Before production simplification, name the production mechanism, duplicated decision, state, fallback, forwarding layer, or public/internal interface this change removes. Compare deletion and inlining against extraction. Moving the same state into another file is not a deletion case.
Every PR reports baseline/final SHAs; added, deleted and net production lines; test/fixture/documentation changes separately; and the exact mechanisms removed. Use a rename-aware diff (git diff --numstat -M BASE...HEAD) and account for all affected production paths, including destination packages. Moving tests out of production files, generated output, formatting churn, and moving code elsewhere are not production-code deletion. Preserve readable code; no compressed formatting to hit a number.
Default for simplification work: net production reduction. A required behavior-preserving size split may be line-neutral. Growth needs an explicit, quantified explanation of the correctness or type guarantee it buys, why deletion/inlining cannot achieve it, and what old mechanism is removed; it is not automatically acceptable because ownership looks cleaner. If the only benefit is relocating fields or adding a wrapper, defer the change. Required regression tests and a minimal correctness fix are allowed to grow without inventing unrelated deletion to offset them.
Priority and scope gate
Early small deletion: remove downstream AX message classification while retaining necessary native-edge normalization. Prefer deleting the redundant branch/helper over introducing a new error hierarchy or classifier abstraction. Report its production delta separately from added negative regression cases. Serialize shared runner Swift edits; #2911 is already merged.
Problem and evidence
The snapshot acquisition edge already turns a recognized AX serialization failure into SnapshotCaptureFailure(code: axSnapshotErrorCode, ...). Downstream policy nevertheless classifies it again from the human message. That allows wording on an unrelated failure to affect recovery, the quality reason, and terminal target invalidation.
Verified against origin/main at 5712a3552e209c9c853bc1f42cb3e007f1213139: apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotAcquisition.swift:116–134 performs native classification, :159–181 constructs the typed failure and then permits message-based reclassification. RunnerTests+SnapshotCapturePlan.swift:361 retains AX failure for terminal policy, and RunnerTests+SnapshotPresentationInvariant.swift:17 derives its quality reason. The latest placeholder changes do not change these sites.
Intended design
Keep unavoidable private-API text interpretation at the native acquisition edge. Above that edge, a snapshot failure's machine code is the sole AX classification authority. Preserve the current recognized native messages, public code, diagnostic message, hint, raw-snapshot failure behavior, and interactive fail-closed behavior.
The intended interface is deliberately small:
static func isAxSnapshotFailure(_ failure: SnapshotCaptureFailure) -> Bool {
failure.code == axSnapshotErrorCode
}
Use the existing failure carrier; do not add a second isAXFailure flag that can disagree with its code. Keep or name a private acquisition-edge classifier for actual native error/exception evidence. If NSError or NSException exposes reliable structured evidence, characterize it first and prefer it only where proven. This issue does not depend on discovering a new native code or eliminating all native-edge text parsing.
Implementation and acceptance
- Inventory every constructor of
SnapshotCaptureFailureand every consumer ofisAxSnapshotFailure, including private-AX and presentation error mapping. Prove each genuine AX path reaches the same machine code before tightening consumers. - Add paired behavioral regressions through existing product seams: a typed AX failure with neutral wording remains AX; an unrelated typed failure containing identical AX-looking prose remains unrelated. Assert both quality classification and the terminal capture-plan consequence.
- Make downstream classification code-only. If a producer currently relies on downstream text classification, move that normalization to its actual acquisition edge while retaining its observed classification and public error details.
- Retain native exception handling, source diagnostic text, and the capture-plan ordering. Keep tests with the owning modules; do not add test-only production exports.
Before/after measures: there is no downstream AX message sniff after normalization; all previously recognized native AX failures keep their code and recovery result; changing diagnostic prose cannot alter quality or terminal policy. Plant the old downstream text fallback and observe the unrelated-code negative fail. A native-edge fixture still recognizes each currently supported signature, including the closest negative that resembles an AX message but lacks the full recognized signature.
Validation
Extend the owning snapshot acquisition/quality/terminal tests, including UnitTests/RunnerTests+SnapshotTests.swift, RunnerTests+SnapshotCapturePlanTests.swift, and RunnerTests+SnapshotPresentationInvariantTests.swift as appropriate to the actual call sites. Pure classification belongs in the macOS host XCTest lane; native exception capture semantics belong in the iOS simulator lane. Preserve RunnerTests+SnapshotCapturePlanOccupancyTests.swift as integration proof of recovery while abandoned native work remains outstanding.
Run pnpm check:xctest-selection, pnpm build:xcuitest:ios, and pnpm build:xcuitest:macos; run the selected host/simulator XCTest cases through the repository's existing harness and record the selected cases. Finish with pnpm check:affected --run and all reported exact-head obligations. Do not substitute compilation for XCTest evidence or claim a real-device AX failure was reproduced unless measured.
Scope and readiness
Ready for implementation after the constructor inventory; unexpected producer classification must be characterized before changing it. Independent child of umbrella #2803, with no dependency on broader capture-plan extraction. No retry, timeout, fallback order, error-message cleanup, or wire-schema changes. Expected effort: S–M; risk: medium because AX classification controls fail-closed behavior.
Source paths in this issue are rooted at apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/; extension names abbreviate the RunnerTests+ prefix where stated. Tests live in its UnitTests/ directory. For a fresh worktree, run pnpm install --frozen-lockfile && pnpm build before the focused checks. Read docs/agents/testing.md and docs/agents/device-verification.md for native lane selection and build freshness.
- Ngôn ngữ chính
- TypeScript
- Star
- 4.7k
- Fork
- 304
- Merge trung bình
- 11 giờ 6 phút
- Pull request đã merge (30 ngày)
- 521
Chuẩn bị môi trường
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của callstack/agent-device
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 88/100
callstack/agent-device#2998 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 84/100
callstack/agent-device#2995 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Proxy lease refuses every iOS install/open: connection platform 'ios' compared with internal 'apple'Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
callstack/agent-device#2962 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
callstack/agent-device#1869 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Maestro `eraseText` fails on real Android devices: `test` and `replay` cannot opt in to the test IMEĐang mở
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 74/100
callstack/agent-device#2997 ·
Maintainer thường phản hồi trong vòng 1 ngày
Tất cả issue của callstack/agent-device
Issue tương tự
-
Độ khó 1/5 1-3 giờ Mức phù hợp với người mới 88/100
supabase/agent-skills#611 ·
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 68/100
polka-codes/test#345 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 1-3 giờ Mức phù hợp với người mới 92/100
GoogleChromeLabs/project-sesame#217 ·
Maintainer thường phản hồi trong vòng 12 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 74/100
solana-foundation/solana-com#2202 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100