Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

iOS: XCTest-channel penalty keeps the runner in private-AX fallback most of a long session (fixed 3 s / 120 s, no override)

Aperta
  • Qualcosa non va?
#3,844 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
ios, typescript
Ambito
mobile, performance, testing

Direzione di ricerca

Read the snapshot penalty and capture plan code under dist/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/, starting with RunnerTests+SnapshotTiming.swift, RunnerTests+SnapshotCapturePlan.swift, and the snapshot CLI entry point. Check the related issue #1542 and presentedSurface work in #3565 before choosing a scope. Done means the requested penalty controls and CLI backend option behave as specified, with tests covering single query timeouts and recovery probes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

On a heavy UIKit app (iPad simulator), one slow or timed-out XCTest-channel snapshot tier puts the runner into private-AX fallback for 120 s. A timed-out 1 s recursiveTree recovery probe re-arms it; that probe runs when the private-AX capture is rejected. And once the penalty expires, the next capture's 1 s query sweep often times out and penalizes the channel again. In one long-running session log, 71 % of snapshot commands ran with the XCTest channel deferred.

In our captures, private-AX returned presented screens together with the screen underneath them. A test then can't reliably tell which screen is top-most, and an element of the covered screen looks "visible". For read-only checks this is mostly harmless. For steps that must act on the right screen (bulk select / destructive bulk actions in a list), it's unsafe, so we refuse those steps whenever a fallback happened in the last 120 s.

Version / environment

  • agent-device 0.21.25 (pinned by @e2e-dev/mobile 0.12.0, used through the e2e 0.20.0 runner). The penalty logic is unchanged in 0.21.28 and 0.21.29-nightly.20261020.
  • Xcode 27.0 (27A266a), iOS 27.0 simulator runtime, iPad Air 11-inch (M4) simulators
  • UIKit app with long table views

What the source shows (0.21.25, dist/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/)

  • An XCTest-channel tier penalizes the channel when it times out, exhausts its deadline, or takes longer than snapshotXCTestSlowCaptureThreshold (3 s): RunnerTests+SnapshotTiming.swift:119-155, constant at RunnerTests.swift:139. Budgets: tree slice 8 s (RunnerTests.swift:143), query sweep 1.0 s (RunnerTests+Snapshot.swift:134-143), viewport read ≤ 1.0 s (RunnerTests+SnapshotAcquisition.swift:50-56). A system-modal probe timeout also penalizes (RunnerTests+Snapshot.swift:360).
  • penalizeSnapshotXCTestChannel sets penaltyUntil = now + 120 s (RunnerTests+SnapshotCapturePlan.swift:133-143, RunnerTests.swift:122). While it's active, the plan puts private-AX first, plus recursiveTree with a 1 s slice (RunnerTests+SnapshotCapturePlan.swift:110, 256-277).
  • Each slow or timed-out XCTest attempt during the penalty re-arms it. In practice that's the 1 s tree probe, which runs when the private-AX tier fails or is rejected (RunnerTests+SnapshotCapturePlan.swift:272, 391-431). penaltySuppressed applies only to the one-shot warm-up after a target PID change (:304, 373-378; RunnerTests+Lifecycle.swift:254).
  • It's cleared only by targetReset / a new target PID (RunnerTests+Lifecycle.swift:228, 252), by the 120 s expiring, or by a runner restart.
  • The thresholds and the duration are let constants. No env var, config or CLI flag changes them. preferredBackend (the SDK snapshot option; daemon flag key snapshotPreferredBackend) does override the plan, even under penalty (RunnerTests+SnapshotCapturePlan.swift:243-255), but the CLI snapshot command doesn't expose it (still true in 0.21.28).

Evidence (runner.log)

  • One SNAPSHOT_XCTEST_CHANNEL_PENALIZED … reason=queries_backend_timeout was followed by 70 PRIVATE_AX_SNAPSHOT_USED captures in the next ~2 min.
  • One session's runner.log, covering several days and 43 runner launches, has 273 PENALIZED lines (196 queries_backend_timeout, 72 tree_backend_timeout, 5 slow_tree_capture_…) against 7 PENALTY_CLEARED. All 50 penalties that re-armed inside 120 s are tree_backend_timeout. Most of the queries_backend_timeout ones come shortly after the previous penalty expired (median about 21 s). 41,053 of 57,766 snapshot commands logged XCTEST_CHANNEL_DEFERRED. Another session's log has 115 PENALIZED (60 / 51 / 4) against 1 cleared.

Related: #1542.

Requests

  1. Make the slow threshold and the penalty duration configurable (env or CLI / session option).
  2. Don't penalize the channel for a single 1 s query-sweep timeout alone, and don't let the 1 s recovery probe re-arm the penalty.
  3. Expose preferredBackend on the CLI snapshot command (and as a session default), so a client can force recursiveTree for steps that need an accurate, top-most-screen tree.
  4. Optionally, mark private-AX nodes that belong to a covered (non-top-most) presented screen, so clients can filter them out. (0.21.28's presentedSurface fact, #3565, may cover part of this; not yet tested on our side.)

Happy to provide more logs or test a nightly.

Lingua principale
TypeScript
Stelle
5.1k
Fork
339
Merge medio
12h 23m
PR unite (30g)
730

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di callstack/agent-device

Tutte le issue di callstack/agent-device

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.