Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

iOS: XCTest-channel penalty keeps the runner in private-AX fallback most of a long session (fixed 3 s / 120 s, no override)

Ouverte
  • Quelque chose ne va pas ?
#3,844 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
35/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
ios, typescript
Domaine
mobile, performance, testing

Piste de recherche

Read the snapshot penalty and capture plan code under dist/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/, starting with RunnerTests+SnapshotTiming.swift, RunnerTests+SnapshotCapturePlan.swift, and the snapshot CLI entry point. Check the related issue #1542 and presentedSurface work in #3565 before choosing a scope. Done means the requested penalty controls and CLI backend option behave as specified, with tests covering single query timeouts and recovery probes.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

bug

Summary

On a heavy UIKit app (iPad simulator), one slow or timed-out XCTest-channel snapshot tier puts the runner into private-AX fallback for 120 s. A timed-out 1 s recursiveTree recovery probe re-arms it; that probe runs when the private-AX capture is rejected. And once the penalty expires, the next capture's 1 s query sweep often times out and penalizes the channel again. In one long-running session log, 71 % of snapshot commands ran with the XCTest channel deferred.

In our captures, private-AX returned presented screens together with the screen underneath them. A test then can't reliably tell which screen is top-most, and an element of the covered screen looks "visible". For read-only checks this is mostly harmless. For steps that must act on the right screen (bulk select / destructive bulk actions in a list), it's unsafe, so we refuse those steps whenever a fallback happened in the last 120 s.

Version / environment

  • agent-device 0.21.25 (pinned by @e2e-dev/mobile 0.12.0, used through the e2e 0.20.0 runner). The penalty logic is unchanged in 0.21.28 and 0.21.29-nightly.20261020.
  • Xcode 27.0 (27A266a), iOS 27.0 simulator runtime, iPad Air 11-inch (M4) simulators
  • UIKit app with long table views

What the source shows (0.21.25, dist/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/)

  • An XCTest-channel tier penalizes the channel when it times out, exhausts its deadline, or takes longer than snapshotXCTestSlowCaptureThreshold (3 s): RunnerTests+SnapshotTiming.swift:119-155, constant at RunnerTests.swift:139. Budgets: tree slice 8 s (RunnerTests.swift:143), query sweep 1.0 s (RunnerTests+Snapshot.swift:134-143), viewport read ≤ 1.0 s (RunnerTests+SnapshotAcquisition.swift:50-56). A system-modal probe timeout also penalizes (RunnerTests+Snapshot.swift:360).
  • penalizeSnapshotXCTestChannel sets penaltyUntil = now + 120 s (RunnerTests+SnapshotCapturePlan.swift:133-143, RunnerTests.swift:122). While it's active, the plan puts private-AX first, plus recursiveTree with a 1 s slice (RunnerTests+SnapshotCapturePlan.swift:110, 256-277).
  • Each slow or timed-out XCTest attempt during the penalty re-arms it. In practice that's the 1 s tree probe, which runs when the private-AX tier fails or is rejected (RunnerTests+SnapshotCapturePlan.swift:272, 391-431). penaltySuppressed applies only to the one-shot warm-up after a target PID change (:304, 373-378; RunnerTests+Lifecycle.swift:254).
  • It's cleared only by targetReset / a new target PID (RunnerTests+Lifecycle.swift:228, 252), by the 120 s expiring, or by a runner restart.
  • The thresholds and the duration are let constants. No env var, config or CLI flag changes them. preferredBackend (the SDK snapshot option; daemon flag key snapshotPreferredBackend) does override the plan, even under penalty (RunnerTests+SnapshotCapturePlan.swift:243-255), but the CLI snapshot command doesn't expose it (still true in 0.21.28).

Evidence (runner.log)

  • One SNAPSHOT_XCTEST_CHANNEL_PENALIZED … reason=queries_backend_timeout was followed by 70 PRIVATE_AX_SNAPSHOT_USED captures in the next ~2 min.
  • One session's runner.log, covering several days and 43 runner launches, has 273 PENALIZED lines (196 queries_backend_timeout, 72 tree_backend_timeout, 5 slow_tree_capture_…) against 7 PENALTY_CLEARED. All 50 penalties that re-armed inside 120 s are tree_backend_timeout. Most of the queries_backend_timeout ones come shortly after the previous penalty expired (median about 21 s). 41,053 of 57,766 snapshot commands logged XCTEST_CHANNEL_DEFERRED. Another session's log has 115 PENALIZED (60 / 51 / 4) against 1 cleared.

Related: #1542.

Requests

  1. Make the slow threshold and the penalty duration configurable (env or CLI / session option).
  2. Don't penalize the channel for a single 1 s query-sweep timeout alone, and don't let the 1 s recovery probe re-arm the penalty.
  3. Expose preferredBackend on the CLI snapshot command (and as a session default), so a client can force recursiveTree for steps that need an accurate, top-most-screen tree.
  4. Optionally, mark private-AX nodes that belong to a covered (non-top-most) presented screen, so clients can filter them out. (0.21.28's presentedSurface fact, #3565, may cover part of this; not yet tested on our side.)

Happy to provide more logs or test a nightly.

Langage dominant
TypeScript
Étoiles
5.1k
Forks
339
Merge moyen
11 h 59 min
PR mergées (30 j)
736

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de callstack/agent-device

Toutes les issues de callstack/agent-device

Issues similaires

Plus d'issues TypeScript

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.