Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Selector taps miss on a rotated iOS simulator: the tap point is not rotated (REST) or is transposed (CLI)

Aperta
  • Qualcosa non va?
#89 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
ios, rust
Ambito
api, cli, mobile-dev

Direzione di ricerca

Start with tap_point_from_snapshot in api/accessibility_query.rs:207 and normalize_accessibility_point_for_display in main/tap_target.rs:164, then trace GET /accessibility-point and display rotation state. Verify the mapping for all four quarter turns through both REST and CLI paths, while preserving SpringBoard alert coordinates and repeated REST rotations; finish by confirming selector taps land on the selected element.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Version: simdeck 0.2.0 (82e61b3), Xcode 26.6, iOS 26.5 simulator (iPad Pro 11-inch M5), headless (simctl boot, no video session).

What happens: on a simulator turned to landscape, a selector tap (POST /api/simulators/{udid}/action with {"action":"tap","selector":{...}}) returns ok but lands somewhere else. Raw normalized taps land correctly once the caller rotates the point itself.

Repro:

  1. Boot an iPad simulator, open any app with a button near a corner (Settings works).
  2. POST .../action {"action":"rotateLeft"}.
  3. POST .../action {"action":"tap","selector":{"label":"<that button>"}} → {"ok":true}, and the button is not pressed.

Why: the accessibility tree reports an app's frames in the turned UI, while touches are injected in the unrotated (portrait) screen.

  • REST path: tap_point_from_snapshot (api/accessibility_query.rs:207) normalizes the element centre by the root frame and sends that as the touch point, with no rotation.
  • CLI path: normalize_accessibility_point_for_display (main/tap_target.rs:164) swaps x and y when root and display orientation differ. A swap is a mirror, not a rotation, so it is wrong in both landscape directions.

The correct mapping depends on the direction. With u, v the element centre normalized in the turned UI, and the device turned q quarter turns:

q touch x touch y
1 1 - v u
2 1 - u 1 - v
3 v 1 - u

So the swap (v, u) should become (1 - v, u) or (v, 1 - u). The direction cannot be read from the tree: the root frame is identical in both landscape directions. It can be read from GET .../accessibility-point, which hit-tests in unrotated screen points and answers with UI-space frames. Hit-test a known element's centre under each candidate turn, and the one that returns that element is the device's turn. Repeated rotateLeft calls also do not accumulate: each REST call creates a new display bridge whose _deviceRotationDegrees starts at 0, so you cannot reach upside-down portrait through the REST API.

SpringBoard alerts (e.g. "Open in …?") on a turned device report their children in unrotated portrait points under a root whose frame is the turned screen, so those must not be rotated.

We work around this on the client side for now (rotate the point ourselves and send a plain normalized tap). Happy to test a fix.

Lingua principale
Rust
Stelle
150
Fork
14
Merge medio
1m
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NativeScript/SimDeck

Tutte le issue di NativeScript/SimDeck

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.