Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Selector taps miss on a rotated iOS simulator: the tap point is not rotated (REST) or is transposed (CLI)

Ouverte
  • Quelque chose ne va pas ?
#89 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
55/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
ios, rust
Domaine
api, cli, mobile-dev

Piste de recherche

Start with tap_point_from_snapshot in api/accessibility_query.rs:207 and normalize_accessibility_point_for_display in main/tap_target.rs:164, then trace GET /accessibility-point and display rotation state. Verify the mapping for all four quarter turns through both REST and CLI paths, while preserving SpringBoard alert coordinates and repeated REST rotations; finish by confirming selector taps land on the selected element.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Version: simdeck 0.2.0 (82e61b3), Xcode 26.6, iOS 26.5 simulator (iPad Pro 11-inch M5), headless (simctl boot, no video session).

What happens: on a simulator turned to landscape, a selector tap (POST /api/simulators/{udid}/action with {"action":"tap","selector":{...}}) returns ok but lands somewhere else. Raw normalized taps land correctly once the caller rotates the point itself.

Repro:

  1. Boot an iPad simulator, open any app with a button near a corner (Settings works).
  2. POST .../action {"action":"rotateLeft"}.
  3. POST .../action {"action":"tap","selector":{"label":"<that button>"}} → {"ok":true}, and the button is not pressed.

Why: the accessibility tree reports an app's frames in the turned UI, while touches are injected in the unrotated (portrait) screen.

  • REST path: tap_point_from_snapshot (api/accessibility_query.rs:207) normalizes the element centre by the root frame and sends that as the touch point, with no rotation.
  • CLI path: normalize_accessibility_point_for_display (main/tap_target.rs:164) swaps x and y when root and display orientation differ. A swap is a mirror, not a rotation, so it is wrong in both landscape directions.

The correct mapping depends on the direction. With u, v the element centre normalized in the turned UI, and the device turned q quarter turns:

q touch x touch y
1 1 - v u
2 1 - u 1 - v
3 v 1 - u

So the swap (v, u) should become (1 - v, u) or (v, 1 - u). The direction cannot be read from the tree: the root frame is identical in both landscape directions. It can be read from GET .../accessibility-point, which hit-tests in unrotated screen points and answers with UI-space frames. Hit-test a known element's centre under each candidate turn, and the one that returns that element is the device's turn. Repeated rotateLeft calls also do not accumulate: each REST call creates a new display bridge whose _deviceRotationDegrees starts at 0, so you cannot reach upside-down portrait through the REST API.

SpringBoard alerts (e.g. "Open in …?") on a turned device report their children in unrotated portrait points under a root whose frame is the turned screen, so those must not be rotated.

We work around this on the client side for now (rotate the point ourselves and send a plain normalized tap). Happy to test a fix.

Langage dominant
Rust
Étoiles
150
Forks
14
Merge moyen
1 min
PR mergées (30 j)
1

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de NativeScript/SimDeck

Toutes les issues de NativeScript/SimDeck

Issues similaires

Plus d'issues Rust

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.