Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Selector taps miss on a rotated iOS simulator: the tap point is not rotated (REST) or is transposed (CLI)

Abierto
  • ¿Algo no está bien?
#89 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
55/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
ios, rust
Área
api, cli, mobile-dev

Línea de trabajo

Start with tap_point_from_snapshot in api/accessibility_query.rs:207 and normalize_accessibility_point_for_display in main/tap_target.rs:164, then trace GET /accessibility-point and display rotation state. Verify the mapping for all four quarter turns through both REST and CLI paths, while preserving SpringBoard alert coordinates and repeated REST rotations; finish by confirming selector taps land on the selected element.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Version: simdeck 0.2.0 (82e61b3), Xcode 26.6, iOS 26.5 simulator (iPad Pro 11-inch M5), headless (simctl boot, no video session).

What happens: on a simulator turned to landscape, a selector tap (POST /api/simulators/{udid}/action with {"action":"tap","selector":{...}}) returns ok but lands somewhere else. Raw normalized taps land correctly once the caller rotates the point itself.

Repro:

  1. Boot an iPad simulator, open any app with a button near a corner (Settings works).
  2. POST .../action {"action":"rotateLeft"}.
  3. POST .../action {"action":"tap","selector":{"label":"<that button>"}} → {"ok":true}, and the button is not pressed.

Why: the accessibility tree reports an app's frames in the turned UI, while touches are injected in the unrotated (portrait) screen.

  • REST path: tap_point_from_snapshot (api/accessibility_query.rs:207) normalizes the element centre by the root frame and sends that as the touch point, with no rotation.
  • CLI path: normalize_accessibility_point_for_display (main/tap_target.rs:164) swaps x and y when root and display orientation differ. A swap is a mirror, not a rotation, so it is wrong in both landscape directions.

The correct mapping depends on the direction. With u, v the element centre normalized in the turned UI, and the device turned q quarter turns:

q touch x touch y
1 1 - v u
2 1 - u 1 - v
3 v 1 - u

So the swap (v, u) should become (1 - v, u) or (v, 1 - u). The direction cannot be read from the tree: the root frame is identical in both landscape directions. It can be read from GET .../accessibility-point, which hit-tests in unrotated screen points and answers with UI-space frames. Hit-test a known element's centre under each candidate turn, and the one that returns that element is the device's turn. Repeated rotateLeft calls also do not accumulate: each REST call creates a new display bridge whose _deviceRotationDegrees starts at 0, so you cannot reach upside-down portrait through the REST API.

SpringBoard alerts (e.g. "Open in …?") on a turned device report their children in unrotated portrait points under a root whose frame is the turned screen, so those must not be rotated.

We work around this on the client side for now (rotate the point ourselves and send a plain normalized tap). Happy to test a fix.

Lenguaje dominante
Rust
Estrellas
150
Forks
14
Merge medio
1 min
PR fusionados (30 d)
1

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NativeScript/SimDeck

Todos los issues de NativeScript/SimDeck

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.