Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Selector taps miss on a rotated iOS simulator: the tap point is not rotated (REST) or is transposed (CLI)

Aberta
  • Algo errado?
#89 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
55/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
ios, rust
Domínio
api, cli, mobile-dev

Direção de pesquisa

Start with tap_point_from_snapshot in api/accessibility_query.rs:207 and normalize_accessibility_point_for_display in main/tap_target.rs:164, then trace GET /accessibility-point and display rotation state. Verify the mapping for all four quarter turns through both REST and CLI paths, while preserving SpringBoard alert coordinates and repeated REST rotations; finish by confirming selector taps land on the selected element.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Version: simdeck 0.2.0 (82e61b3), Xcode 26.6, iOS 26.5 simulator (iPad Pro 11-inch M5), headless (simctl boot, no video session).

What happens: on a simulator turned to landscape, a selector tap (POST /api/simulators/{udid}/action with {"action":"tap","selector":{...}}) returns ok but lands somewhere else. Raw normalized taps land correctly once the caller rotates the point itself.

Repro:

  1. Boot an iPad simulator, open any app with a button near a corner (Settings works).
  2. POST .../action {"action":"rotateLeft"}.
  3. POST .../action {"action":"tap","selector":{"label":"<that button>"}} → {"ok":true}, and the button is not pressed.

Why: the accessibility tree reports an app's frames in the turned UI, while touches are injected in the unrotated (portrait) screen.

  • REST path: tap_point_from_snapshot (api/accessibility_query.rs:207) normalizes the element centre by the root frame and sends that as the touch point, with no rotation.
  • CLI path: normalize_accessibility_point_for_display (main/tap_target.rs:164) swaps x and y when root and display orientation differ. A swap is a mirror, not a rotation, so it is wrong in both landscape directions.

The correct mapping depends on the direction. With u, v the element centre normalized in the turned UI, and the device turned q quarter turns:

q touch x touch y
1 1 - v u
2 1 - u 1 - v
3 v 1 - u

So the swap (v, u) should become (1 - v, u) or (v, 1 - u). The direction cannot be read from the tree: the root frame is identical in both landscape directions. It can be read from GET .../accessibility-point, which hit-tests in unrotated screen points and answers with UI-space frames. Hit-test a known element's centre under each candidate turn, and the one that returns that element is the device's turn. Repeated rotateLeft calls also do not accumulate: each REST call creates a new display bridge whose _deviceRotationDegrees starts at 0, so you cannot reach upside-down portrait through the REST API.

SpringBoard alerts (e.g. "Open in …?") on a turned device report their children in unrotated portrait points under a root whose frame is the turned screen, so those must not be rotated.

We work around this on the client side for now (rotate the point ourselves and send a plain normalized tap). Happy to test a fix.

Linguagem predominante
Rust
Estrelas
150
Forks
14
Merge médio
1min
PRs com merge (30d)
1

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de NativeScript/SimDeck

Todas as issues de NativeScript/SimDeck

Issues semelhantes

Mais issues de Rust

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.