Finish native pointer delivery and validation
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 15/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- swift
調査の方向性
Start from the merged click, scroll and drag work in PRs #82, #70 and #85, and from the open coverage list in this issue. The remaining work is verification, not new code: worker-crash and in-flight Stop cases, self-Ace scrolling, multiple displays, and a second machine. Done means each open item has a recorded real run or an explicit note that it stays unverified, tracked in githubnext/ace2#8.
索引モデルが issue の本文から書いたものです。
説明
Status (Oct 6): clicks, scrolling and atomic drag are merged (#82, githubnext/ace2#70, githubnext/ace2#85), and githubnext/ace2#156 now gives pixels-only snapshots exact-window coordinate authority for point clicks, scrolls and drags. Remaining evidence gaps: one full model workflow using githubnext/ace2#156 pixel snapshots that includes an effective drag (an earlier real pi/Anthropic run with Accessibility snapshots completed click, scroll and drop; see below), worker crash and in-flight Stop during input, self-Ace scrolling, multiple displays/negative origins/hotplug, and a second machine. Moved-window stale-click classification is githubnext/ace2#155.
Continue githubnext/ace2#8's complete-input slice with native pointer operations through Ace's embedded bridge and existing pi tools.
- Extend observed-control clicks with useful click variants and exact-window screenshot-coordinate clicks.
- Add targeted scrolling to an observed control or screenshot point, with explicit direction and bounded native scroll amount.
- Add an atomic drag with one complete press/move/release sequence, bounded duration, and cleanup on cancellation. Do not expose unowned cross-call held input.
- Resolve screenshot points through Peekaboo's stored capture coordinate authority. Account for Ace image resizing, Retina scale, multiple/negative-origin displays, and moved windows. Reject stale target identities/bounds rather than silently remapping against a new window.
- Reuse exact process/window receipts, single-use snapshots, native coordination, durable completed/refused/unknown outcomes, and the channel owner's desktop-tools setting (see githubnext/ace2#8). Make supported background versus foreground operations explicit; no silent global-input fallback.
Done when real native and pi/model workflows can click by visual coordinates, scroll to initially offscreen content, and complete a real drag-and-drop, with resulting state verified and interruption cleanup/no-replay validated. Include the hosted workspace path and clearly track unverified multi-display or second-machine behavior in githubnext/ace2#8.
Click variants and normalized screenshot-coordinate clicks landed in PR githubnext/ace2#82, merge e5618c1. Real native callbacks covered single/double/right/middle/triple behavior and resized screenshot coordinates; consumed snapshots were refused. The real pi/Anthropic check on exact PR82 head 5f98d94 performed one coordinate click with exactly one intended callback and unchanged unrelated state. Exact tool text/images and usage survived reopening the same pi store without repeated input.
Targeted scrolling landed in PR githubnext/ace2#70, merge 44312423ce66443ab38cd1ba6666bec32904e068, from reviewed head f1682d028a9f810c93c103286949c1ad69c8ef13. It accepts an observed element or normalized screenshot point, four directions, and 1–20 native units. Final types, formatting/lint, Swift parsing, diff checks, and independent source review passed. The extracted scroll implementation is unchanged from signed candidate d9b0c869; the extraction/base integration was not rebuilt into another signed app.
On that signed candidate, five real AppKit scrolls passed: element down, point down revealing previously offscreen content, element up, point right, and element left. A subsequent real pi/Anthropic run performed inspect → element up1 → inspect, moving the actual vertical origin from 2063 to 1834 while horizontal origin 68 and unrelated pointer state stayed unchanged. Exact native receipts and tool text/images survived completed-store reopen with unchanged usage and no repeated input. This does not claim worker-crash or hosted behavior.
The earlier delivered-but-unknown scroll was traced to omitted snapshot focus in the request, despite that focus being retained in the signed native receipt. The landed fix carries the original focus into scroll's expected-window target without relaxing receipt validation or coordinate authority. No uncertain input was replayed.
Atomic dragging landed in PR githubnext/ace2#85, merge 68b4a3f641858526d80de352de84260a7f3e8f5a, from reviewed head bf930584700306ec379036c320dd1c61be4f6db9. The native code is unchanged from signed candidate 2531311; final changes were tool/documentation guidance, with fresh type/lint checks. Native signatures, compilation, and independent source review passed.
Real focused left/right drags each produced one press, twenty moves and one release, moved the tile and completed the drop. Invalid endpoints/stationary gestures/durations refused without input. A new, verified window move made its fresh prior observation stale, and a drag using it refused with the receiver unchanged. Cancelling a five-second gesture only after actual mouse-down returned unknown, delivered exactly one release, and drained every native call without changing scroll position.
A real pi/Anthropic run in local Workers/Durable Objects then performed inspect → Activate click → one scroll → drag → inspect through the actual workspace adapter and signed native bridge. The fixture recorded one click, scroll from 0 to 229, and one completed tile drop. Native text and images crossed the workspace unchanged. Restarting the same persistence with the workspace offline replayed all five exact results/images with unchanged receiver state and provider usage. This is completed-run replay, not a crash during input.
The earlier inactive receiver accepted dispatch but recorded no input or movement. Separate explicit activation/focus before the successful cases establishes the supported workflow; delivery acceptance alone never proves a drop. One Retina display (native scale 2) and Ace's resized 1600×1053 image were exercised.
Remaining coverage stays open here and in githubnext/ace2#8: worker-crash/in-flight Stop cases, self-Ace scrolling, multiple displays/negative origins/hotplug, and another physical machine. Pixels-only observations could not authorize pointer actions without canonical capture authority; githubnext/ace2#156 resolved that. No broader hardware or inactive-view delivery claim is made.
- 主要言語
- Swift
- スター
- 0
- フォーク
- 0
- 平均マージ
- 19分
- マージ済み PR(30日)
- 107
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
githubnext/desktop-tools のほかの issue
-
ace documentation
難易度 2/5 1〜3時間 初心者へのやさしさ 79/100
githubnext/desktop-tools#104 ·
メンテナーはふだん 1 日以内に返信
-
Linux support (meta)オープン
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
githubnext/desktop-tools#136 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
Select text in a PDFオープン
難易度 3/5 1〜2日 初心者へのやさしさ 52/100
githubnext/desktop-tools#130 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 42/100
githubnext/desktop-tools#129 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
githubnext/desktop-tools#117 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
githubnext/desktop-tools の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
Emanuele-web04/remodex#203 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 92/100
メンテナーはふだん 1 日以内に返信
-
test(BrainBar): inject synthetic observabilityCadence in BackupsHealth/one-page fixtures (DCI-1)オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
EtanHey/brainlayer#1367 ·
メンテナーはふだん 1 日以内に返信
-
cmux-next: new workspaces show no name instead of workspace-N対応中かも このイシューにリンクされたプルリクエストがオープン中、またはマージ済みです。 オープンarea: workspaces
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
manaflow-ai/cmux#21357 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
メンテナーはふだん 4 日以内に返信