Focus and paste into WebKit composers with native desktop tools
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 48/100
- issue の種類
- バグ
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- typescript
- 領域
- tooling
調査の方向性
Start from the refusal text "Point ... is occluded for the pinned target window" to locate the accessibility hit-testing check behind desktop_click, then the focus path used by desktop_focus and element clicks targeting the AXTextArea "Message" (a ProseMirror contenteditable in a WKWebView). Use the numbered repro steps and the target PIDs/windows as your test matrix. The still-unclaimed cases are background desktop_key Cmd+V delivery and direct AX value replacement; done means a point or element click on an Electrobun web control yields verified keyboard focus without occlusion refusals, and Cmd+V either pastes or explains why it cannot.
索引モデルが issue の本文から書いたものです。
説明
While validating PR #104 (#91) in Ace Canary 0.0.11, Ace's desktop tools couldn't place focus in the Ace-dev (Electrobun/WKWebView) composer or paste into it. This issue records the tested failures only. #103 keeps the Automation-prompt and agent-guidance problem.
October 5 progress: #120 fixes verified editable element focus; #131 fixes the reproduced external WebKit button’s false coordinate occlusion. A combined signed-main check also passed activation → element focus → one dedicated literal insertion in Ace’s actual composer, with exact Unicode/multiline draft/caret, unchanged chat history, and clipboard restoration. #141 additionally fixes actual-Ace editable coordinate focus: one normalized point click gave the Message field verified keyboard focus, left the existing draft unchanged, and preserved the empty history across worker exit/reopen. Background desktop_key Cmd+V, direct AX value replacement, and the other remaining cases below are not claimed fixed.
Targets were Ace-dev builds from the #104 branch:
- PID 50933, window 4742, 1100x760 at (160,120)
- PID 56752, window 4821, same bounds
The Message composer is a ProseMirror contenteditable, exposed as an AXTextArea "Message".
Tested steps and results
- Point click, window in the background.
desktop_clickat (0.5, 0.32), on the composer, was refused: "Point (710, 363) is occluded for the pinned target window 4742: accessibility hit-testing resolved an element in a different window of the same app." A CGWindowList check at that point showed Ace-dev window 4742 as the topmost layer-0 window. Electrobun also creates invisible strip windows (2560x30 at y=0 and 1512x33 at y=1440); none of them contains that point. desktop_focus, then the same point click. Focus reportedconfirmed_change, but the click was refused again with the same occlusion error. On 56752, afterdesktop_activatereported the app active, a point click at (820, 804) on the composer was also refused as occluded.- Element click on the textarea.
desktop_clickon the AXTextArea "Message" was accepted (accessibility_action,dispatched_unverified). The next inspect still reported the AXWindow as the focused element, and the textarea hadisFocused=false. This was seen on both PIDs, the second time right afterdesktop_activate. desktop_selectwithcursor_beforeon the textarea returnedconfirmed_no_change, and focus stayed on the AXWindow.desktop_typewriting "x" returned "The accessibility value write was accepted, but its requested result could not be verified." Nothing appeared in the composer.- Background Cmd+V. Setting
AXFocused=trueon the textarea outside the tools made the next inspect report the AXTextArea as focused.desktop_keyCmd+V, delivered as backgroundwindow_targeted_events, then changed nothing while Ace-dev wasn't frontmost. A plain-text clipboard didn't paste either, so this isn't specific to images. desktop_focusrefused. Once, while the user was working in another app,desktop_focuswas refused with "Timeout while waiting for condition",operation_unsupported.
What worked: with Ace-dev frontmost after desktop_activate, setting AXFocused on the textarea and pressing the app's Edit > Paste menu item through Accessibility pasted reliably. Native desktop_key Cmd+V wasn't retried in the frontmost state because steps 2–4 never produced a focused control.
Not tested (corrected assumption)
In Finder, desktop_key Cmd+C was refused: "The snapshot has no exact focused control. Click a control, then inspect the window again." The agent didn't follow that hint (click a file row, inspect again) before switching to osascript. Whether the native tools can copy from Finder's file list is therefore unverified. #103 originally listed it as a gap; the review comment there corrected that.
Expected
desktop_clickon a visible web control in Electrobun windows isn't refused as occluded.- An element or point click on a WKWebView contenteditable gives it keyboard focus.
desktop_keydelivers Cmd+V to that control, or explains why it can't.
This issue covers capability bugs; prompt guidance is in #103. Sub-issue of #8 (slices 3–5, tracked under #5). Related: #65, #68, #73. Found during #91 / PR #104.
- 主要言語
- TypeScript
- スター
- 1
- フォーク
- 0
- PR マージ指標
- 30日以内にマージされた PR はありません
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドなし
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
githubnext/ace2 のほかの issue
-
難易度 1/5 1時間未満 初心者へのやさしさ 94/100
githubnext/ace2#487 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 90/100
githubnext/ace2#486 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 67/100
githubnext/ace2#459 ·
-
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
githubnext/ace2#437 ·
-
難易度 4/5 3〜5日 初心者へのやさしさ 35/100
githubnext/ace2#442 ·
githubnext/ace2 の issue をすべて見る
似ている issue
-
Unsampled pageload span is profiled when profileLifecycle is 'trace' (browser UI profiling)対応中かも @Jxxunnn が今日担当しました。 オープンBrowser Waiting for: Product Owner
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
getsentry/sentry-javascript#25497 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
e2e-failure ready-to-code
難易度 2/5 1〜3時間 初心者へのやさしさ 77/100
redhat-developer/rhdh-plugin-export-overlays#4438 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
area: core
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
angular/angular#71422 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
bug status: needs triage
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
facebook/docusaurus#12739 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
factory-active task-identify-harness-labels-done task-identify-issue-type-done
難易度 1/5 1〜3時間 初心者へのやさしさ 68/100
メンテナーはふだん 1 日以内に返信