Focus and paste into WebKit composers with native desktop tools
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
This is a macOS accessibility and input-delivery bug, and the payload names no repository files. Start with the desktop_click, desktop_focus and desktop_key paths that the failing steps exercise, and reproduce step 1 against an Electrobun WKWebView window. Done means a point click on the Message composer is not refused as occluded, and an element or point click gives the contenteditable keyboard focus. Background Cmd+V is still open and is not claimed fixed.
由索引模型根据 Issue 内容生成。
描述
While validating PR githubnext/ace2#104 (#91) in Ace Canary 0.0.11, Ace's desktop tools couldn't place focus in the Ace-dev (Electrobun/WKWebView) composer or paste into it. This issue records the tested failures only. githubnext/ace2#103 keeps the Automation-prompt and agent-guidance problem.
October 5 progress: githubnext/ace2#120 fixes verified editable element focus; githubnext/ace2#131 fixes the reproduced external WebKit button’s false coordinate occlusion. A combined signed-main check also passed activation → element focus → one dedicated literal insertion in Ace’s actual composer, with exact Unicode/multiline draft/caret, unchanged chat history, and clipboard restoration. githubnext/ace2#141 additionally fixes actual-Ace editable coordinate focus: one normalized point click gave the Message field verified keyboard focus, left the existing draft unchanged, and preserved the empty history across worker exit/reopen. Background desktop_key Cmd+V, direct AX value replacement, and the other remaining cases below are not claimed fixed.
Targets were Ace-dev builds from the githubnext/ace2#104 branch:
- PID 50933, window 4742, 1100x760 at (160,120)
- PID 56752, window 4821, same bounds
The Message composer is a ProseMirror contenteditable, exposed as an AXTextArea "Message".
Tested steps and results
- Point click, window in the background.
desktop_clickat (0.5, 0.32), on the composer, was refused: "Point (710, 363) is occluded for the pinned target window 4742: accessibility hit-testing resolved an element in a different window of the same app." A CGWindowList check at that point showed Ace-dev window 4742 as the topmost layer-0 window. Electrobun also creates invisible strip windows (2560x30 at y=0 and 1512x33 at y=1440); none of them contains that point. desktop_focus, then the same point click. Focus reportedconfirmed_change, but the click was refused again with the same occlusion error. On 56752, afterdesktop_activatereported the app active, a point click at (820, 804) on the composer was also refused as occluded.- Element click on the textarea.
desktop_clickon the AXTextArea "Message" was accepted (accessibility_action,dispatched_unverified). The next inspect still reported the AXWindow as the focused element, and the textarea hadisFocused=false. This was seen on both PIDs, the second time right afterdesktop_activate. desktop_selectwithcursor_beforeon the textarea returnedconfirmed_no_change, and focus stayed on the AXWindow.desktop_typewriting "x" returned "The accessibility value write was accepted, but its requested result could not be verified." Nothing appeared in the composer.- Background Cmd+V. Setting
AXFocused=trueon the textarea outside the tools made the next inspect report the AXTextArea as focused.desktop_keyCmd+V, delivered as backgroundwindow_targeted_events, then changed nothing while Ace-dev wasn't frontmost. A plain-text clipboard didn't paste either, so this isn't specific to images. desktop_focusrefused. Once, while the user was working in another app,desktop_focuswas refused with "Timeout while waiting for condition",operation_unsupported.
What worked: with Ace-dev frontmost after desktop_activate, setting AXFocused on the textarea and pressing the app's Edit > Paste menu item through Accessibility pasted reliably. Native desktop_key Cmd+V wasn't retried in the frontmost state because steps 2–4 never produced a focused control.
Not tested (corrected assumption)
In Finder, desktop_key Cmd+C was refused: "The snapshot has no exact focused control. Click a control, then inspect the window again." The agent didn't follow that hint (click a file row, inspect again) before switching to osascript. Whether the native tools can copy from Finder's file list is therefore unverified. githubnext/ace2#103 originally listed it as a gap; the review comment there corrected that.
Expected
desktop_clickon a visible web control in Electrobun windows isn't refused as occluded.- An element or point click on a WKWebView contenteditable gives it keyboard focus.
desktop_keydelivers Cmd+V to that control, or explains why it can't.
This issue covers capability bugs; prompt guidance is in githubnext/ace2#103. Sub-issue of githubnext/ace2#8 (slices 3–5, tracked under githubnext/ace2#5). Related: githubnext/ace2#65, githubnext/desktop-tools#2, githubnext/desktop-tools#4. Found during githubnext/ace2#91 / PR githubnext/ace2#104.
- 主要语言
- Swift
- 星标
- 0
- 派生
- 0
- 平均合并
- 15 分钟
- 30 天内合并 PR
- 94
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
githubnext/desktop-tools 的其他 Issue
-
ace documentation
难度 2/5 1-3 小时 新手友好度 79/100
githubnext/desktop-tools#104 ·
维护者通常 1 天内回复
-
documentation
难度 4/5 3-5 天 新手友好度 35/100
githubnext/desktop-tools#75 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 55/100
githubnext/desktop-tools#42 · 29 条评论 ·
维护者通常 1 天内回复
-
enhancement
难度 5/5 一周以上 新手友好度 15/100
githubnext/desktop-tools#27 · 3 条评论 ·
维护者通常 1 天内回复
-
question
难度 5/5 一周以上 新手友好度 8/100
githubnext/desktop-tools#24 ·
维护者通常 1 天内回复
查看 githubnext/desktop-tools 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 72/100
-
type: cleanup
难度 2/5 1-3 小时 新手友好度 78/100
googleapis/google-cloud-swift#1477 ·
维护者通常 1 天内回复
-
product / auth product / console product / databases product / messaging product / self-hosted product / sites product / storage
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
P1 status: first-line-handled status: needs-triage
难度 2/5 1-3 小时 新手友好度 62/100
a2ui-project/a2ui#3282 · 1 条评论 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
element-hq/element-x-ios#6357 ·
维护者通常 1 天内回复