No usable route to type into a web text field when the app loses keyboard focus between calls
维护者通常 1 天内回复
已经有一个关联 PR 被合并了。
- #214 来自 @iamnbutler —— 已合并
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 48/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- macos, swift
调研方向
Start by tracing the desktop_insert, desktop_set_value, and desktop_focus tool entry points and the refusal hints described in the issue; the payload names no source files or tests. Compare ways to deliver text when the target window loses keyboard focus, including the suggested foreground typing or atomic focus-and-type routes. Done means text entry works reliably in the described Safari scenario and the hint no longer recommends a refused route.
由索引模型根据 Issue 内容生成。
描述
What happened
Filling a text input on a React page (Proton VPN's WireGuard config form) in
Safari. The agent runs from a terminal, so the terminal keeps taking keyboard
focus back between tool calls. Every available route refuses, and two of the
refusals point at each other:
desktop_insert route=type
→ "Invalid target: Exact-window keyboard focus is not on the target window at
dispatch... To write into this window without focusing it, observe it and
set its editable element's accessibility value (`set-value` command,
`set_value` tool)."
desktop_set_value
→ "Set a date as ISO 8601... Use desktop_type for text."
insert says use set_value; set_value says use typing. For a text field in
a window that doesn't hold keyboard focus there is no route that succeeds.
desktop_focus does work, but focus is lost again before the next call lands
(each action needs a fresh snapshot, and the terminal reclaims focus in
between), so focus → inspect → insert is racy and failed repeatedly.
Why set_value is refused here is understandable
I'm aware AX value writes on web inputs don't fire JS input/change events, so
React would keep the old value — that's documented behaviour and refusing is
reasonable. The problem is that no alternative is offered for this situation.
Suggestions (any one would unblock)
- A foreground typing route — activate the app, type with real keys,
returnunknownwith a readback, consistent with how modifier-clicks and
drags already work. - Make
desktop_focus+ type atomic, so focus can't be lost between the
observation and the dispatch. - Let
expectondesktop_focusgate until the window actually holds
keyboard focus, so the agent can sequence reliably.
The refusal hint should also not recommend set_value for a text field, since
that path is unconditionally refused.
Environment
Desktop Tools 0.16.1, macOS 27, Safari 27, agent running from Ghostty
- 主要语言
- Swift
- 星标
- 0
- 派生
- 0
- 平均合并
- 39 分钟
- 30 天内合并 PR
- 162
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
githubnext/desktop-tools 的其他 Issue
-
Accessibility inspect returns far-offscreen elements, blowing the budget on long pages可能已有人在做 @iamnbutler 于 1 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 48/100
githubnext/desktop-tools#178 ·
维护者通常 1 天内回复
-
难度 5/5 一周以上 新手友好度 15/100
githubnext/desktop-tools#143 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 42/100
githubnext/desktop-tools#129 ·
维护者通常 1 天内回复
-
enhancement
难度 5/5 一周以上 新手友好度 8/100
githubnext/desktop-tools#5 · 11 条评论 ·
维护者通常 1 天内回复
查看 githubnext/desktop-tools 的全部 Issue
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 65/100
jellyfin/Swiftfin#2388 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 86/100
维护者通常 1 天内回复
-
area: browser area: cli
难度 2/5 1-3 小时 新手友好度 76/100
manaflow-ai/cmux#25105 · 1 条评论 ·
维护者通常 1 天内回复
-
📦 expo-modules-core contributor: external iOS needs review sdk-58 severity: medium
难度 2/5 1-3 小时 新手友好度 67/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 74/100
firebase/firebase-ios-sdk#16950 ·
维护者通常 1 天内回复