Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Remote AG-UI Bots can't use the browser in chat: `computer_*` calls refused as "unavailable for this headless turn"

未关闭
  • 哪里不对？
#804 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@charan-rathore 已经在做这个了。

开始于 2026年10月18日。

  • #805 来自 @charan-rathore —— 未关闭

评估

难度
3/5
预计耗时
1-2 天
新手友好度
68/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
react, typescript
领域
ai-infra-agents, backend-api-design

调研方向

Start with toolkit/src/headless-tools.ts, server/src/copilot.ts, and server/src/index.ts, then inspect the browser tools in app/src/lib/copilot/computer-tools.tsx. Trace how toolsForTurn and HeadlessToolsMiddleware receive computer_* tools for remote chat, and run server/tests/headless-tools.test.ts plus the relevant toolkit tests. Done means a remote AG-UI bot can use caller-offered browser tools while retaining headless behavior and gateway auditing; confirm whether the intended fix is caller pass-through or server-side computer tools.

由索引模型根据 Issue 内容生成。

描述

What happens

In a normal chat with a remote AG-UI Bot (the picked harness, agent-claude-sdk), every browser tool call is refused. The Bot reports:

The tool computer_navigate is unavailable for this headless turn.

The built-in General Assistant browses fine on the same deployment, same page and same request.

Steps to reproduce

Tested at 12c3a69 (main), laptop stack via scripts/start.sh, macOS, Docker (OrbStack).

  1. Register the picked harness ghcr.io/copilotkit/openbot-agent-claude-sdk:v0.2.0 (PICKED_HARNESS_* set, endpoint allowed via AGENT_ENDPOINT_ALLOWED_HOSTS).
  2. Open /channel/new?agent=picked-harness.
  3. Ask: Open news.ycombinator.com and tell me the top story.
  4. The Bot calls computer_navigate, gets the refusal above, and falls back to its own WebFetch.

Doing the same with general-assistant works and is audited as computer.action_allowed.

I only tested agent-claude-sdk. From the code it looks like it affects any remote_ag_ui / remote_mastra Bot (see below), but I haven't confirmed that with the LangGraph Bot.

Why it happens
  • server/src/copilot.ts:1322 builds clientTools from the Bot's tools that have an execute function. loadToolsForActor (server/src/index.ts:1271) always adds wake-up, memory and host-access tools, so for a remote Bot this list is never empty. That means copilot.ts:1445 always wraps the run in HeadlessToolsMiddleware.
  • HeadlessToolsMiddleware (toolkit/src/headless-tools.ts:164) treats every pending call whose name starts with computer_ as its own, whether or not it holds that tool.
  • In a chat turn the middleware isn't given any computer tools. They're only passed through toolsForTurn on the routine, approval and hop paths in index.ts. So computer_navigate isn't in its map, and line 184 throws "unavailable for this headless turn".
  • The computer_* tools the open page offered in input.tools (app/src/lib/copilot/computer-tools.tsx) never get to run, even though a browser is open. The middleware's own doc comment says it's for when "the caller has no browser".
Workaround I'm running locally

Leave computer_* calls to the caller when the caller offered that tool:

       const executed = new Set<string>();
+      const offeredByCaller = new Set(
+        originalInput.tools
+          .map((tool) => tool.name)
+          .filter((name) => !tools.has(name)),
+      );
 ...
               tools.has(call.function.name) ||
-              call.function.name.startsWith("computer_"),
+              (call.function.name.startsWith("computer_") &&
+                !offeredByCaller.has(call.function.name)),

With this change the picked harness browses, and the action still goes through the gateway: it shows up in audit_events as computer.action_allowed with bot: picked-harness and the policy decision attached. toolkit tests pass (59/59), and server/tests/headless-tools.test.ts passes.

This may not be the direction you want, though. If the intent is for remote Bots to run browser tools server-side with no page attached (as the comments in agent-langgraph/src/index.ts suggest), the fix would be to pass createHeadlessComputerTools(...) into the remote chat path the way toolsForTurn does for routines. Either way, refusing the call outright while a page is offering the tool seems unintended.

主要语言
TypeScript
星标
6.6k
派生
914
平均合并
17 小时 52 分钟
30 天内合并 PR
155

环境准备

  • 提供 Dockerfile 或 Docker Compose 文件
  • 有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

CopilotKit/OpenBot 的其他 Issue

查看 CopilotKit/OpenBot 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。