Studio automation: optional visible mode for agent-driven sessions (design)

未关闭
#389 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
25/100
Issue 类型
功能
描述清晰度
需要澄清
活跃度
冷清
技术栈
dart, flutter
领域
testing-qa

调研方向

Start by reading the existing Studio automation endpoint and MCP tool implementations, then trace the current boundaries for loopback access, launch tokens, path confinement, and local device targets. The issue leaves visibility control, banner behavior, display handling, replay recording, and supported in-app actions undecided, so a first step is to resolve those design questions; implementation is done only after the scope and boundaries are agreed.

由索引模型根据 Issue 内容生成。

描述

area/studio enhancement

Split out of #331 (its headed-mode addendum). This issue is for design only: nothing here is decided and nothing is assigned.

Context

The automation endpoint (PROBE_STUDIO_AUTOMATION=1, off by default) lets a script or an MCP client drive a running Studio without a window, cursor or focus. The browser suite covers the UI in headed or headless Chromium. Neither shows an agent-driven session in the real Studio window.

Idea under discussion

  • An opt-in visible mode in which the real Studio window shows what an automation client is doing: the open file, the selected device, the live step highlight and the results update as the client works.
  • A visible flag on the Studio MCP tools (studio_open_workspace, studio_run_file and the others), with the default to be decided.
  • A small in-app banner while automation is driving the session, so a person at the machine can see it.
  • Actions would go through the app (backend methods and in-page actions), not operating-system mouse or keyboard events, so the person's cursor and focus are not taken.

Open questions

  1. How visible mode fits the existing boundaries: loopback only, the per-launch token, the path confinement, and the local-target policy for device connections. Visible mode must not add a way around any of them.
  2. Who decides visibility: only the Studio process (for example through an environment variable), or can an automation client request it per call?
  3. The banner: its text, when it appears and clears, and whether the person can dismiss it.
  4. What happens with no second display attached, or with no display at all.
  5. Whether a visible session is also recorded for replay, and where that data would live.
  6. Which Studio actions can run inside the app without operating-system input (for example connect, run, cancel), and which would need a separate design.

Not in scope

Changing the current automation or MCP behaviour, the local-target policy, the token or the confinement rules. Any such change would need its own review.

主要语言
Go
星标
6
派生
3
平均合并
6 小时 28 分钟
30 天内合并 PR
362

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

AlphaWaveSystems/flutter-probe 的其他 Issue

查看 AlphaWaveSystems/flutter-probe 的全部 Issue

相似的 Issue

更多 Go Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。