Bot browsing on a remote machine can't be watched: the machine's screen takes the browser's place

Open
#73 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
react, typescript
Domain
frontend

Research direction

Start in src/components/bot-panel/ComputerTab.tsx and src/components/ComputerPanel.tsx, tracing how the machine screen and browser panel are selected for a bot running on a paired server. Check the browser stream client and the stated panel behavior for guidance. Done means a bot with browsing enabled can view its live browser from the server's page while retaining access to the machine screen.

Written by the indexing model from the issue text.

Description

What happened

A bot that browses on a remote machine (a paired server of your own, or My Cloud) keeps browsing, but the panel never shows it: the machine's screen takes the browser's place.

On This computer the Computer tab shows the bot's live browser — the page, its tab strip (LocalBrowserSection, src/components/bot-panel/LocalBrowser.tsx) — because the page is this computer's. On the server's page it shows that machine's screen instead, and the Browser tab is not in the tab row either.

Where the panel decides:

  • src/components/bot-panel/ComputerTab.tsx:314 — ownScreen = !local || cloudOwned || current === "vm", with local = isLocalHost(runsOnHost(...)) (:187, :304). runsOnHost reads otherServer from the desktop (src/lib/other-server.ts), so a window on a paired server comes out as kind: "vps", local is false, and the screen renders in place of the browser section.
  • src/components/ComputerPanel.tsx:275-278 — the Browser tab exists only when the bot's computer is "browser" or the stored view for that bot says browser; src/lib/computer-panel-view.ts resolves any other stored view to "computer", and the stored view is per origin, so a Browser tab chosen on This computer does not exist on the server's origin.
  • The bot's browser still mounts on whichever server runs the turn. It is gated only on features.browser, bot.browser !== false, bot.computer !== "off" and not the person's own Chrome (server/host/browser.ts:316, :404), and it is streamed at /api/bots/<id>/browser (src/lib/browser-stream-client.ts), so the server's page could render it.

So the browsing is real — its tool lines appear in the chat — but there is no live view of it.

Verified
  • DOM test against the real ComputerPanel: a bot with computer: "local" and features.browser on has no computer-panel-tabs-tab-browser and no browser panel; computer: "browser" shows both. (Temporary test, deleted after the run.)
  • computerPanelTabs({ browser: false }) → ["computer"]; resolveComputerPanelView("browser", { browser: false }) → "computer".
  • runsOnHost({ cloudHome: false, otherServer: "box-1", platform: "darwin" }) → { kind: "vps" }, which isLocalHost rejects, so ownScreen is true.
Steps to reproduce
  1. Pair a server of your own (or move a bot to My Cloud) and leave its browsing on.
  2. Set the bot's computer to that machine, not "Browser only".
  3. Open the bot's Computer tab with the window on the server: no Browser tab (unless it was chosen earlier on that origin), and the machine's screen where This computer shows the bot's live browser.
  4. Give the bot a task that browses: it works — browser tool lines land in the chat — but the page is nowhere in the panel.
Version

0.2.17

Platform

macOS

Engine

No response

Logs or screenshots
Expected: a bot's browser is watchable wherever the bot runs.
Fix direction, not a decision: render the bot's browser section above the machine's screen whenever
`bot.browser !== false` and a browser is open (the stream route is the current origin's, so it already
works from the server's page), or keep the Browser tab in the row for every bot with browsing on
instead of only `computer === "browser"` ones.
Dominant language
TypeScript
Stars
277
Forks
27
Avg merge
2h 47m
Merged PRs (30d)
1

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DuskBotAI/duskbot

All issues in DuskBotAI/duskbot

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.