[Bug]: run-test-mcp-server over stdio does not stop a paused test on stdin close, so worker fixture teardown never runs

Open
#44,260 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@microsoft-playwright-automation is already working on this.

Since Nov 20, 2026.

  • #44263 by @microsoft-playwright-automation — open

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
node.js, playwright, typescript
Domain
backend, testing

Research direction

Start with packages/playwright/src/mcp/test/testBackend.ts and trace TestServerBackend.dispose() alongside the watchdog path in packages/playwright-core/src/tools/mcp/watchdog.ts and packages/utils/processLauncher.ts. The issue includes a proposed registration change and a regression test, tests/mcp/test-mcp-shutdown.spec.ts, covering paused tests and worker fixture teardown. Done means closing stdio stops the paused test, runs fixture teardown, and exits cleanly; an open linked pull request indicates work is already underway.

Written by the indexing model from the issue text.

Description

agent-triage-done
Version

1.65.1 (latest on npm at the time of writing; the affected source is identical on main at 2f1592a70b)

Steps to reproduce

Self-contained, one dependency (@playwright/[email protected]), no browsers beyond the default Chromium. The test has a worker-scoped fixture that owns a long-lived child process (stand-in for a launched app or browser). The client is a small raw JSON-RPC stdio client that starts the server, pauses the test with planner_setup_page, closes the server's stdin like an MCP client does on exit, and then checks whether the fixture was torn down.

mkdir repro && cd repro
# create the 4 files below
npm install
npx playwright install chromium
node client.mjs

package.json

{
  "name": "pw-mcp-shutdown-repro",
  "private": true,
  "devDependencies": { "@playwright/test": "1.65.1" }
}

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({ testDir: '.' });

seed.spec.ts

import { test as base, expect } from '@playwright/test';
import { spawn } from 'child_process';
import fs from 'fs';
import path from 'path';
import os from 'os';

// Stand-in for a resource owned by a worker fixture (a launched app, a browser, a server, ...).
const dir = process.env.REPRO_DIR || path.join(os.tmpdir(), 'pw-mcp-shutdown-repro-out');

const test = base.extend<{}, { resource: void }>({
  resource: [async ({}, use) => {
    fs.mkdirSync(dir, { recursive: true });
    const child = spawn(process.execPath, ['-e', 'setInterval(()=>{},1000)'], { stdio: 'ignore', detached: true });
    fs.writeFileSync(path.join(dir, 'child.pid'), String(child.pid));
    await use();
    // Teardown: only runs if the worker is stopped gracefully.
    child.kill();
    fs.writeFileSync(path.join(dir, 'teardown.marker'), 'torn down');
  }, { scope: 'worker', auto: true }],
});

test('seed', async ({ page }) => {
  await page.setContent('<button>Submit</button>');
  await expect(page.getByRole('button', { name: 'Submit' })).toBeVisible();
});

client.mjs

// Minimal MCP stdio client (raw JSON-RPC, newline-delimited). Usage: node client.mjs
import { spawn } from 'child_process';
import fs from 'fs';
import path from 'path';
import os from 'os';
import { createRequire } from 'module';

const dir = path.join(os.tmpdir(), 'pw-mcp-shutdown-repro-out');
process.env.REPRO_DIR = dir;
fs.rmSync(dir, { recursive: true, force: true });

const cli = createRequire(import.meta.url).resolve('@playwright/test/cli'); // same entry point as `npx playwright`
const server = spawn(process.execPath, [cli, 'run-test-mcp-server'], { cwd: import.meta.dirname, stdio: ['pipe', 'pipe', 'inherit'] });
const exited = new Promise(resolve => server.on('exit', (code, signal) => resolve({ code, signal })));
const pending = new Map();
let buffer = '';
server.stdout.on('data', chunk => {
  buffer += chunk;
  for (let i; (i = buffer.indexOf('\n')) >= 0;) {
    const line = buffer.slice(0, i);
    buffer = buffer.slice(i + 1);
    if (!line.trim())
      continue;
    const msg = JSON.parse(line);
    pending.get(msg.id)?.(msg);
  }
});
let nextId = 1;
const send = msg => server.stdin.write(JSON.stringify(msg) + '\n');
const request = (method, params) => new Promise(resolve => {
  const id = nextId++;
  pending.set(id, resolve);
  send({ jsonrpc: '2.0', id, method, params });
});
const alive = pid => { try { process.kill(pid, 0); return true; } catch { return false; } };
const sleep = ms => new Promise(r => setTimeout(r, ms));
const log = (...a) => console.log(new Date().toISOString(), ...a);

await request('initialize', { protocolVersion: '2025-06-18', capabilities: {}, clientInfo: { name: 'repro', version: '0' } });
send({ jsonrpc: '2.0', method: 'notifications/initialized' });
const res = await request('tools/call', { name: 'planner_setup_page', arguments: { seedFile: 'seed.spec.ts' } });
const text = res.result?.content?.map(c => c.text).join('\n') ?? JSON.stringify(res);
log('tool result contains "Paused":', /Paused/.test(text));
if (!/Paused/.test(text)) {
  log(text);
  server.kill();
  process.exit(2);
}

const childPid = Number(fs.readFileSync(path.join(dir, 'child.pid'), 'utf8'));
log('fixture child pid:', childPid, 'alive:', alive(childPid));

log('closing server stdin');
server.stdin.end();
const result = await Promise.race([exited, sleep(30000).then(() => 'timeout')]);
log('server exit:', JSON.stringify(result));

const markerAtExit = fs.existsSync(path.join(dir, 'teardown.marker'));
await sleep(5000); // grace period in case the orphaned worker finishes on its own
const marker = fs.existsSync(path.join(dir, 'teardown.marker'));
const childAlive = alive(childPid);
log('teardown marker at server exit:', markerAtExit, '; 5 s later:', marker);
log('fixture child alive after server exit:', childAlive);
if (childAlive) {
  process.kill(childPid);
  log('(killed leftover child by pid', childPid + ')');
}
log(marker && !childAlive ? 'RESULT: OK, worker fixtures were torn down' : 'RESULT: BUG, worker fixture teardown did not run');
process.exit(marker && !childAlive ? 0 : 1);

The client kills the leftover child itself, by PID, so the repro cleans up after itself.

Expected behavior

Closing the server's stdin shuts the server down the same way closing the MCP connection does: the paused test is stopped, the worker runs its fixture teardown (worker-scoped fixtures included), and then the process exits. The marker file exists and the fixture's child process is gone.

Actual behavior

The server exits within about 20 ms. Worker fixture teardown does not run (no marker file), and whatever the worker fixtures own is left running. From the code path below, the paused test is never stopped through TestServerBackend.dispose(), so the worker is never asked to stop.

tool result contains "Paused": true
fixture child pid: 40044 alive: true
closing server stdin
server exit: {"code":0,"signal":null}
teardown marker at server exit: false ; 5 s later: false
fixture child alive after server exit: true
(killed leftover child by pid 40044)
RESULT: BUG, worker fixture teardown did not run

Same result on a second run. With the proposed fix below applied to node_modules/playwright/lib/mcp/test/testBackend.js:

tool result contains "Paused": true
fixture child pid: 32900 alive: true
closing server stdin
server exit: {"code":0,"signal":null}        <- about 210 ms later, after teardown
teardown marker at server exit: true ; 5 s later: true
fixture child alive after server exit: false
RESULT: OK, worker fixtures were torn down

The same happens with test_debug (paused on error) and generator_setup_page, because they all leave the test paused in a worker. In my real-world case the worker fixture owns an Electron app. The app tree outlives the server and the worker, and MCP clients that wait for the whole process tree to finish never exit.

Additional context

Root cause. Source references are to main (2f1592a70b); the same files are byte-identical at the v1.65.1 tag.

  1. packages/playwright/src/program.ts (line 148) calls tools.setupExitWatchdog() for run-test-mcp-server.
  2. packages/playwright-core/src/tools/utils/mcp/server.ts (lines 191 to 193) deliberately does not close the stdio transport on stdin end, with the comment "The exit watchdog closes the browsers on stdin close". So over stdio the server's close event never fires.
  3. The only thing that reacts to stdin close is packages/playwright-core/src/tools/mcp/watchdog.ts (lines 28 to 35): await gracefullyCloseAll() and then processExit(0). gracefullyCloseAll (packages/utils/processLauncher.ts, lines 55 to 59) only runs functions in gracefullyCloseSet.
  4. TestServerBackend.dispose() (packages/playwright/src/mcp/test/testBackend.ts, lines 72 to 74), which calls TestContext.close(), stopTests() and the worker's __stop__, is only invoked from the server close event (server.ts, lines 82 to 83). It is never registered with gracefullyCloseSet, so it never runs on this path.
  5. The process then exits while a test is paused (planner_setup_page, test_debug, generator_setup_page). The worker is left without its parent and does not tear down its fixtures (observed: no marker file).

The browser tools do not have this problem because browsers register themselves in gracefullyCloseSet; the test backend does not.

Proposed fix. Register the backend's dispose() with the watchdog's existing mechanism. The watchdog's existing 15 s fallback processExit(0) bounds the wait. In my runs teardown took about 200 ms in the repro and about 3 s with a real Electron app.

--- a/packages/playwright/src/mcp/test/testBackend.ts
+++ b/packages/playwright/src/mcp/test/testBackend.ts
@@ -18,6 +18,7 @@ import EventEmitter from 'events';
 
 import * as zod from 'zod';
 import { tools } from 'playwright-core/lib/coreBundle';
+import { gracefullyCloseSet } from '@utils/processLauncher';
 
 import { TestContext } from './testContext';
 import * as testTools from './testTools.js';
@@ -47,6 +48,7 @@ export class TestServerBackend extends EventEmitter implements tools.ServerBacke
   private _options: { muteConsole?: boolean, headless?: boolean };
   private _context: TestContext | undefined;
   private _configPath: string | undefined;
+  private _gracefullyClose = () => this.dispose();
 
   constructor(configPath: string | undefined, options?: { muteConsole?: boolean, headless?: boolean }) {
     super();
@@ -56,6 +58,9 @@ export class TestServerBackend extends EventEmitter implements tools.ServerBacke
 
   async initialize(clientInfo: tools.ClientInfo): Promise<void> {
     this._context = new TestContext(clientInfo, this._configPath, this._options);
+    // Over stdio the server is not closed on stdin end, only the exit watchdog runs. Register with it so a
+    // paused test is stopped and its fixtures are torn down before the process exits.
+    gracefullyCloseSet.add(this._gracefullyClose);
   }
 
   async callTool(name: string, args: tools.CallToolRequest['params']['arguments'], signal: AbortSignal): Promise<tools.CallToolResult> {
@@ -70,6 +75,7 @@ export class TestServerBackend extends EventEmitter implements tools.ServerBacke
   }
 
   async dispose() {
+    gracefullyCloseSet.delete(this._gracefullyClose);
     await this._context?.close();
   }
 }

Regression test. I have a test for tests/mcp ready (test-mcp-shutdown.spec.ts). It starts the test MCP server, pauses a test with planner_setup_page in one case and test_debug on a failing test in the other, closes the client, and polls for a marker file written by a worker-scoped fixture's teardown. Both cases fail on main (marker never written, 15 s poll timeout) and pass with the fix. I can open a PR if that is welcome.

Notes on the repro.

  • The fixture child is detached: true on purpose. On Windows I observed that a non-detached child dies together with the worker (probably Node's job object with kill-on-close, I did not verify that), which hides the leak for this stand-in. Real resources that outlive their worker, such as an Electron app, do not die with it. On POSIX a plain child would also be orphaned. The teardown marker is the primary signal and does not depend on this.
  • I ran the repro on Windows only. The code path described above is not platform specific, but I have not run the repro on Linux or macOS.
  • client.mjs runs playwright/cli through the Playwright package entry point instead of npx playwright run-test-mcp-server, to avoid the extra npx/shell process in the tree. The server command line is the same.
  • It surfaced with Kiro CLI, which waits for the whole process tree; any MCP client that closes stdin while a test is paused is affected.
Environment
System:
  OS: Windows 11 10.0.26100
  CPU: (28) x64
Binaries:
  Node: 22.22.2
  npm: 10.9.7
npmPackages:
  @playwright/test: 1.65.1 => 1.65.1
Dominant language
TypeScript
Stars
100k
Forks
6.9k
Avg merge
16h 8m
Merged PRs (30d)
281

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/playwright

All issues in microsoft/playwright

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.