🤖 tests: three loop test files fail on unchanged main under the loop-test runner
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- bun, typescript
- Domain
- testing
Research direction
Start with scripts/perf/loopTests.preload.ts and the named tests in src/node/services/tools/bash_output.test.ts, src/node/services/tools/bash_background_list.test.ts, and src/node/services/backgroundProcessManager.test.ts. Run make test-loop-baseline while other test processes are active and capture each bun test failure output; compare behavior with and without the preload and check whether concurrent runs share /tmp/mux-bashes/test-workspace. Done means the failures are understood and the tests pass reliably under concurrent load.
Written by the indexing model from the issue text.
Description
Summary
Three loop test files fail on unchanged main when the loop-test runner (make test-loop-baseline) runs them. The phase 1 baseline on 563475d0c9 (10 rounds, 6 jobs, bun 1.4.3) recorded these failed rounds:
| file | failed rounds in the baseline |
|---|---|
src/node/services/tools/bash_output.test.ts |
9 of 10 |
src/node/services/tools/bash_background_list.test.ts |
2 of 10 |
src/node/services/backgroundProcessManager.test.ts |
1 of 10 |
Other agents shared the host during the baseline: the 1-minute load average was 5.5 to 56.8. Baseline comment: https://github.com/coder/xum/issues/7255#issuecomment-6353444207
I did not find the cause. This issue records what the runs show.
What the logs show
The runner discards the output of each bun test process, so the baseline has no test names. To get them, I ran the two tool test files again on a detached checkout of 563475d0c9: 5 runs each with the runner's preload (scripts/perf/loopTests.preload.ts) and 5 runs each without it, 6 processes at a time, load average 12 to 14. Unlike the runner, this run could start two processes of the same file at the same time.
tools/bash_output.test.ts
Failed 2 of 5 runs with the preload and 2 of 5 without it, so it also fails without the preload.
| test | runs that failed | first error line |
|---|---|---|
bash_output tool > should return incremental output from process |
2 | error: expect(received).toContain(expected), Expected to contain: "line1", Received: "" |
bash_output tool > should block and wait for output when timeout_secs > 0 |
2 | error: expect(received).toContain(expected), Expected to contain: "delayed output", Received: "" |
tools/bash_background_list.test.ts
Failed 2 of 5 runs with the preload and 2 of 5 without it, so it also fails without the preload.
| test | runs that failed | first error line |
|---|---|---|
bash_background_list tool > should only list processes for the current workspace |
4 | error: Failed to spawn processes |
bash_background_list tool > should list spawned processes with correct fields |
2 | error: Failed to spawn process |
bash_background_list tool > should include display_name in listed processes |
1 | error: Failed to spawn process: Failed to persist the spawn record (meta.json): writeMeta failed with exit code 1: .../bash: line 7: /tmp/mux-bashes/test-workspace/Dev Server (2)/meta.json: No such file or directory |
The last error names the fixed path /tmp/mux-bashes/test-workspace. I did not check whether other test processes use the same path at the same time.
backgroundProcessManager.test.ts
It failed in 1 of the 10 baseline rounds. The baseline has no output of that round, so I have no test name or error line. One more run with the preload and one without it both passed.
An earlier phase 1 parity run, before the stack merged (one pass of all loop test files with the preload and one pass without it), showed the same pattern: bash_output.test.ts failed on both passes, bash_background_list.test.ts failed only on the pass with the preload, and backgroundProcessManager.test.ts passed on both.
Next step
Find why these tests fail when other test processes run beside them, and fix the tests.
Refs #7255
Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high • Cost: $53.93
- Dominant language
- TypeScript
- Stars
- 2.1k
- Forks
- 140
- Avg merge
- 3h 23m
- Merged PRs (30d)
- 953
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from coder/xum
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
keptinventory/kept#336 ·
Maintainers usually reply within 1 day
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
hemi-earn
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
vetro-protocol/vetro-monorepo#1013 ·
Maintainers usually reply within 1 day