Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Flaky: MtpServerProcessTests.StartAsyncTimeoutKillsProcessAndReleasesListener races between timeout and early-exit failure

Closed Beginner friendly
  • Is something wrong?
#11,631 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp
Domain
testing-qa

Research direction

Start with test/UnitTests/Microsoft.Testing.Platform.ServerMode.Client.Sources.UnitTests/MtpServerProcessTests.cs and the StartAsyncTimeoutKillsProcessAndReleasesListener test; inspect the NeverConnects.cmd asset and MtpServerProcess.Startup.cs. Run the Windows-only net8.0 test to reproduce the race. Done means the test reliably passes while retaining the process-killed, survived.txt, and port-release assertions.

Written by the indexing model from the issue text.

Description

area/mtp area/testing-platform area/timeout type/flaky-test

Summary

MtpServerProcessTests.StartAsyncTimeoutKillsProcessAndReleasesListener (Windows-only, net8.0) fails intermittently on CI with an assertion mismatch between the two possible startup-failure messages.

Observed in build 1615940 (Build Windows Debug, running the merge commit of PR #11607, whose changes are unrelated to server-mode client sources):

test/UnitTests/Microsoft.Testing.Platform.ServerMode.Client.Sources.UnitTests/MtpServerProcessTests.cs(79,1): error : StartAsyncTimeoutKillsProcessAndReleasesListener [net8.0]
Assertion failed. Expected string to contain the specified substring.

expected substring: "did not connect back within"
actual:             "The Microsoft.Testing.Platform application '...\NeverConnects.cmd' exited with code 0 before connecting back. "

The test took 2s 185ms, i.e. roughly the runtime of the ping 127.0.0.1 -n 3 in the NeverConnects.cmd asset, instead of completing near the configured 300 ms ConnectionTimeout.

Root cause

The test configures ConnectionTimeout = TimeSpan.FromMilliseconds(300) and asserts the timeout-specific message. But in MtpServerProcess.Startup.cs, CreateTimeoutOrStoppedFailureAsync first calls TryGetProcessStoppedFailureAsync(...) and only falls back to the "did not connect back within {timeout}s" message when the process has not been observed as exited:

Exception? stopped = await TryGetProcessStoppedFailureAsync(...);
return stopped ?? new MtpServerConnectionClosedException($"... did not connect back within ...");

On a loaded agent the launched .cmd can be observed as exited by the time the failure message is constructed, so the early-exit message wins and the assertion fails. The 300 ms timeout leaves essentially no margin against Windows process-start and exit-observation latency, which makes the race easy to lose.

Suggested fix directions

  • Make the asset's lifetime much longer than the configured timeout (or make the timeout meaningfully larger) so the timeout path deterministically wins the race, while keeping the existing "the process was killed and the port was released" assertions.
  • Alternatively, assert the behaviour the test actually guards (process killed, survived.txt absent, port rebindable) without pinning the exact failure-message variant, or accept either message when the process is observed as stopped.

Please avoid "fixing" this by relaxing the kill/port-release assertions — those are the meaningful part of the test.

Notes

  • Introduced with the added coverage in #11613.
  • Not caused by #11607; the failure reproduces on its merge commit only because that commit includes main.
Dominant language
C#
Stars
1k
Forks
313
Avg merge
8h 27m
Merged PRs (30d)
456

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/testfx

All issues in microsoft/testfx

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.