dotnet/runtime

experiment with running tests in parallel within some collections

Open

#93,273 建立於 2023年10月10日

在 GitHub 查看
 (7 留言) (0 反應) (0 負責人)C# (5,445 fork)batch import
area-Metadiscussionhelp wantedtest-enhancement

倉庫指標

Star
 (17,886 star)
PR 合併指標
 (平均合併 12天 11小時) (30 天內合併 661 個 PR)

描述

Our unit tests in each test assembly are divided across various classes, usually (but certainly not always) chosen to group related tests together rather than for any regard for parallelism.

Xunit's default is that each class is its own test collection (we generally do not change this) and default parallelism is between test collections, but not within them (we do limit this in some cases example)

In some (many?) cases tests within a class would be happy to run in parallel with each other. Note that classes deriving from FileCleanupTestBase ought not to be unsafe for that reason alone, if it is careful to put each test in their own folder https://github.com/dotnet/runtime/blob/e555a0e16635c6ff2d88366f19d0781ac1fef0ee/src/libraries/Common/tests/TestUtilities/System/IO/FileCleanupTestBase.cs#L37

Suggestion: determine whether there are large categories of tests in some test assembly or other that we can identify could safely run concurrently with others in the same class, and experiment with enabling that to see whether it materially speeds up tests.

Ways to enable that

  1. obviously, breaking into their own classes.
  2. an extension point eg https://www.meziantou.net/parallelize-test-cases-execution-in-xunit.htm

貢獻者指南