[test-improver] Add unit tests for ConsoleTestExecutionFilterFactory
Maintainers usually reply within 1 day
Assessment
This issue has not been assessed yet.
Description
Goal and rationale
ConsoleTestExecutionFilterFactory (src/Platform/Microsoft.Testing.Platform/Requests/ConsoleTestExecutionFilterFactory.cs) is a small but real production class wired into ConsoleTestHost that decides which ITestExecutionFilter to hand to the test run: TreeNodeFilter when --treenode-filter is set, TestNodeUidListFilter when --filter-uid is set, NopFilter when neither is set, and an unreachable-guard throw when both are set simultaneously. It had zero direct unit tests — only indirectly exercised through acceptance tests that pass real CLI arguments end-to-end.
Approach
Added ConsoleTestExecutionFilterFactoryTests.cs with 4 tests mocking ICommandLineOptions (same Moq pattern used elsewhere in this test project, e.g. ArgumentArityTests.cs) to drive all four branches of the switch directly:
- Neither option set →
NopFilter. - Only
--treenode-filterset →TreeNodeFilterconstructed with the exact pattern string. - Only
--filter-uidset →TestNodeUidListFiltercontaining the exact UIDs, in order. - Both options set → throws (matching
ApplicationStateGuardTests's cross-TFM-safe pattern of asserting onUnreachableException's full type name, since the type is an internal polyfill on non-.NETCoreAppTFMs).
Coverage impact
| Metric | Before | After |
|---|---|---|
Microsoft.Testing.Platform.UnitTests total (net8.0) |
2866 | 2870 |
Microsoft.Testing.Platform.UnitTests total (net9.0) |
2867 | 2871 |
| Failed | 0 | 0 |
| Skipped | 22 (pre-existing) | 22 (pre-existing) |
Trade-offs
Low maintenance burden: pure unit tests against a sealed, 4-branch class with a stable public contract (ITestExecutionFilterFactory.TryCreateAsync()); no new test infrastructure or dependencies.
Reproducibility
./build.sh -restore
export PATH="$PWD/.dotnet:$PATH"
dotnet build test/UnitTests/Microsoft.Testing.Platform.UnitTests/Microsoft.Testing.Platform.UnitTests.csproj -c Debug -f net8.0
./artifacts/bin/Microsoft.Testing.Platform.UnitTests/Debug/net8.0/Microsoft.Testing.Platform.UnitTests
(repeat with -f net9.0 for the second TFM)
Test Status
Build succeeded (0 warnings) on net8.0 and net9.0. Full Microsoft.Testing.Platform.UnitTests suite passes on both TFMs, no regressions. dotnet format whitespace TestFx.slnx --verify-no-changes clean for the new file (only the expected, pre-existing F#-project warning noted).
Effectiveness evidence
| Behavior and inputs | Independent expectation and source | Wrong implementation and distinguishing assertion | Real path and boundary/locale coverage | Evidence |
|---|---|---|---|---|
TryCreateAsync() when only --treenode-filter "/Assembly/Namespace/Class/Method" is set |
Per the method's own switch contract (doc comment on ITestExecutionFilterFactory/method body): (true, false) branch must construct new TreeNodeFilter(treenodeFilter![0]) — independently derived from reading the production switch arms, not by calling the method itself |
Swapped (true, false) arm to return new NopFilter() instead of new TreeNodeFilter(...) — assertion Assert.IsInstanceOfType<TreeNodeFilter>(filter) rejects it |
Exercises the real ConsoleTestExecutionFilterFactory.TryCreateAsync() via mocked ICommandLineOptions.TryGetOptionArgumentList, the same entry point ConsoleTestHost uses; also covers the "neither set" (NopFilter), "only UID set" (TestNodeUidListFilter with exact ordered UIDs), and "both set" (throws) boundary branches |
Executed red/green: injected the (true, false) fault above, rebuilt (dotnet build ... -f net8.0), ran the full suite — TryCreateAsync_WhenOnlyTreeNodeFilterIsSet_ReturnsTreeNodeFilterWithGivenPattern failed exactly as expected (actual type: NopFilter), 1/2870 failed. Reverted the fault, rebuilt, reran — 0/2870 failed (net8.0) and 0/2871 failed (net9.0), 22 pre-existing skips, no regressions. |
[!NOTE]
This was originally intended as a pull request, but the git push operation failed.Original error: The process '/usr/bin/git' failed with exit code 128
Workflow Run: View run details and download bundle artifact
The bundle file is available in the
agentartifact in the workflow run linked above.
Create the pull request manually
# Download the artifact from the workflow run
gh run download '38093201907' -n agent -D '/tmp/agent-38093201907'
# Resolve the bundle source ref, fetch it into a temporary ref, then create the local branch
bundle_path='/tmp/agent-38093201907/aw-test-assist-console-test-execution-filter-factory-tests.bundle'
temp_ref='refs/bundles/create-pr-test-assist-console-test-execution-filter-factory-tests-086b4c4af5ecfd74-0013527c'
target_ref='refs/heads/test-assist/console-test-execution-filter-factory-tests-086b4c4af5ecfd74'
bundle_source_ref=$(git bundle list-heads "$bundle_path" | awk '$2 ~ /^refs\/heads\// { print $2 }')
if [ -z "$bundle_source_ref" ]; then
bundle_source_ref=$(git bundle list-heads "$bundle_path" | awk '$2 == "HEAD" { print $2 }')
fi
if [ "$(printf '%s\n' "$bundle_source_ref" | sed '/^$/d' | wc -l | tr -d ' ')" != "1" ]; then
echo "Expected exactly one bundle source ref, found: $bundle_source_ref" >&2
exit 1
fi
git fetch "$bundle_path" "${bundle_source_ref}:${temp_ref}"
git update-ref "$target_ref" "$temp_ref"
git checkout 'test-assist/console-test-execution-filter-factory-tests-086b4c4af5ecfd74'
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d "$temp_ref"
# Push the branch to the target remote
git push 'origin' 'test-assist/console-test-execution-filter-factory-tests-086b4c4af5ecfd74'
# Create the pull request
gh pr create --title '[test-improver] Add unit tests for ConsoleTestExecutionFilterFactory' --base 'main' --head 'test-assist/console-test-execution-filter-factory-tests-086b4c4af5ecfd74' --repo 'microsoft/testfx'
[!WARNING]
Firewall blocked 93 domains
The following domains were blocked by the firewall during workflow execution:
0t3vsblobprodcus362.vsblob.vsassets.io11vvsblobprodcus336.vsblob.vsassets.io1javsblobprodcus364.vsblob.vsassets.io1k9vsblobprodcus379.vsblob.vsassets.io1oavsblobprodcus350.vsblob.vsassets.io1p7vsblobprodcus324.vsblob.vsassets.io1s1vsblobprodcus386.vsblob.vsassets.io2kmvsblobprodcus39.vsblob.vsassets.io2zrvsblobprodcus388.vsblob.vsassets.io37bvsblobprodcus311.vsblob.vsassets.io37cvsblobprodcus359.vsblob.vsassets.io4m6vsblobprodcus384.vsblob.vsassets.io4myvsblobprodcus32.vsblob.vsassets.io4vyvsblobprodcus361.vsblob.vsassets.io4zjvsblobprodcus390.vsblob.vsassets.io51yvsblobprodcus36.vsblob.vsassets.io5dkvsblobprodcus355.vsblob.vsassets.io5rqvsblobprodcus385.vsblob.vsassets.io6s7vsblobprodcus313.vsblob.vsassets.io7devsblobprodcus323.vsblob.vsassets.io7k6vsblobprodcus337.vsblob.vsassets.io7tjvsblobprodcus341.vsblob.vsassets.io80zvsblobprodcus35.vsblob.vsassets.io8xbvsblobprodcus382.vsblob.vsassets.io96bvsblobprodcus338.vsblob.vsassets.ioajhvsblobprodcus363.vsblob.vsassets.iobcnvsblobprodcus378.vsblob.vsassets.iobuilds.dotnet.microsoft.comc50vsblobprodcus330.vsblob.vsassets.ioc78vsblobprodcus322.vsblob.vsassets.iocflvsblobprodcus383.vsblob.vsassets.ioci.dot.netckzvsblobprodcus347.vsblob.vsassets.iod0svsblobprodcus381.vsblob.vsassets.iodata.nuget.orgdlbvsblobprodcus316.vsblob.vsassets.ioe7bvsblobprodcus348.vsblob.vsassets.iofdpvsblobprodcus345.vsblob.vsassets.iofrdvsblobprodcus327.vsblob.vsassets.iogbsvsblobprodcus365.vsblob.vsassets.iogervsblobprodcus329.vsblob.vsassets.ioh6tvsblobprodcus346.vsblob.vsassets.ioi1qvsblobprodcus353.vsblob.vsassets.ioibzvsblobprodcus369.vsblob.vsassets.ioimzvsblobprodcus368.vsblob.vsassets.iojd4vsblobprodcus366.vsblob.vsassets.iojosvsblobprodcus372.vsblob.vsassets.iojrqvsblobprodcus343.vsblob.vsassets.iok0ivsblobprodcus356.vsblob.vsassets.iok4kvsblobprodcus344.vsblob.vsassets.iokgfvsblobprodcus314.vsblob.vsassets.iokh4vsblobprodcus325.vsblob.vsassets.iokijvsblobprodcus387.vsblob.vsassets.iokmuvsblobprodcus389.vsblob.vsassets.iokxqvsblobprodcus376.vsblob.vsassets.iol49vsblobprodcus358.vsblob.vsassets.iol7avsblobprodcus319.vsblob.vsassets.ioljcvsblobprodcus317.vsblob.vsassets.iolylvsblobprodcus31.vsblob.vsassets.iom16vsblobprodcus374.vsblob.vsassets.iom6xvsblobprodcus342.vsblob.vsassets.iom8dvsblobprodcus37.vsblob.vsassets.iomfjvsblobprodcus373.vsblob.vsassets.ion3kvsblobprodcus335.vsblob.vsassets.ionn8vsblobprodcus340.vsblob.vsassets.ioo3svsblobprodcus318.vsblob.vsassets.ioofvvsblobprodcus315.vsblob.vsassets.iop2ovsblobprodcus312.vsblob.vsassets.iopc2vsblobprodcus360.vsblob.vsassets.iopdfvsblobprodcus380.vsblob.vsassets.iope3vsblobprodcus354.vsblob.vsassets.iope4vsblobprodcus351.vsblob.vsassets.iopkgs.dev.azure.compkvvsblobprodcus321.vsblob.vsassets.iorcxvsblobprodcus328.vsblob.vsassets.ios4uvsblobprodcus326.vsblob.vsassets.ios8mvsblobprodcus38.vsblob.vsassets.iose1vsblobprodcus349.vsblob.vsassets.iosqdvsblobprodcus333.vsblob.vsassets.iost8vsblobprodcus339.vsblob.vsassets.iotphvsblobprodcus375.vsblob.vsassets.iou3hvsblobprodcus371.vsblob.vsassets.iou6ovsblobprodcus377.vsblob.vsassets.ioukkvsblobprodcus352.vsblob.vsassets.iouy6vsblobprodcus34.vsblob.vsassets.iov53vsblobprodcus320.vsblob.vsassets.iovb4vsblobprodcus33.vsblob.vsassets.iovwvvsblobprodcus334.vsblob.vsassets.iox3yvsblobprodcus370.vsblob.vsassets.ioxupvsblobprodcus332.vsblob.vsassets.ioyluvsblobprodcus367.vsblob.vsassets.ioyttvsblobprodcus357.vsblob.vsassets.ioytvvsblobprodcus310.vsblob.vsassets.ioTo allow these domains, add them to the
network.allowedlist in your workflow frontmatter:network: allowed: - defaults - "0t3vsblobprodcus362.vsblob.vsassets.io" - "11vvsblobprodcus336.vsblob.vsassets.io" - "1javsblobprodcus364.vsblob.vsassets.io" - "1k9vsblobprodcus379.vsblob.vsassets.io" - "1oavsblobprodcus350.vsblob.vsassets.io" - "1p7vsblobprodcus324.vsblob.vsassets.io" - "1s1vsblobprodcus386.vsblob.vsassets.io" - "2kmvsblobprodcus39.vsblob.vsassets.io" - "2zrvsblobprodcus388.vsblob.vsassets.io" - "37bvsblobprodcus311.vsblob.vsassets.io" - "37cvsblobprodcus359.vsblob.vsassets.io" - "4m6vsblobprodcus384.vsblob.vsassets.io" - "4myvsblobprodcus32.vsblob.vsassets.io" - "4vyvsblobprodcus361.vsblob.vsassets.io" - "4zjvsblobprodcus390.vsblob.vsassets.io" - "51yvsblobprodcus36.vsblob.vsassets.io" - "5dkvsblobprodcus355.vsblob.vsassets.io" - "5rqvsblobprodcus385.vsblob.vsassets.io" - "6s7vsblobprodcus313.vsblob.vsassets.io" - "7devsblobprodcus323.vsblob.vsassets.io" - "7k6vsblobprodcus337.vsblob.vsassets.io" - "7tjvsblobprodcus341.vsblob.vsassets.io" - "80zvsblobprodcus35.vsblob.vsassets.io" - "8xbvsblobprodcus382.vsblob.vsassets.io" - "96bvsblobprodcus338.vsblob.vsassets.io" - "ajhvsblobprodcus363.vsblob.vsassets.io" - "bcnvsblobprodcus378.vsblob.vsassets.io" - "builds.dotnet.microsoft.com" - "c50vsblobprodcus330.vsblob.vsassets.io" - "c78vsblobprodcus322.vsblob.vsassets.io" - "cflvsblobprodcus383.vsblob.vsassets.io" - "ci.dot.net" - "ckzvsblobprodcus347.vsblob.vsassets.io" - "d0svsblobprodcus381.vsblob.vsassets.io" - "data.nuget.org" - "dlbvsblobprodcus316.vsblob.vsassets.io" - "e7bvsblobprodcus348.vsblob.vsassets.io" - "fdpvsblobprodcus345.vsblob.vsassets.io" - "frdvsblobprodcus327.vsblob.vsassets.io" - "gbsvsblobprodcus365.vsblob.vsassets.io" - "gervsblobprodcus329.vsblob.vsassets.io" - "h6tvsblobprodcus346.vsblob.vsassets.io" - "i1qvsblobprodcus353.vsblob.vsassets.io" - "ibzvsblobprodcus369.vsblob.vsassets.io" - "imzvsblobprodcus368.vsblob.vsassets.io" - "jd4vsblobprodcus366.vsblob.vsassets.io" - "josvsblobprodcus372.vsblob.vsassets.io" - "jrqvsblobprodcus343.vsblob.vsassets.io" - "k0ivsblobprodcus356.vsblob.vsassets.io" - "k4kvsblobprodcus344.vsblob.vsassets.io" - "kgfvsblobprodcus314.vsblob.vsassets.io" - "kh4vsblobprodcus325.vsblob.vsassets.io" - "kijvsblobprodcus387.vsblob.vsassets.io" - "kmuvsblobprodcus389.vsblob.vsassets.io" - "kxqvsblobprodcus376.vsblob.vsassets.io" - "l49vsblobprodcus358.vsblob.vsassets.io" - "l7avsblobprodcus319.vsblob.vsassets.io" - "ljcvsblobprodcus317.vsblob.vsassets.io" - "lylvsblobprodcus31.vsblob.vsassets.io" - "m16vsblobprodcus374.vsblob.vsassets.io" - "m6xvsblobprodcus342.vsblob.vsassets.io" - "m8dvsblobprodcus37.vsblob.vsassets.io" - "mfjvsblobprodcus373.vsblob.vsassets.io" - "n3kvsblobprodcus335.vsblob.vsassets.io" - "nn8vsblobprodcus340.vsblob.vsassets.io" - "o3svsblobprodcus318.vsblob.vsassets.io" - "ofvvsblobprodcus315.vsblob.vsassets.io" - "p2ovsblobprodcus312.vsblob.vsassets.io" - "pc2vsblobprodcus360.vsblob.vsassets.io" - "pdfvsblobprodcus380.vsblob.vsassets.io" - "pe3vsblobprodcus354.vsblob.vsassets.io" - "pe4vsblobprodcus351.vsblob.vsassets.io" - "pkgs.dev.azure.com" - "pkvvsblobprodcus321.vsblob.vsassets.io" - "rcxvsblobprodcus328.vsblob.vsassets.io" - "s4uvsblobprodcus326.vsblob.vsassets.io" - "s8mvsblobprodcus38.vsblob.vsassets.io" - "se1vsblobprodcus349.vsblob.vsassets.io" - "sqdvsblobprodcus333.vsblob.vsassets.io" - "st8vsblobprodcus339.vsblob.vsassets.io" - "tphvsblobprodcus375.vsblob.vsassets.io" - "u3hvsblobprodcus371.vsblob.vsassets.io" - "u6ovsblobprodcus377.vsblob.vsassets.io" - "ukkvsblobprodcus352.vsblob.vsassets.io" - "uy6vsblobprodcus34.vsblob.vsassets.io" - "v53vsblobprodcus320.vsblob.vsassets.io" - "vb4vsblobprodcus33.vsblob.vsassets.io" - "vwvvsblobprodcus334.vsblob.vsassets.io" - "x3yvsblobprodcus370.vsblob.vsassets.io" - "xupvsblobprodcus332.vsblob.vsassets.io" - "yluvsblobprodcus367.vsblob.vsassets.io" - "yttvsblobprodcus357.vsblob.vsassets.io" - "ytvvsblobprodcus310.vsblob.vsassets.io"See Network Configuration for more information.
🤖 Automated content by GitHub Copilot. Generated by the Test Improver workflow. · copilot · auto · 414.2 AIC · ⌖ 15.7 AIC · ⊞ 22.1K · [◷]( · ◷)
Comment /test-assist to run again
Add this agentic workflow to your repo
To install this agentic workflow, run
gh aw add githubnext/agentics/workflows/test-improver.md@main
- Dominant language
- C#
- Stars
- 1k
- Forks
- 314
- Avg merge
- 8h 39m
- Merged PRs (30d)
- 446
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/testfx
-
type/automation type/tech-debt
Difficulty 3/5 1-2 days Newbie friendliness 40/100
Maintainers usually reply within 1 day
-
[file-diet] Refactor MtpServerClient.cs (581 lines) into focused partial filesPossibly taken @Evangelink claimed this today. Opentype/automation type/tech-debt
microsoft/testfx#11918 · 1 reaction · 2 assignees ·
Maintainers usually reply within 1 day
-
[quality-improver] Mutation-testing manifest omits 13 production projects, including the 623-file Microsoft.Testing.Platform core assemblyPossibly taken @Evangelink claimed this today. Opentype/automation type/tech-debt
Difficulty 5/5 Over a week Newbie friendliness 25/100
microsoft/testfx#11907 · 1 reaction · 2 assignees ·
Maintainers usually reply within 1 day
-
area/server-mode-jsonrpc
Difficulty 5/5 Over a week Newbie friendliness 8/100
Maintainers usually reply within 1 day
-
testing type/automation type/test-gap
Difficulty 2/5 1-3 hours Newbie friendliness 20/100
Maintainers usually reply within 1 day
All issues in microsoft/testfx
Similar issues
-
:watch: Not Triaged async-task-programming/subsvc dotnet-csharp/svc
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
python triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
microsoft/semantic-kernel#14590 ·
Maintainers usually reply within 3 days
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
Maintainers usually reply within 1 day
-
Needs-Triage
Difficulty 1/5 Under an hour Newbie friendliness 88/100
PowerShell/PowerShell#28194 ·
Maintainers usually reply within 1 day
-
AI Projects Client customer-reported needs-team-attention question Service Attention
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Azure/azure-sdk-for-net#63804 · 3 comments · 1 reaction ·
Maintainers usually reply within 1 day