[quality-improver] Solution file project coverage gap — TestIdProject.FullyQualifiedTestStrategy orphaned from TestFx.slnx/MSTest.slnf

Start by checking the listed project path against TestFx.slnx and MSTest.slnf, then read ParameterizedTestTests.cs and the sibling TestIdProject.DefaultStrategy setup to understand existing coverage. The issue proposes wiring the project into both solution files and adding acceptance coverage for its fully qualified test IDs; done means the project is built and its behavior is exercised by a test. The report also suggests a project rename and a CI guard, which need broader repo investigation.

🎯 Repository Quality Improvement Report — Solution/Test-Asset Discoverability

Analysis Date: 2026-10-19
Focus Area: Solution File Project Coverage (orphaned test assets excluded from TestFx.slnx/MSTest.slnf)
Strategy Type: Custom

Executive Summary

While auditing the full set of .csproj files against TestFx.slnx (and its filtered views MSTest.slnf, Microsoft.Testing.Platform.slnf, NonWindowsTests.slnf), I found that test/IntegrationTests/TestAssets/TestIdProject.FullyQualifiedTestStrategy/TestIdProject.FullyQualifiedStrategy.csproj is never referenced by any solution file, solution filter, CI workflow, or acceptance/integration test. Its sibling project in the same feature pair, TestIdProject.DefaultStrategy, is wired into both TestFx.slnx and MSTest.slnf and its TestIdCases.cs is exercised (indirectly, via the shared file link) only through that sibling.

This is distinct from two previously filed/closed issues (#9715, #9198) that also mention TestIdProject.FullyQualifiedTestStrategy — but only as one line item in a list of 25+ projects hard-coding net462/IsPackable=false. Neither of those reports identified that this specific project is completely unreachable from any build or test entry point, which is a different and more severe problem: the FULLY_QUALIFIED_ID compilation symbol and the ITestDataSource/DynamicData fully-qualified test-ID code paths it was presumably created to validate (see the TestIdGenerationStrategy comments in ParameterizedTestTests.cs) are not covered by any automated run today.

Current State Assessment

Metrics Collected:

Findings

Strengths

• TestFx.slnx and MSTest.slnf are otherwise well-maintained — the vast majority of product and test-asset projects are correctly wired in.
• The sibling TestIdProject.DefaultStrategy project (added in the same commit, 780c775) is correctly included and actively used to validate DataRow/DynamicData/ITestDataSource test-ID generation (see comments referencing DynamicDataAttribute.TestIdGenerationStrategy in ParameterizedTestTests.cs).

Areas for Improvement

• ⚠️ MEDIUM: test/IntegrationTests/TestAssets/TestIdProject.FullyQualifiedTestStrategy/TestIdProject.FullyQualifiedStrategy.csproj defines the FULLY_QUALIFIED_ID constant and links in the shared TestIdCases.cs, clearly intended as the "fully qualified" counterpart used to validate DynamicDataAttribute.TestIdGenerationStrategy.FullyQualified — but it is never built, never restored, and never executed by any CI pipeline, local solution, or acceptance test. It currently provides zero test-value: a regression in the fully-qualified ID strategy for DataRow/DynamicData/ITestDataSource tests would not be caught by this asset.
• ⚠️ LOW: The project file itself has a naming mismatch between its folder (TestIdProject.FullyQualifiedTestStrategy) and its file name (TestIdProject.FullyQualifiedStrategy.csproj, missing Test), which likely contributed to it being overlooked when the sibling was added to the solution.

🤖 Suggested Improvement Tasks

Task 1: Add TestIdProject.FullyQualifiedTestStrategy to TestFx.slnx and MSTest.slnf

Priority: High
Estimated Effort: Small

Add a <Project Path="test/IntegrationTests/TestAssets/TestIdProject.FullyQualifiedTestStrategy/TestIdProject.FullyQualifiedStrategy.csproj" /> entry next to the existing TestIdProject.DefaultStrategy entry in TestFx.slnx (around line 110), and the equivalent entry in MSTest.slnf's project list (around line 31), so the project at least builds as part of the normal solution/filter restore-build cycle.

Task 2: Add (or restore) an acceptance/unit test that actually exercises the fully-qualified strategy build output

Priority: High
Estimated Effort: Medium

Find or write an acceptance test (likely in test/IntegrationTests/MSTest.Acceptance.IntegrationTests/ParameterizedTestTests.cs, alongside the existing DynamicDataAttribute.TestIdGenerationStrategy coverage) that runs the TestIdProject.FullyQualifiedTestStrategy test host and asserts the fully-qualified test IDs/display names it produces, mirroring how TestIdProject.DefaultStrategy is presumably exercised. Without this, simply adding the project to the solution (Task 1) only guarantees it compiles, not that its behavior is verified.

Task 3: Rename the .csproj file to match its folder name

Priority: Low
Estimated Effort: Small

Rename TestIdProject.FullyQualifiedStrategy.csproj to TestIdProject.FullyQualifiedTestStrategy.csproj to match the containing folder and the naming convention used by the sibling (TestIdProject.DefaultStrategy/TestIdProject.DefaultStrategy.csproj). This is a pure rename with no behavior change, but it removes a Test-dropping inconsistency that likely contributed to the project being missed during solution wiring.

Task 4: Add a lightweight CI/build check that flags .csproj files absent from TestFx.slnx

Priority: Medium
Estimated Effort: Medium

Add a script (PowerShell, per repo convention, under eng/) that enumerates all non-sample, non-playground .csproj files under test/ and src/ and fails if any are not referenced by TestFx.slnx (excluding intentionally-excluded folders such as samples/public/* sub-solutions and generated obj/bin outputs). Run it in CI to prevent a repeat of this specific class of gap, where a newly-added test asset's companion project silently never gets wired into the main solution.

📊 Historical Context

This is the 81st run of this workflow. Prior related-but-distinct reports: #9715 and #9198 both listed TestIdProject.FullyQualifiedTestStrategy as one of 20+ projects with hardcoded net462 targeting, but neither identified that the project is completely unreferenced by any solution file or test — a build/coverage gap, not a TFM-modernization gap.

🎯 Recommendations

Immediate Actions (This Week)

1. Wire TestIdProject.FullyQualifiedTestStrategy.csproj into TestFx.slnx and MSTest.slnf (Task 1) — Priority: High

Short-term Actions (This Month)

1. Add acceptance-test coverage exercising the fully-qualified test-ID strategy build output (Task 2) — Priority: High
2. Add a CI guard against future solution-coverage gaps (Task 4) — Priority: Medium

Next analysis: 2026-10-20 — Focus area selected based on diversity algorithm

🤖 Automated content by GitHub Copilot. Generated by the Repository Quality Improver workflow. · copilot · auto · 187.8 AIC · ⌖ 11.9 AIC · ⊞ 17.3K · [◷]( · ◷)

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Oct 21, 2026, 10:32 PM UTC