Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

`--report-gh` annotations, groups and notices are dropped under `dotnet test`; only the job summary survives

Open
  • Is something wrong?
#11,996 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
56/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp, github-actions
Domain
testing

Research direction

Start by comparing direct executable output with the dotnet test paths in the linked repro, especially src/Repro and the dotnet-test.yml workflow. Read AzureDevOpsCommandOutputDeviceData, which the issue identifies as a possible analogue, and trace how GitHub workflow commands are emitted under dotnet test. Done when annotations, notices, and log groups reach the workflow-command stream for project, solution, and bare dotnet test runs while job summaries still work.

Written by the indexing model from the issue text.

Description

area/mtp-github-actions-report external/dotnet-test needs/triage

Describe the bug

With Microsoft.Testing.Extensions.GitHubActionsReport and --report-gh on GitHub Actions, running through dotnet test (MTP mode) writes the Markdown job summary but none of the workflow commands the extension emits: no ::error / ::warning annotations, no ::notice slow-test notices, no ::group:: log groups. Running the same test exe directly emits all of them.

The docs describe these as defaults whenever GITHUB_ACTIONS=true and --report-gh is set, and don't mention a dotnet test exception. The only dotnet test mention is that the summary aggregates across modules.

Version used

  • .NET SDK 10.0.402 (CI: latest 10.0.x)
  • Microsoft.Testing.Extensions.GitHubActionsReport 2.5.1 (also reproduced on 2.4.1)
  • MSTest.Sdk 4.5.1 (also 4.3.0)
  • Microsoft.Testing.Platform: 2.5.1
  • Runner: ubuntu-latest; also reproduced locally on my Windows 11 machine.

Steps To Reproduce

A minimal repro runs on GitHub Actions: https://github.com/BenjaminMichaelis/GitHubActionsReport.POC

  • src/Repro: MSTest.Sdk project with a passing, a failing, a skipped ([Ignore]) and a slow (3s) test. src/Repro2 is a second test project so the solution run has two modules.
  • exe-direct.yml runs the built exe with --report-gh --report-gh-slow-test-threshold 1s.
  • dotnet-test.yml runs the same tests through dotnet test --project, dotnet test --solution and bare dotnet test.
  • Each job tees stdout to a file and counts workflow-command lines. GitHub parses commands from that same stream, so a line missing from the file never reached the runner.

Expected behavior

Actual behavior

Invocation ::group:: ::error ::warning (skipped) ::notice (slow) Job summary
exe directly 1 1 1 2 written
dotnet test --project 0 0 0 0 written
dotnet test --solution 0 0 0 0 written (both modules)
bare dotnet test 0 0 0 0 written

It also shows on pull requests. There is one PR per scenario (#1 exe-direct, #2 project, #3 solution, #4 bare), each editing the annotated lines. Only the exe-direct PR gets the failed-test and skipped-test annotations on Tests.cs in the Files changed tab.

Additional context

At a glance, I think that the GitHub path likely needs something similar to AzureDevOpsCommandOutputDeviceData.

Dominant language
C#
Stars
1.1k
Forks
312
Avg merge
8h 29m
Merged PRs (30d)
421

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/testfx

All issues in microsoft/testfx

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.