dotnet/roslyn

Generate expected source file for the whole unit test session

Open

#26,161 opened on Apr 14, 2018

View on GitHub
 (4 comments) (0 reactions) (0 assignees)C# (20,414 stars) (4,257 forks)batch import
Area-CompilersTesthelp wanted

Description

I found this method in the test suite: TryGenerateExpectedSourceFileAndGetDiffLink

It generates the whole source file with just the expected assertion string replaced with actual.

It has some limitations and cannot be used for bulk baseline changes,

  • It only works if a diff tool is available
  • It only works for a single test

The most needed methods that need to have this is VerifyIL and VerifyDiagnostics mainly in case of bulk baseline changes.

The idea is that to keep a single file and apply new changes to that file so the expected source file contains changes for all tests in the file.

However, in order to do that, we should keep track of line numbers because they might get out of date once a test expected string is updated in the source.

Also, since VerifyDiagnostics accepts a params, we can't have line number at all. maybe fallback to stackframe? (my motivating use case for this feature request https://github.com/dotnet/csharplang/issues/1458)

Thoughts?

/cc @tmat

Contributor guide

Tech stack
csharp
Domain
testingtooling
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
C#Roslyn test frameworkunit testing basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25
Research direction
Investigate the existing method TryGenerateExpectedSourceFileAndGetDiffLink in the test utilities of Roslyn. Study how VerifyIL and VerifyDiagnostics work and how they generate expected source files. Propose a design to handle line number tracking across multiple tests in a file. Consider the limitations mentioned in the issue and the linked csharplang discussion.