Ability to fail tests if minimum coverage not met

Open
#160 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
csharp
Domain
build-system, cli, testing-qa

Research direction

The issue names the coverage extension, command-line option, MSBuild property, and TestingPlatformCommandLineArguments as the relevant entry points; start by tracing how coverage settings are accepted and how the build result is reported. Done means a configurable threshold can be supplied in the shown command-line and per-test-project forms, with builds failing when coverage is below it.

Written by the indexing model from the issue text.

Description

enhancement

Summary

Add the ability to fail the build if a minimum level of coverage is not achieved.

Background and Motivation

Playing around with an existing xunit v3 project to have it use Microsoft Testing Platform, and as part of that work to maintain code coverage I've needed to drop coverlet to instead use the coverage extension.

A missing feature by comparison compared to coverlet is the ability to fail the build if a minimum code threshold isn't met.

In coverlet this is achieved via an MSBuild property in one of two formats:

<!-- Fail if line, method or branch coverage is less than 70 -->
<Threshold>70</Threshold>

<!-- Fail if line, method or branch coverage is less than 70, 75 and 50 respectively -->
<Threshold>70,75,50</Threshold>

Proposed Feature

Add a new feature that can be controlled through the command-line, for example:

--coverage-threshold 75

This could then be customised on a per-test-project basis:

<TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --coverage-threshold 42</TestingPlatformCommandLineArguments>

Alternative Designs

None proposed.

Dominant language
C#
Stars
125
Forks
17
Avg merge
1h 17m
Merged PRs (30d)
2

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/codecoverage

All issues in microsoft/codecoverage

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.