stretchr/testify

Show a summary of failing tests within a suite

Open

#244 opened on Nov 20, 2015

View on GitHub
 (5 comments) (3 reactions) (0 assignees)Go (25,958 stars) (1,704 forks)batch import
help wantedrevisit

Description

Currently, if some but not all tests within a suite fail, this is opaque to go get. Thus, all tests within a failing suite are printed. This can make it hard to figure out which test has failed without carefully scrolling, searching or grepping through a test result.

A simple improvement would be to display a Suite level summary of the tests. For example, instead of displaying only "--FAIL: TestMySuite (0.00)" at the end a failing suite, how about:

--- FAIL: TestMySuite (0.00s)
FAIL     github.com/brycefisher/my-project/my-package MySuite -- TestWithinSuite
FAIL     github.com/brycefisher/my-project/my-package MySuite -- TestWithinSuite2

Rerun the failing tests with:
go test github.com/brycefisher/my-project/my-package -run=TestMySuite -m=TestWithinSuite && \
go test github.com/brycefisher/my-project/my-package -run=TestMySuite -m=TestWithinSuite2

This would at a glance make it clear which specific tests failed, and make it easy to run just the failing tests over and over.

I'm not super familiar with the internals of mockery and suites, but I imagine that dynamically looking up the package and determining the suite and test are not super challening.

Thoughts?

Contributor guide

Tech stack
go
Domain
testing
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Go testingtestify suitestest output formatting
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Examine the suite test runner's output formatting logic in the testify repository, likely in files like 'suite.go' or 'assertions.go'. Check existing comments in the issue for implementation ideas. Consider how to capture individual test failures within a suite and format a summary. Ensure the rerun command suggestion is compatible with Go's test flags.