unisonweb/unison

Tests that crash should get converted to failures

Open

#2,089 opened on Jun 7, 2021

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Haskell (6,624 stars) (304 forks)batch import
error-messagehelp wanted

Description

Right now the test command isn’t expecting exceptions when evaluating the tests. When exceptions (like calls to bug) do occur, it isn’t correctly attributed to the failing test.

Contributor guide

Tech stack
None
Domain
testing
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Understanding of Unison language fundamentals
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50
Research direction
Investigate the test command implementation in the Unison repository to understand how exceptions are currently handled. Look for the function that runs tests and see where exceptions are not being caught. The comment on the issue may provide additional insights. Consider modifying the test runner to catch exceptions and attribute them to the failing test.