hedgehogqa/haskell-hedgehog
在 GitHub 查看Adjust coverage resolution based on the number of tests
Open
#309 创建于 2019年7月9日
bughelp wanted
描述
When running more than 100 tests, the resolution of the coverage output can be confusing:
✗ prop_validates_age failed
after 1000 tests.
exactly 18, validated on common day 13% ██▌················· ✓ 5%
exactly 18, validated on leap day 5% █··················· ✓ 1%
close to 18, non-leapling 27% █████▎·············· ✓ 5%
⚠ close to 18, leapling 1% ···················· ✗ 1%
too young 30% █████▉·············· ✓ 10%
not yet born 5% ▉··················· ✓ 1%
old enough 70% ██████████████······ ✓ 30%
In this case:
⚠ close to 18, leapling 1% ···················· ✗ 1%
We only covered 0.9% but it is rounded to 1% when displayed.
We should adjust the precision of displayed coverage based on the number of tests, so something like:
-- 100 tests
close to 18, non-leapling 27% █████▎·············· ✓ 5%
⚠ close to 18, leapling 0% ···················· ✗ 1%
-- 1000 tests
close to 18, non-leapling 27.4% █████▎·············· ✓ 5%
⚠ close to 18, leapling 0.9% ···················· ✗ 1%
-- 10000 tests
close to 18, non-leapling 27.39% █████▎·············· ✓ 5%
⚠ close to 18, leapling 0.09% ···················· ✗ 1%
-- 100000 tests
close to 18, non-leapling 27.392% █████▎·············· ✓ 5%
⚠ close to 18, leapling 0.009% ···················· ✗ 1%
-- etc
Coverage requirements (the right column) can only be integers, at least at the moment. So we might as well keep them with no decimal places.
I suspect this might need to be changed so there is no rounding. It is probably a hangover from before the rounding was done in the reporting code itself.