Duplicated counting test case numbers with pytest-rerunfailures plugin
@gnikonorov 已经在做这个了。
开始于 2020年8月2日。
评估
这个 Issue 还没有评估数据。
描述
If one test cases failed in teardown method with --reruns=1, this case will be calculated as 2 cases.
Test code:
class Test1(object):
def teardown_method(self, method):
raise Exception('tewrdown method error')
def test1(self):
pass
with --reruns=1:
$ py.test test_rerun.py -v -s --junitxml=test-result.xml --reruns=1
=============================================================== test session starts ================================================================
platform darwin -- Python 2.7.14, pytest-2.9.1, py-1.7.0, pluggy-0.3.1 -- /Users/jzhang/miniconda2/envs/p2/bin/python
cachedir: .cache
rootdir: /Users/jzhang/Desktop/test_log, inifile:
plugins: rerunfailures-4.1
collected 1 items
test_rerun.py::Test1::test1 PASSED
test_rerun.py::Test1::test1 RERUN
test_rerun.py::Test1::test1 PASSED
test_rerun.py::Test1::test1 ERROR
---------------------------------------- generated xml file: /Users/jzhang/Desktop/test_log/test-result.xml ----------------------------------------
====================================================================== ERRORS ======================================================================
_________________________________________________________ ERROR at teardown of Test1.test1 _________________________________________________________
self = <test_rerun.Test1 object at 0x104073190>, method = <bound method Test1.test1 of <test_rerun.Test1 object at 0x104073190>>
def teardown_method(self, method):
> raise Exception('tewrdown method error')
E Exception: tewrdown method error
test_rerun.py:4: Exception
==================================================== 2 passed, 1 error, 1 rerun in 0.02 seconds ====================================================
(p2) # ~/Desktop/test_log [18:10:33] C:1
$ more test-result.xml
<?xml version="1.0" encoding="utf-8"?><testsuite errors="1" failures="0" name="pytest" skips="0" tests="2" time="0.015"><testcase classname="test_rerun.Test1" file="test_rerun.py" line="5" name="test1" time="0.00033712387085"></testcase><testcase classname="test_rerun.Test1" file="test_rerun.py" line="5" name="test1" time="0.000302791595459"><error message="test setup failure">self = <test_rerun.Test1 object at 0x104073190>, method = <bound method Test1.test1 of <test_rerun.Test1 object at 0x104073190>>
def teardown_method(self, method):
> raise Exception('tewrdown method error')
E Exception: tewrdown method error
test_rerun.py:4: Exception</error></testcase></testsuite>
It calculated the number of test cases as 2, we can see tests="2".
when run this case without --retrun=1:
$ py.test test_rerun.py -v -s --junitxml=test-result.xml
=============================================================== test session starts ================================================================
platform darwin -- Python 2.7.14, pytest-2.9.1, py-1.7.0, pluggy-0.3.1 -- /Users/jzhang/miniconda2/envs/p2/bin/python
cachedir: .cache
rootdir: /Users/jzhang/Desktop/test_log, inifile:
plugins: rerunfailures-4.1
collected 1 items
test_rerun.py::Test1::test1 PASSED
test_rerun.py::Test1::test1 ERROR
---------------------------------------- generated xml file: /Users/jzhang/Desktop/test_log/test-result.xml ----------------------------------------
====================================================================== ERRORS ======================================================================
_________________________________________________________ ERROR at teardown of Test1.test1 _________________________________________________________
self = <test_rerun.Test1 object at 0x103d40d50>, method = <bound method Test1.test1 of <test_rerun.Test1 object at 0x103d40d50>>
def teardown_method(self, method):
> raise Exception('tewrdown method error')
E Exception: tewrdown method error
test_rerun.py:4: Exception
======================================================== 1 passed, 1 error in 0.02 seconds =========================================================
(p2) # ~/Desktop/test_log [18:19:55] C:1
$ more test-result.xml
<?xml version="1.0" encoding="utf-8"?><testsuite errors="1" failures="0" name="pytest" skips="0" tests="1" time="0.015"><testcase classname="test_rerun.Test1" file="test_rerun.py" line="5" name="test1" time="0.000448226928711"><error message="test setup failure">self = <test_rerun.Test1 object at 0x103d40d50>, method = <bound method Test1.test1 of <test_rerun.Test1 object at 0x103d40d50>>
def teardown_method(self, method):
> raise Exception('tewrdown method error')
E Exception: tewrdown method error
test_rerun.py:4: Exception</error></testcase></testsuite>
It calculated the number of test cases as 1, we can see tests="1", it looks more reasonable.
- 主要语言
- Python
- 星标
- 481
- 派生
- 110
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
pytest-dev/pytest-rerunfailures 的其他 Issue
-
Zero rerun delay can abort with OverflowError for a large finite backoff factor可能已有人在做 @AryanRG 于 13 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
-
Non-finite rerun delay settings abort pytest with INTERNALERROR可能已有人在做 @Arham-Begani 于 11 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 35/100
-
fail-on-flaky returns success after a rerun with xdist可能已有人在做 @harbinresearcher 于 14 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 15/100
-
难度 5/5 一周以上 新手友好度 25/100
pytest-dev/pytest-rerunfailures#326 · 7 条评论 ·
-
Add ability to raise runtime warnings on reruns可能已有人在做 @LouisDeconinck 于 42 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 45/100
pytest-dev/pytest-rerunfailures#318 · 2 条评论 ·
查看 pytest-dev/pytest-rerunfailures 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 79/100
aws-samples/sample-per-model-bedrock#639 ·
维护者通常 1 天内回复
-
bug
难度 1/5 1 小时以内 新手友好度 86/100
awslabs/visual-asset-management-system#937 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 79/100
project-codeflare/codeflare-sdk#1195 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
pandas-dev/pandas-stubs#2051 ·
维护者通常 1 天内回复