pytest-html version 3.1.1 bug- Internal error in pytest_runtest_makereport causes test to show as Passed in HTML report

Open
  • Is something wrong?
#1,027 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
html, python
Domain
testing-qa

Research direction

Start by reproducing the issue with pytest-html 3.1.1, pytest, --html=report.html, and an exception or keyboard interrupt in the pytest_runtest_makereport hook. Trace how the hook result reaches the HTML report and ensure the affected test is not displayed as Passed; the intended ERROR versus FAILED outcome still needs resolution.

Written by the indexing model from the issue text.

Description

Description

When an internal error occurs inside the pytest_runtest_makereport hook,
pytest-html 3.1.1 does not receive a valid report outcome and silently
defaults to displaying the test as Passed in the HTML report.

Environment

  • pytest-html: 3.1.1
  • pytest: 7.1.2 or 9.0.3
  • Python: 3.13.7
  • OS: Windows

Expected Behavior

Test should be marked as ERROR in the HTML report.

Actual Behavior

Test is marked as Passed in the HTML report, giving a false green result.

Steps to Reproduce

  1. Trigger an exception inside pytest_runtest_makereport hook for eg raise RuntimeError("BOOM")
  2. Run pytest with --html=report.html
  3. Open the HTML report
  4. Observe the affected test shows as Passed

Additional Notes

pytest-html 4.2.0 handles this differently — the report is empty instead
of showing a false Passed — but neither behavior is correct.
the correct expected outcome is debatable:
ERROR — represents a framework/fixture level problem, not a test assertion failure. This is probably the most accurate.
FAILED — could argue this since something clearly went wrong.
At minimum — not Passed, since that's factually wrong.
-In case of keyboard interrupt too same issue

Dominant language
Python
Stars
779
Forks
261
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pytest-dev/pytest-html

All issues in pytest-dev/pytest-html

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.