cucumberjson compatibility with pytest-timeout

tests/test_timeout.py、timeout.feature、pytest-timeout、および示されているコマンドを使ってケースを再現し、その後、生成されたoutput.jsonを調べます。タイムアウトしたステップがどのように記録されるかを追跡し、cucumberjsonの結果に失敗が反映されるようにします。出力でそのステップが合格として報告されなくなっていることを確認します。

This isn't truly an issue with pytest-bdd, but I thought this was worth mentioning given the popularity of the pytest-timeout plugin.

I noticed that if the pytest-timeout plugin is used, it can correctly fail a scenario, but all steps show as passed in the cucumberjson output. A minimal example:

test_timeout.py:

from pytest_bdd import scenario, when
import time

@scenario('timeout.feature', 'Long running operation')
def test_long_running():
    pass

@when('I wait for a really long time')
def wait_long_time():
    time.sleep(3)

timeout.feature:

Feature: Timeout demonstration
    Demonstration of a long-running operation

    Scenario: Long running operation
        When I wait for a really long time

then run pytest:

pytest -s tests/test_timeout.py --cucumberjson=output.json --timeout 1

to get the following logs:

=================================================================================================================================================== test session starts ====================================================================================================================================================
platform linux -- Python 3.12.7, pytest-8.3.5, pluggy-1.5.0
rootdir: /media/jason/e/pytest-bdd
configfile: pytest.ini
plugins: bdd-8.1.0, timeout-2.3.1
timeout: 1.0s
timeout method: signal
timeout func_only: False
collected 1 item                                                                                                                                                                                                                                                                                                           

tests/test_timeout.py OUTCOME: failed
F

========================================================================================================================================================= FAILURES =========================================================================================================================================================
____________________________________________________________________________________________________________________________________________________ test_long_running _____________________________________________________________________________________________________________________________________________________

fixturefunc = <function wait_long_time at 0x7130aa194360>, request = <FixtureRequest for <Function test_long_running>>, kwargs = {}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

venv/lib/python3.12/site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @when('I wait for a really long time')
    def wait_long_time():
>       time.sleep(3)
E       Failed: Timeout >1.0s

tests/test_timeout.py:10: Failed
------------------------------------------------------------------------------------------------------------------------- generated json file: /media/jason/e/pytest-bdd/output.json -------------------------------------------------------------------------------------------------------------------------
================================================================================================================================================= short test summary info ==================================================================================================================================================
FAILED tests/test_timeout.py::test_long_running - Failed: Timeout >1.0s

but the following output.json shows the step passed:

[
  {
    "keyword": "Feature",
    "uri": "tests/timeout.feature",
    "name": "Timeout demonstration",
    "id": "tests/timeout.feature",
    "line": 1,
    "description": "Demonstration of a long-running operation",
    "language": "en",
    "tags": [],
    "elements": [
      {
        "keyword": "Scenario",
        "id": "test_long_running",
        "name": "Long running operation",
        "line": 4,
        "description": "",
        "tags": [],
        "type": "scenario",
        "steps": [
          {
            "keyword": "When",
            "name": "I wait for a really long time",
            "line": 5,
            "match": {
              "location": ""
            },
            "result": {
              "status": "passed",
              "duration": 0
            }
          }
        ]
      }
    ]
  }
]

pytest==8.3.5
pytest-bdd==8.1.0
pytest-timeout==2.3.1