[allure-behave] Steps from tag-excluded scenarios bleed into the next matching scenario's result when hide_excluded=true

Open Beginner friendly
#902 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python
Domain
testing-qa

Research direction

Start in listener.py at AllureListener.stop_scenario(), where excluded scenarios are dropped, and reproduce the issue with the provided behave command using hide_excluded=true. Inspect the generated allure-results/*-result.json files; done means a matching scenario's result contains only its own steps, with no steps from preceding tag-excluded scenarios.

Written by the indexing model from the issue text.

Description

Environment

  • allure-behave: 2.15.3
  • allure-commons: (run pip show allure-commons)
  • behave: 1.3.3
  • Python: 3.12

Describe the Bug

When running behave with a tag filter (e.g. -t ETS-TC-372) and
-D AllureFormatter.hide_excluded=true, the Allure result JSON for the
matching scenario accumulates steps from all tag-excluded scenarios that
appeared before it in the same feature file.

For example: if Sample.feature has 12 scenarios before ETS-TC-372, and
those 12 scenarios collectively have 179 steps, the result JSON for ETS-TC-372
shows 227 steps (179 orphaned + 48 real) instead of 48.

Root Cause

Behave calls formatter.step(step) for every scenario — including
tag-excluded ones — because show_skipped=True by default
(run_scenario or show_skipped in Scenario.run()).

This schedules excluded scenario steps into AllureListener.self.steps (a
deque).

In stop_scenario(), excluded scenarios hit the should_drop_excluded=True
branch and call self.logger.drop_test(uuid) immediately, without ever
calling flush_steps() or clearing the deque
. All accumulated steps remain
in the deque.

When the first matching scenario runs, its steps are appended to the
already-populated deque. match_step() pops from the front (FIFO), consuming
the orphaned steps and stamping them into the matching scenario's TestResult.

Reproduction Steps

  1. Create a feature file with 5+ scenarios where only the last one has
    @target-tag.
  2. Run:
    behave -t target-tag my.feature \
      --format allure_behave.formatter:AllureFormatter \
      --outfile allure-results \
      -D AllureFormatter.hide_excluded=true
    
  3. Inspect allure-results/*-result.json — the step count will be
    (sum of all excluded scenario steps) + (target scenario steps).

Expected Behaviour

The result JSON for the matching scenario should contain only its own
steps.

Fix

In AllureListener.stop_scenario(), clear the deque before dropping the
test so orphaned steps do not leak into the next scenario:

listener.py — stop_scenario()

if should_drop_skipped_by_option or should_drop_excluded:
self.steps.clear() # ← add this line
self.logger.drop_test(self.current_scenario_uuid)

This one-line fix prevents accumulated steps from tag-excluded scenarios
bleeding into any subsequent scenario's result.

Dominant language
Python
Stars
815
Forks
260
Avg merge
9h 12m
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from allure-framework/allure-python

All issues in allure-framework/allure-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.