[allure-behave] Steps from tag-excluded scenarios bleed into the next matching scenario's result when hide_excluded=true
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 84/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- python
- Domain
- testing-qa
Research direction
Start in listener.py at AllureListener.stop_scenario(), where excluded scenarios are dropped, and reproduce the issue with the provided behave command using hide_excluded=true. Inspect the generated allure-results/*-result.json files; done means a matching scenario's result contains only its own steps, with no steps from preceding tag-excluded scenarios.
Written by the indexing model from the issue text.
Description
Environment
- allure-behave: 2.15.3
- allure-commons: (run
pip show allure-commons) - behave: 1.3.3
- Python: 3.12
Describe the Bug
When running behave with a tag filter (e.g. -t ETS-TC-372) and
-D AllureFormatter.hide_excluded=true, the Allure result JSON for the
matching scenario accumulates steps from all tag-excluded scenarios that
appeared before it in the same feature file.
For example: if Sample.feature has 12 scenarios before ETS-TC-372, and
those 12 scenarios collectively have 179 steps, the result JSON for ETS-TC-372
shows 227 steps (179 orphaned + 48 real) instead of 48.
Root Cause
Behave calls formatter.step(step) for every scenario — including
tag-excluded ones — because show_skipped=True by default
(run_scenario or show_skipped in Scenario.run()).
This schedules excluded scenario steps into AllureListener.self.steps (a
deque).
In stop_scenario(), excluded scenarios hit the should_drop_excluded=True
branch and call self.logger.drop_test(uuid) immediately, without ever
calling flush_steps() or clearing the deque. All accumulated steps remain
in the deque.
When the first matching scenario runs, its steps are appended to the
already-populated deque. match_step() pops from the front (FIFO), consuming
the orphaned steps and stamping them into the matching scenario's TestResult.
Reproduction Steps
- Create a feature file with 5+ scenarios where only the last one has
@target-tag. - Run:
behave -t target-tag my.feature \ --format allure_behave.formatter:AllureFormatter \ --outfile allure-results \ -D AllureFormatter.hide_excluded=true - Inspect allure-results/*-result.json — the step count will be
(sum of all excluded scenario steps) + (target scenario steps).
Expected Behaviour
The result JSON for the matching scenario should contain only its own
steps.
Fix
In AllureListener.stop_scenario(), clear the deque before dropping the
test so orphaned steps do not leak into the next scenario:
listener.py — stop_scenario()
if should_drop_skipped_by_option or should_drop_excluded:
self.steps.clear() # ← add this line
self.logger.drop_test(self.current_scenario_uuid)
This one-line fix prevents accumulated steps from tag-excluded scenarios
bleeding into any subsequent scenario's result.
- Dominant language
- Python
- Stars
- 815
- Forks
- 260
- Avg merge
- 9h 12m
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from allure-framework/allure-python
-
Added Categories Open
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
Difficulty 3/5 1-2 days Newbie friendliness 58/100
allure-framework/allure-python#918 · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
allure-framework/allure-python#903 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
-
Difficulty 3/5 1-2 days Newbie friendliness 42/100
All issues in allure-framework/allure-python
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100