Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Silent Failures of Assertions When Using --grep with Full Feature-Scenario Name

Closed
#5,528 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Domain
cli, testing

Research direction

Reproduce the issue with the Feature/Scenario Outline and the .steps.ts step definition, comparing npx codeceptjs run --grep "Featurename: Scenarioname" with the scenario-only command. Start at the CLI --grep handling and follow how the full feature-scenario match executes assertions and records results. Done means failed assertions are reported correctly and the run no longer shows a misleading OK | 0 passed.

Written by the indexing model from the issue text.

Description

stale

Summary

I am encountering an issue when running specific scenarios with the --grep flag. Specifically, when using npx codeceptjs run --grep "Featurename: Scenarioname", node assertions like assert.equal(1,2) are not being processed correctly and test failures are silent. However, running the same test with npx codeceptjs run --grep "Scenarioname" works as expected. Since a couple of versions CodeceptJS gives the first command as a suggestion if no matching scenario was found for a test run:

No tests found by pattern: /Featurename: Scenarioname_misspelled/
Maybe you wanted to run one of these tests?
...
To run the first test use the following command:
npx codeceptjs run --debug --grep "Featurename: Scenarioname "

This issue leads to assertions seemingly passing, and the test run result shows a green OK | 0 passed, despite a failure condition and the test run itself being aborted correctly.
If a pattern is suggested for use, it should work as expected and handle failing assertions correctly, which in this case it does not.

Steps to Reproduce
  1. Define a Feature and Scenario Outline in a .feature file
  2. Add a step definition for the Scenario that contains a false node assertion in a .steps.ts file
  3. Run CodeceptJS with the full feature-scenario name as a filter:
    npx codeceptjs run --grep "Featurename: Scenarioname"
    If the assertion evaluates to a failure the test will silently pass, and the terminal will show OK | 0 passed even if the whole test should have failed.
  4. If the same test is run with just the scenario name filter:
    npx codeceptjs run --grep "Scenarioname"
    The test executes correctly, fails and prints FAIL | 0 passed, 1 failed.

Code Example

Feature File:
Feature: Billing

  As a user, I want to do billing calculations.

  Scenario Outline: Monthly <usecase>
    Given I am on page billing
    When I settle the bill
    Then I see that the number of billed items <billedItems> is correct

  Examples:
    | usecase | dateFrom   | dateTo         | billedItems |
    | Normal  | 2023-01-01| 2023-01-31  | 10                 |
   

Step Definition:

import assert from "node:assert"
....
Then("I see that the number of billed items {int} is correct", async (billedItems: number) => {
    assert.equal(billedItems, 10);  // Example check, should fail if numbers don't match
})

Expected Behavior

When running npx codeceptjs run --grep "Featurename: Scenarioname", the assertions should execute, and any failures should be properly tracked and printed by CodeceptJS.
The output should show either 1 failed or 1 passed, depending on the assertion and should not silently fail with 0 passed.

Environment

CodeceptJS version: 3.7.8
Node.js version: v25.9.0

Dominant language
JavaScript
Stars
4.2k
Forks
756
Avg merge
2d 9h
Merged PRs (30d)
16

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from codeceptjs/CodeceptJS

All issues in codeceptjs/CodeceptJS

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.