Silent Failures of Assertions When Using --grep with Full Feature-Scenario Name
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
- 難度
- 3/5
- 預估耗時
- 1-2 天
- 新手友好度
- 52/100
- Issue 類型
- 缺陷
- 描述清晰度
- 基本清楚
- 活躍度
- 冷清
- 技術堆疊
- javascript, node.js, typescript
研究方向
使用 Feature/Scenario Outline 和 .steps.ts 中的步驟定義重現此問題,並將 npx codeceptjs run --grep "Featurename: Scenarioname" 與僅執行 scenario 的命令進行比較。從 CLI 的 --grep 處理開始,追蹤完整的 feature-scenario 比對如何執行 assertion 並記錄結果。完成標準是失敗的 assertion 能正確回報,且執行結果不再顯示誤導性的 OK | 0 passed。
由索引模型根據 Issue 內容生成。
描述
Summary
I am encountering an issue when running specific scenarios with the --grep flag. Specifically, when using npx codeceptjs run --grep "Featurename: Scenarioname", node assertions like assert.equal(1,2) are not being processed correctly and test failures are silent. However, running the same test with npx codeceptjs run --grep "Scenarioname" works as expected. Since a couple of versions CodeceptJS gives the first command as a suggestion if no matching scenario was found for a test run:
No tests found by pattern: /Featurename: Scenarioname_misspelled/
Maybe you wanted to run one of these tests?
...
To run the first test use the following command:
npx codeceptjs run --debug --grep "Featurename: Scenarioname "
This issue leads to assertions seemingly passing, and the test run result shows a green OK | 0 passed, despite a failure condition and the test run itself being aborted correctly.
If a pattern is suggested for use, it should work as expected and handle failing assertions correctly, which in this case it does not.
Steps to Reproduce
- Define a Feature and Scenario Outline in a .feature file
- Add a step definition for the Scenario that contains a false node assertion in a .steps.ts file
- Run CodeceptJS with the full feature-scenario name as a filter:
npx codeceptjs run --grep "Featurename: Scenarioname"
If the assertion evaluates to a failure the test will silently pass, and the terminal will showOK | 0 passedeven if the whole test should have failed. - If the same test is run with just the scenario name filter:
npx codeceptjs run --grep "Scenarioname"
The test executes correctly, fails and printsFAIL | 0 passed, 1 failed.
Code Example
Feature File:
Feature: Billing
As a user, I want to do billing calculations.
Scenario Outline: Monthly <usecase>
Given I am on page billing
When I settle the bill
Then I see that the number of billed items <billedItems> is correct
Examples:
| usecase | dateFrom | dateTo | billedItems |
| Normal | 2023-01-01| 2023-01-31 | 10 |
Step Definition:
import assert from "node:assert"
....
Then("I see that the number of billed items {int} is correct", async (billedItems: number) => {
assert.equal(billedItems, 10); // Example check, should fail if numbers don't match
})
Expected Behavior
When running npx codeceptjs run --grep "Featurename: Scenarioname", the assertions should execute, and any failures should be properly tracked and printed by CodeceptJS.
The output should show either 1 failed or 1 passed, depending on the assertion and should not silently fail with 0 passed.
Environment
CodeceptJS version: 3.7.8
Node.js version: v25.9.0
- 主要語言
- JavaScript
- 星號
- 4.2k
- 分支
- 756
- 平均合併
- 1 天 6 小時
- 30 天內合併 PR
- 19
環境準備
- 提供 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
codeceptjs/CodeceptJS 的其他 Issue
-
stale
難度 2/5 1-3 小時 新手友好度 72/100
codeceptjs/CodeceptJS#5420 · 1 則留言 ·
維護者通常 1 天內回覆
-
stale
難度 2/5 1-3 小時 新手友好度 76/100
codeceptjs/CodeceptJS#5358 · 1 則留言 ·
維護者通常 1 天內回覆
-
stale
難度 2/5 1-3 小時 新手友好度 78/100
codeceptjs/CodeceptJS#4958 · 10 則留言 · 2 個 reaction ·
維護者通常 1 天內回覆
-
stale
難度 1/5 1 小時以內 新手友好度 72/100
codeceptjs/CodeceptJS#4778 · 3 則留言 ·
維護者通常 1 天內回覆
-
難度 3/5 1-2 天 新手友好度 68/100
codeceptjs/CodeceptJS#5725 ·
維護者通常 1 天內回覆
查看 codeceptjs/CodeceptJS 的全部 Issue
相似的 Issue
-
external-issue to-triage
難度 2/5 1-3 小時 新手友好度 82/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 84/100
LearningCircuit/local-deep-research#7067 ·
維護者通常 1 天內回覆
-
automated issue report
難度 1/5 1 小時以內 新手友好度 65/100
lirantal/discoprint#32 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 86/100
維護者通常 1 天內回覆