Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[bug] Retry + assert in async scenario does not work

Open
#2,892 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
javascript, node.js, playwright
Domain
testing-qa

Research direction

Reproduce the two async scenarios from the issue with CodeceptJS 3.0.5 or 3.0.6, Node.js 14.16.1, and Playwright, comparing retry and no-retry runs. Trace the async scenario retry handling from that reproduction; done means the unexpected assertion remains a test failure after retries and the final output reports one failed test.

Written by the indexing model from the issue text.

Description

bug
What are you trying to achieve?

Assertion like the doc example (but with an unexpected value in the assertion):

const assert = require('assert')

Scenario.only('test page title 1', async ({ I }) => {
  I.amOnPage('https://www.google.com')
  const title = await I.grabTitle()
  assert.strictEqual(title, 'Bad value')
}).retry(2)

Scenario('test page title 2', { retry: 2 }, async ({ I }) => {
  I.amOnPage('https://www.google.com')
  const title = await I.grabTitle()
  assert.strictEqual(title, 'Bad value')
})
What do you get instead?

With retry option, the test does not fail (ok at the end), it's unexpected:

> codeceptjs run --steps "--verbose"

CodeceptJS v3.0.5
Using test root "---"
Helpers: Playwright, Context, Network
Plugins: screenshotOnFail, retryFailedStep, tryTo

US checkout / LP0001 --
    [1]  Starting recording promises
 › [Session] Starting singleton browser session
  test page title 1
    I am on page "https://www.google.com"
    I grab title
    [1] Error | AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
    [1] Starting <teardown> session
    [1] <teardown> Stopping recording promises
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /----/codecept/output/test_page_title_1.failed.png
    [2]  Starting recording promises
 › [Session] cleaning cookies and localStorage

  OK  | 0 passed   // 2s

Without the retry, it's good, the test fails as expected:

> lv-front@1.0.2 codecept
> codeceptjs run --steps

CodeceptJS v3.0.5
Using test root "-----"

US checkout / LP0001 --
  test page title 1
    I am on page "https://www.google.com"
    I grab title
  ✖ FAILED in 1059ms


-- FAILURES:

  1) US checkout / LP0001
       test page title 1:

      Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
      + expected - actual

      -Google
      +Bad value

  AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
  + actual - expected



  Artifacts:
  - screenshot: /----/codecept/output/test_page_title_1.failed.png


  FAIL  | 0 passed, 1 failed   // 1s
Details
  • CodeceptJS version: 3.0.5 (but 3.0.6 tested too with the same bug)
  • NodeJS Version: 14.16.1
  • Operating System: macosX
  • playwright
Dominant language
JavaScript
Stars
4.2k
Forks
756
Avg merge
2d 9h
Merged PRs (30d)
16

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from codeceptjs/CodeceptJS

All issues in codeceptjs/CodeceptJS

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.