tryTo() triggering false positive to pauseOnFail - stopping at first scenario

Reproduce the provided test.js scenarios with npx codeceptjs run ./tests/test.js --verbose -p pauseOnFail and compare the pauseOnFail and tryTo plugin behavior. No implementation file or test file beyond test.js is named; done means the first scenario completes without an erroneous pause and the second scenario runs.

What are you trying to achieve?

Trying to run 2 scenarios with one containing tryTo().

What do you get instead?

With the flag -p pauseOnFail, codeceptjs is stopping after scenario 1, even when no error or failure happens.

I believe this is a false positive triggered by tryTo() as seeing in the verbose:

[1] Error (Non-Terminated) | Error: Element "NOPE" is not visible on page. | (err) => { step.status = 'failed'; step.endTime =

Provide console output if related. Use --verbose mode for more details.

> npx codeceptjs run ./tests/test.js --verbose -p pauseOnFail

***************************************
nodeInfo:  18.20.4
osInfo:  Linux 5.15 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
cpuInfo:  (32) x64 Intel(R) Core(TM) i9-14900HX
chromeInfo:  Not Found
edgeInfo:  "N/A"
firefoxInfo:  undefined
safariInfo:  N/A
If you need more detailed info, just run this: npx codeceptjs info
***************************************
CodeceptJS v3.6.6 #StandWithUkraine
Using test root "xxx"
Helpers: Playwright, Hooks
Plugins: screenshotOnFail, tryTo, retryFailedStep, retryTo, eachElement, pauseOnFail

Fixed Assets --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  01 - Test
    I am on page "https://github.com/login"
      I am on page "https://github.com/login"
    Info: RetryFailedStep plugin is disabled inside tryTo block
    I see element "NOPE"
      I see element "NOPE"
    [1] <tryTo>  Error (Non-Terminated) | Error: Element "NOPE" is not visible on page. | (err) => { step.status = 'failed'; step.endTime = ...
 › Unsuccessful try > Error: Element "NOPE" is not visible on page.
  ✔ OK in 3961ms

 Interactive shell started
 Use JavaScript syntax to try steps in action
 - Press ENTER to run the next step
 - Press TAB twice to see all available commands
 - Type exit + Enter to exit the interactive shell
 - Prefix => to run js commands 
 I.

Provide test source code if related

Feature('Fixed Assets');

Scenario('01 - Test',  async ({ I }) => {
  await I.amOnPage('https://github.com/login');
  await tryTo(() => I.seeElement('NOPE'))
});

Scenario('02 - Test',  async ({ I }) => {
  await I.amOnPage('https://github.com/login');
});

Details

• CodeceptJS version: 3.6.6
• NodeJS Version: 18.20.4
• Operating System: Ubuntu 22.04.4
• Playwright
• Configuration file:

require('dotenv').config();
const http = require('http');
require('./heal');
const { setHeadlessWhen, setCommonPlugins } = require('@codeceptjs/configure');

// Turn on headless mode when running with HEADLESS=true environment variable
// Export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS);

// Enable all common plugins https://github.com/codeceptjs/configure#setcommonplugins
setCommonPlugins();

/** @type {CodeceptJS.MainConfig} */
exports.config = {
  name: 'xxx',
  // For local development we rename the files we want to test
  tests: './tests/*.js',
  output: './output',
  helpers: {
    Playwright: {
      browser: 'chromium',
      url: 'http://localhost',
      // Default time for all wait* actions
      waitForTimeout: 60000,
      // Time to wait after each action
      waitForAction: 1000,
      // Highlight element only work with --verbose
      highlightElement: true,
      show: true,
      video: true,
      keepVideoForPassedTests: true,
      windowSize: '1024x768',
      recordVideo: { // Seems to not work but required to use windowSize -- solved in 3.6.2
        size: {
          width: 1024,
          height: 768
        }
      },
      trace: false
    },
    Hooks: {
      require: './events/hooks.js'
    },
  },
}