BDD skips tests in feature file if background fails in one test

Commencez par reproduire le fichier de feature fourni avec Playwright et examinez les étapes dans step_definitions/mock.steps.js. Suivez la manière dont l'exécuteur BDD gère un Background en échec entre les deux scénarios ; le travail est terminé lorsque les scénarios suivant un background en échec sont tentés indépendamment au lieu d'être marqués comme ignorés, avec une couverture de régression pour cet exemple.

What are you trying to achieve?

Run a feature file without failing fast and skipping tests if background fails.

What do you get instead?

If you have a feature file with 5 scenarios and a Background. If a background step fails on scenario 3, then scenario 4 and 5 are marked as skipped. As we know sometimes a step fails because of timing issues or a page fails to load, this doesnt mean the background will fail each time. Isn't a Background treated as a beforeEach in mocha anyway? So it should run independently for each scenario.

Provide console output if related. Use --verbose mode for more details.

CodeceptJS v3.0.7
Helpers: Playwright, MochiavelliHelper, PlaywrightHelper, ChaiWrapper
Plugins:

Failing Background --
    [1]  Starting recording promises
 › [Session] Starting singleton browser session
  Search for something @test1
    Given I am on page "https://google.com"
      I am on page "https://google.com"
      › [Browser:Error]
      › [Browser:Error]
      › [Browser:Error]
    Then I should see "Fail test here"
      I see "Fail test here"
    [1] Error | Error
    [1] Error | Error
    [1] Starting <teardown> session
    [1] <teardown> Stopping recording promises
  ✖ FAILED in 23592ms

    [2]  Starting recording promises
 › [Session] cleaning cookies and localStorage
  S Search for something else @test2

-- FAILURES:

  1) Failing Background
       "before each" hook: Before for "Search for something @test1":

      expected web application to include "Fail test here"
      + expected - actual

      -About
      -Store
      -GmailImages
      -Sign in
      -
      -
      -
      -Celebrating the diverse stories of Asian and Pacific Islander communities
      -Advertising
      -Business
      ---( 6 lines more )---
      +Fail test here



  Scenario Steps:
  - I.see("Fail test here") at ./step_definitions/mock.steps.js:148:5
  - I.amOnPage("https://google.com") at ./step_definitions/mock.steps.js:144:5



  FAIL  | 0 passed, 1 failed, 1 skipped   // 24s

Provide test source code if related

steps:

Given('I am on page {string}', (url) => {
  I.amOnPage(url);
});

Then('I should see {string}', (text) => {
  I.see(text);
});

When('I fill in {string} in the search field', (text) => {
  I.fillField('.gLFyf.gsfi', text);
  I.pressKey('Enter');
});

feature file

Feature: Failing Background

  Background: Test
    Given I am on page "https://google.com"
    Then I should see "Fail test here"

  @test1
  Scenario: Search for something
    When I fill in "Test" in the search field

  @test2
  Scenario: Search for something else
    When I fill in "Google" in the search field

Details

• CodeceptJS version: 3.0.7
• NodeJS Version: v14.15.1
• Operating System: osx 10.15
• playwright 1.10