[Dynamic configuration] [Playwright] Can't reconfigure configuration per scenario

Ejecuta la reproducción en dynamic-configuration/override-config.e2e-test.js y compara las llamadas a .config() de cada Scenario con el caso funcional Feature('Dynamic Configuration').config(). Rastrea cómo llega la configuración de Scenario al helper de Playwright y, a continuación, verifica que el override del navegador surta efecto para cada Scenario, mientras el comportamiento de --override en la línea de comandos permanece sin cambios.

What are you trying to achieve?

I'm trying to reconfigure browser configuration per scenario

What do you get instead?

Provide console output if related. Use --verbose mode for more details.

CodeceptJS v3.0.4
Helpers: Playwright
Plugins: screenshotOnFail, retryFailedStep, tryTo

Dynamic Configuration --
    [1]  Starting recording promises
 › [Session] Starting singleton browser session
  should override browser in config per scenario try#1 @dynamic-configuration
 › [Test Config] Playwright {"browser":"firefox"}
    I am on page "/"
    › [Browser:Warning] 
    I execute script () => !!window.chrome
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  ✔ OK in 3224ms

 › [Test Config] Reverted for Playwright
 › [Session] cleaning cookies and localStorage
 › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  should override browser in config per scenario try#2 @dynamic-configuration
 › [Test Config] Playwright {"Playwright":{"browser":"firefox"}}
    I am on page "/"
    › [Browser:Warning] 
    I execute script () => !!window.chrome
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  ✔ OK in 611ms

 › [Test Config] Reverted for Playwright
 › [Session] cleaning cookies and localStorage
 › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  should override browser in config per scenario try#3 @dynamic-configuration
 › [Test Config] Playwright {"helpers":{"Playwright":{"browser":"webkit"}}}
    I am on page "/"
    › [Browser:Warning] 
    I execute script () => !!window.chrome
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  ✔ OK in 552ms

 › [Test Config] Reverted for Playwright
 › [Session] cleaning cookies and localStorage
 › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@

  OK  | 3 passed   // 5s

Provide test source code if related

const assert = require('assert');

Feature('Dynamic Configuration');

Scenario('should override browser in config per scenario try#1 @dynamic-configuration', async ({ I }) => {
    I.amOnPage('/');
    const isChrome = await I.executeScript(() => !!window.chrome);

    assert(isChrome, 'should check that browser is chrome');
}).config({"browser": "firefox"});

Scenario('should override browser in config per scenario try#2 @dynamic-configuration', async ({ I }) => {
    I.amOnPage('/');
    const isChrome = await I.executeScript(() => !!window.chrome);

    assert(isChrome, 'should check that browser is chrome');
}).config({"Playwright": {"browser": "firefox"}});

Scenario('should override browser in config per scenario try#3 @dynamic-configuration', async ({ I }) => {
    I.amOnPage('/');
    const isChrome = await I.executeScript(() => !!window.chrome);

    assert(isChrome, 'should check that browser is chrome');
}).config({ "helpers": {"Playwright": {"browser": "webkit"}}});

Details

• CodeceptJS version: v3.0.4
• NodeJS Version: v12.18.3
• Operating System: macOS Catalina 10.15.7
• playwright
• Configuration file:

const { setHeadlessWhen } = require('@codeceptjs/configure');

// turn on headless mode when running with HEADLESS=true environment variable
// export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS);

exports.config = {
  tests: '*/*.e2e-test.js',
  output: './output',
  helpers: {
    Playwright: {
      url: 'http://github.com',
      show: true,
      browser: 'chromium'
    }
  },
  include: {
    I: './steps_file.js'
  },
  bootstrap: null,
  mocha: {},
  name: 'experiments-with-codeceptjs',
  plugins: {
    pauseOnFail: {},
    retryFailedStep: {
      enabled: true
    },
    tryTo: {
      enabled: true
    },
    screenshotOnFail: {
      enabled: true
    }
  }
}

NOTE:

But it works for the Feature

Feature('Dynamic Configuration').config({"browser": "firefox"})

and I can override the configuration in runtime by using --override or -o option in the command line, passing valid JSON as a value:

codeceptjs run -o '{ "helpers": {"Playwright": {"browser": "firefox"}}}'

Also here is the link to reproduce the issue:
https://github.com/elaichenkov/experiments-with-codeceptjs/blob/main/dynamic-configuration/override-config.e2e-test.js