[Dynamic configuration] [Playwright] Can't reconfigure configuration per scenario

Run the reproduction in dynamic-configuration/override-config.e2e-test.js and compare the per-Scenario .config() calls with the working Feature('Dynamic Configuration').config() case. Trace how Scenario configuration reaches the Playwright helper, then verify that the browser override takes effect for each scenario while the command-line --override behavior remains unchanged.

What are you trying to achieve?

I'm trying to reconfigure browser configuration per scenario

What do you get instead?

Provide console output if related. Use --verbose mode for more details.

CodeceptJS v3.0.4
Helpers: Playwright
Plugins: screenshotOnFail, retryFailedStep, tryTo

Dynamic Configuration --
    [1]  Starting recording promises
 › [Session] Starting singleton browser session
  should override browser in config per scenario try#1 @dynamic-configuration
 › [Test Config] Playwright {"browser":"firefox"}
    I am on page "/"
    › [Browser:Warning] 
    I execute script () => !!window.chrome
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  ✔ OK in 3224ms

 › [Test Config] Reverted for Playwright
 › [Session] cleaning cookies and localStorage
 › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  should override browser in config per scenario try#2 @dynamic-configuration
 › [Test Config] Playwright {"Playwright":{"browser":"firefox"}}
    I am on page "/"
    › [Browser:Warning] 
    I execute script () => !!window.chrome
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  ✔ OK in 611ms

 › [Test Config] Reverted for Playwright
 › [Session] cleaning cookies and localStorage
 › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  should override browser in config per scenario try#3 @dynamic-configuration
 › [Test Config] Playwright {"helpers":{"Playwright":{"browser":"webkit"}}}
    I am on page "/"
    › [Browser:Warning] 
    I execute script () => !!window.chrome
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
    › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@
  ✔ OK in 552ms

 › [Test Config] Reverted for Playwright
 › [Session] cleaning cookies and localStorage
 › [Browser:Warning] JSHandle@THREE.WebGLProgram: gl.getProgramInfoLog() JSHandle@

  OK  | 3 passed   // 5s

Provide test source code if related

const assert = require('assert');

Feature('Dynamic Configuration');

Scenario('should override browser in config per scenario try#1 @dynamic-configuration', async ({ I }) => {
    I.amOnPage('/');
    const isChrome = await I.executeScript(() => !!window.chrome);

    assert(isChrome, 'should check that browser is chrome');
}).config({"browser": "firefox"});

Scenario('should override browser in config per scenario try#2 @dynamic-configuration', async ({ I }) => {
    I.amOnPage('/');
    const isChrome = await I.executeScript(() => !!window.chrome);

    assert(isChrome, 'should check that browser is chrome');
}).config({"Playwright": {"browser": "firefox"}});

Scenario('should override browser in config per scenario try#3 @dynamic-configuration', async ({ I }) => {
    I.amOnPage('/');
    const isChrome = await I.executeScript(() => !!window.chrome);

    assert(isChrome, 'should check that browser is chrome');
}).config({ "helpers": {"Playwright": {"browser": "webkit"}}});

Details

• CodeceptJS version: v3.0.4
• NodeJS Version: v12.18.3
• Operating System: macOS Catalina 10.15.7
• playwright
• Configuration file:

const { setHeadlessWhen } = require('@codeceptjs/configure');

// turn on headless mode when running with HEADLESS=true environment variable
// export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS);

exports.config = {
  tests: '*/*.e2e-test.js',
  output: './output',
  helpers: {
    Playwright: {
      url: 'http://github.com',
      show: true,
      browser: 'chromium'
    }
  },
  include: {
    I: './steps_file.js'
  },
  bootstrap: null,
  mocha: {},
  name: 'experiments-with-codeceptjs',
  plugins: {
    pauseOnFail: {},
    retryFailedStep: {
      enabled: true
    },
    tryTo: {
      enabled: true
    },
    screenshotOnFail: {
      enabled: true
    }
  }
}

NOTE:

But it works for the Feature

Feature('Dynamic Configuration').config({"browser": "firefox"})

and I can override the configuration in runtime by using --override or -o option in the command line, passing valid JSON as a value:

codeceptjs run -o '{ "helpers": {"Playwright": {"browser": "firefox"}}}'

Also here is the link to reproduce the issue:
https://github.com/elaichenkov/experiments-with-codeceptjs/blob/main/dynamic-configuration/override-config.e2e-test.js