Bug Report: stepByStepReport not capturing screenshots in CodeceptJS 3.6+

Inizia da lib/plugin/stepByStepReport.js e riproduci il problema con la configurazione minima di CodeceptJS fornita nella versione 3.7.5. Esamina l’handler event.step.after e l’uso di recorder, quindi verifica che un test fallito produca gli screenshot PNG previsti in output/record*/ sia per le esecuzioni normali sia per quelle parallele.

Hello,

The stepByStepReport plugin does not capture screenshots after each step in CodeceptJS versions 3.6.0 through 3.7.5. The record_*/ directory is created but only contains an empty index.html file with no screenshots (PNG files).

Affected Versions

• Broken: CodeceptJS 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.7.5
• Working: CodeceptJS 3.5.0 and earlier

Root Cause

The bug was introduced in commit 1040494 (Jun 8, 2024) titled "fix: screenshot error in beforeSuite/AfterSuite (#4385)".

The Change

In lib/plugin/stepByStepReport.js, the event.step.after handler was changed:

Before (working - v3.5.0):

event.dispatcher.on(event.step.after, (step) => {
  recorder.add('screenshot of failed test', async () => persistStep(step), true);
});

After (broken - v3.6+):

event.dispatcher.on(event.step.after, persistStep);

Why This Breaks

The persistStep function is an async function that calls helper.saveScreenshot(). In the original implementation, this was wrapped in recorder.add(), which properly queues the async operation and waits for completion.

In the new implementation, persistStep is called directly as an event handler callback. Since it's an async function, the event dispatcher doesn't wait for it to complete. The screenshots are never captured because:

1. The async function starts executing
2. The event dispatcher immediately moves on to the next event
3. The screenshot operation may be cancelled or never completes

Reproduction Steps

1. Create a minimal CodeceptJS config with only stepByStepReport enabled
2. Run any test that fails (to keep the report)
3. Check the _output/record_*/ directory
4. The directory contains only index.html with no PNG files

Minimal Reproduction

// codecept.minimal.conf.js
const path = require('path');

exports.config = {
  output: path.resolve(__dirname, '_output'),
  helpers: {
    Playwright: {
      browser: 'chromium',
      url: 'https://www.google.com',
      show: true
    }
  },
  plugins: {
    stepByStepReport: {
      enabled: true
    }
  },
  tests: './test_minimal.js',
  name: 'minimal-test'
};

// test_minimal.js
Feature('Minimal Test');

Scenario('Test stepByStepReport', ({ I }) => {
  I.amOnPage('https://www.google.com');
  I.wait(2);
  I.see('Google');
  I.see('TEXT THAT DOES NOT EXIST'); // Force failure to keep report
});

Run with:

npx codeceptjs@3.7.5 run --config codecept.minimal.conf.js

Expected: _output/record_*/ contains 0000.png, 0001.png, etc.
Actual: _output/record_*/ contains only index.html with no PNGs

Suggested Fix

Restore the original behavior by wrapping persistStep in recorder.add():

event.dispatcher.on(event.step.after, step => {
  recorder.add('screenshot of step of test', async () => persistStep(step), true);
});

This ensures the async screenshot operation is properly queued and executed.

Additional Context

• This issue has been reported before as #4856 (Feb 2025) and is a follow-up of #4637
• The original fix was intended to prevent errors in BeforeSuite/AfterSuite hooks, but it inadvertently broke the core screenshot functionality
• The issue affects both single-run and parallel execution modes

Environment

• CodeceptJS: 3.7.5
• Node.js: 20.x
• Playwright: 1.57.0
• OS: Linux (Ubuntu 22.04)

Written with help from Claude 4.5 Opus.