Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

retryFailedStep ignores factor/minTimeout/maxTimeout with the Playwright helper (and applies them nondeterministically under Bun)

Aberta
  • Algo errado?
#5,723 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

@mirao já está trabalhando nisso.

Desde 30/9/2026.

  • #5724 de @mirao — aberto

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
68/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
bun, javascript, node.js, playwright
Domínio
testing, tooling

Direção de pesquisa

Start with lib/helper/Playwright.js and the recorder.add() retry selection described in the issue; inspect how the plugin and Playwright helper configs are pushed and selected. Reproduce the minimal case under Node and Bun with DEBUG=codeceptjs:recorder, then verify that retryFailedStep consistently honors its matched factor, minTimeout, maxTimeout, and retries across runtimes.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

What are you trying to achieve?

Configure the backoff of retryFailedStep (factor, minTimeout, maxTimeout) with the Playwright helper, and get the same retry timing on every run and runtime.

What do you get instead?

With the Playwright helper, the plugin's timing options are:

  • ignored under Node: the backoff always uses promise-retry's defaults (factor: 2);
  • applied only to some steps under Bun: the backoff alternates between the plugin's options and the defaults, so it depends on how many recorder tasks ran before the failing step.

Only retries is always right.

Minimal repro (a step that fails instantly, so the duration is pure backoff):

// codecept.conf.js
export const config = {
  tests: './*_test.js',
  output: './output',
  helpers: { Playwright: { url: 'about:blank', show: false, browser: 'chromium' } },
  plugins: { retryFailedStep: { enabled: true, retries: 4 } }, // default factor 1.5
}

// retry_test.js
Feature('retry')
Scenario('1 pass', ({ I }) => { I.amOnPage('about:blank') })
Scenario('2 fail', ({ I }) => { I.seeElement('#nope') })
Scenario('3 fail', ({ I }) => { I.seeElement('#nope') })
Runtime Failed scenario duration Backoff used
Node 24 ~2280 ms 150+300+600+1200 → factor 2 (plugin's 1.5 ignored)
Bun 1.4.2 ~1250 ms here, but varies per step in a real suite (we saw ×1.5 and ×2 in the same run) 150+225+338+506 → factor 1.5, or factor 2

DEBUG=codeceptjs:recorder shows the Retry opts alternating between the plugin config and {"retries":4} on consecutive tasks under Bun.

Cause

Two retry configs are active in each test:

  1. the plugin's (factor: 1.5, minTimeout, maxTimeout, …), pushed in test.before;
  2. the Playwright helper's own, pushed in _before() (lib/helper/Playwright.js, since 3.7.0): 
    this._retryConfig = {
  retries: test?.opts?.conditionalRetries || 3,
  when: err => /* only errors containing 'context' */,
}
recorder.retry(this._retryConfig)
    It copies only the retry count from the plugin, via conditionalRetries. Without factor/minTimeout/maxTimeout it falls back to promise-retry's defaults.

recorder.add() picks one config per task:

const retryOpts = this.retries
  .sort((r1, r2) => r1.when && !r2.when)
  .slice(-1)
  .pop()
  • The comparator returns a boolean. Per the spec that isn't a consistent comparator, so the order is implementation-defined. V8 keeps the order, so the helper's config (pushed last) always wins. JavaScriptCore (Bun, also Safari/WebKit) swaps the two elements on every call, and .sort() mutates this.retries in place, so the winner alternates from task to task. WebKit closed the same pattern as not a bug: https://bugs.webkit.org/show_bug.cgi?id=289223
  • Even with a deterministic sort, the timing comes from the helper's config, not from the rule whose when actually matched the error. For an element-not-found error only the plugin's when matches, yet the helper's defaults set the backoff.
Suggested fix
  • Use a numeric comparator, e.g. (r1.when ? 0 : 1) - (r2.when ? 0 : 1), and avoid sorting this.retries in place.
  • Take the promise-retry options from the retry rule that matched the error, or have the Playwright helper inherit all of the plugin's retry options, not only retries.

Workaround: set factor: 2 (and leave minTimeout/maxTimeout at their defaults) in the plugin config, so both configs are identical.

Details
  • CodeceptJS version: 4.2.0 (same code since 3.7.0)
  • NodeJS Version: 24.18.0 / Bun 1.4.2
  • Operating System: Linux
  • Playwright helper

Created by AI agent (Claude Code)

Linguagem predominante
JavaScript
Estrelas
4.2k
Forks
756
Merge médio
1d 8h
PRs com merge (30d)
18

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de codeceptjs/CodeceptJS

Todas as issues de codeceptjs/CodeceptJS

Issues semelhantes

Mais issues de JavaScript

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.