Bug: retryOnFailedStep works only for the first test in the same run

retryFailedStep プラグインと Playwright ヘルパーとの相互作用から始めます。issue の 2 つのシナリオを再現し、特に store.autoRetries と recorder.retries に関して、テスト間で retry の状態がどのようにリセットされるかを調べます。両方のシナリオで失敗したステップに設定済みの retry が適用されれば完了です。

I have the following test file:

Feature("github testing")

Scenario("test", async ({ I }) => {
  await I.amOnPage('https://github.com');
  await I.click('thistextdoesnotexist');
})

Scenario("test2", async ({ I }) => {
  await I.amOnPage('https://github.com');
  await I.click('thistextdoesnotexisttoo');
})

and I enabled the retryFailedStep plugin (this is the only plugin, also using playwright helper)

I noticed that the plugin doesn't retry the click step in test2

I patched two functions in the plugin (codeceptjs v4.0.3):

99   const when = err => {
100     if (!enableRetry) return
101     if (store.debugMode) return false
102     console.log("store.autoRetries", store.autoRetries); // <-- this is new
103     if (!store.autoRetries) return false
104     if (err && err.isTerminal) return false
105     if (err && err.message && (err.message.includes('ERR_ABORTED') || err.message.includes('frame was detached') || err.message.includes('Target page, context or browser has been closed'))) return false
106     return true
107   }

and

140     if (scenarioRetries > 0 && config.deferToScenarioRetries !== false) {
141       store.autoRetries = false
142       return
143     }
144 
145     const hasManualRetries = recorder.retries.some(retry => retry !== config) // in the second test, recorder.retries.length === 3
146     console.log("hasManualRetries", hasManualRetries); // <-- this is new
147     if (hasManualRetries) {
148       store.autoRetries = false
149       return
150     }
151

I ran it and this is the output:

github testing --
hasManualRetries false
store.autoRetries true
store.autoRetries true
store.autoRetries true
store.autoRetries true
  ✖ test in 7886ms
hasManualRetries true
store.autoRetries false
store.autoRetries false
  ✖ test2 in 5760ms

Thanks!