Browser plugin 5.0.1: waitForExpectation's hardcoded 1s per-attempt timeout now takes effect, silently re-running page scripts

Aperta
#1,852 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 5 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
php, playwright
Ambito
testing

Direzione di ricerca

Start with src/Execution.php:138-160 and src/Api/AwaitableWebpage.php:48-58, then inspect src/Playwright/Client.php:84-89 and src/Api/Webpage.php:85-88. Reproduce the issue with the provided slow script and attempt counter. Done means the configured timeout no longer causes unsafe repeated script execution, with regression coverage for the observed behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Environment
  • pestphp/pest v5.1.0
  • pestphp/pest-plugin-browser v5.0.1 (upgraded from v4.3.1)
  • Playwright 1.62.x, PHP 8.4, macOS
Summary

After upgrading the browser plugin from 4.3.1 to 5.0.1, browser tests whose page scripts take longer than one second began failing with measurements taken from a page that had been mutated by its own earlier, partially-completed executions.

The cause is not new code — it is a pre-existing hardcoded 1s per-attempt timeout in waitForExpectation() that only became effective in 5.0.1, because 5.0.1 fixed how the timeout is transmitted to Playwright.

Root cause

Execution::waitForExpectation() caps every attempt at a hardcoded 1000 ms, then makes one final uncapped call after the loop:

// src/Execution.php:138-160
public function waitForExpectation(callable $callback): mixed
{
    $timeout = Playwright::timeout();
    // ...
    while (microtime(true) < $end) {
        try {
            return Playwright::usingTimeout(1_000, $callback);   // <-- hardcoded
        } catch (ExpectationFailedException) {
            //
        }
        $this->resetAssertions($originalCount);
        self::instance()->tick();
    }

    return $callback();   // <-- final attempt, uncapped
}

AwaitableWebpage::__call() routes calls through that retry loop (unless the method is in $nonAwaitableMethods, or the configured timeout is <= 1000):

// src/Api/AwaitableWebpage.php:48-58
if (in_array($name, $this->nonAwaitableMethods, true) || Playwright::timeout() <= 1000) {
    $result = $webpage->{$name}(...$arguments);
} else {
    $result = Execution::instance()->waitForExpectation(
        fn () => $webpage->{$name}(...$arguments),
    );
}

So visit(...)->script(...) — and anything else going through AwaitableWebpage — is executed under a 1s cap, repeatedly.

Why 4.3.1 was unaffected

The cap existed before, but never reached Playwright. 5.0.1's client now sends the action timeout in both params and metadata, with the reason documented in the code itself:

// src/Playwright/Client.php:84-89
// Playwright reads the action timeout from the metadata since 1.62, and read it
// from the params before that. Both are validated with a schema that silently
// drops unknown keys, so sending it twice also covers an older install.
'params' => ['timeout' => $timeout, ...$params],
'metadata' => ['timeout' => $timeout, ...$meta],

That change is correct on its own terms. Its side effect is that the hardcoded 1s per-attempt cap in waitForExpectation() is now enforced by the server, where under 4.3.1 + Playwright ≥ 1.62 it was silently ignored.

Observed behaviour

For a page script that legitimately takes longer than 1s:

  1. The attempt is aborted PHP-side at 1s while its page-side effects continue to land.
  2. The loop retries. With a 30s configured timeout we measured 31 attempts (confirmed with an in-page attempt counter incremented on window).
  3. The final, uncapped call then returns a measurement of a page that has been mutated by up to 30 partially-completed predecessors.

The failure does not look like a timeout. It looks like wrong data — which is considerably harder to diagnose.

Counterintuitive consequence

Because the per-attempt cap is a constant and the loop bound is the configured timeout, raising the timeout does not give a slow script more time — it grants it more 1s-truncated retries. Conversely, setting the timeout to <= 1000 disables the retry wrapper entirely (per the AwaitableWebpage guard above) and the script runs once, uncapped. Both directions are the opposite of what a user would expect.

Minimal reproduction
it('reproduces the 1s per-attempt cap', function (): void {
    $result = visit('/')->script(<<<'JS'
        () => {
            window.__attempts = (window.__attempts ?? 0) + 1;
            const start = Date.now();
            while (Date.now() - start < 1500) { /* > 1s of work */ }
            return window.__attempts;
        }
    JS);

    expect($result)->toBe(1); // observed: a much higher number
});
Workaround (for anyone hitting this)

Call ->page()->evaluate(...) instead of ->script(...). Webpage::script() is literally a passthrough:

// src/Api/Webpage.php:85-88
public function script(string $content): mixed
{
    return $this->page->evaluate($content);
}

so the two are semantically identical, but going through page() bypasses the AwaitableWebpage retry wrapper: one attempt, full timeout budget. Converting our call sites resolved the failures (one affected file went from 7 failures in 311s to 14/14 in 113s; the full browser suite from mixed failures to 58/58 in 164s).

Suggested direction
  1. Derive the per-attempt timeout from the configured timeout rather than hardcoding 1000 ms (or make it configurable).
  2. Consider whether user-supplied scripts should be routed through a retry loop at all — retrying a callback that mutates page state cannot be safe in general. If retries stay, documenting that expectations passed to waitForExpectation() must be idempotent would help.

Happy to test a patch against a real suite.

Lingua principale
PHP
Stelle
11.8k
Fork
559
Merge medio
5g 23h
PR unite (30g)
24

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pestphp/pest

Tutte le issue di pestphp/pest

Issue simili

Altre issue su PHP

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.