[Bug]: Browser plugin: a test that fails on its first check is reported as risky, and an earlier page's server exception reaches later pages
I maintainer di solito rispondono entro 5 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 30/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Ambito
- testing-qa
Direzione di ricerca
The fixes sit in the browser plugin's sources (the issue says its tracker is closed, so the work lands upstream): src/Api/Concerns/MakesElementAssertions.php for assertSee(), src/Drivers/LaravelHttpServer.php for $lastThrowable, and src/Plugin.php for afterEach. Start by running the two reproduction tests from the issue on a Laravel app with the plugin installed. Done means the first-check failure reports 1 assertion with no risky event, and the second test fails on Goodbye without the earlier server exception in its message.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
pestphp/pest-plugin-browser has issues turned off, so I file this here.
Each case below reproduces in a fresh laravel/laravel app with only Pest and the browser plugin added, on plugin 5.0.1 and on 5.2.0 (the 5.x head).
Line numbers are for 5.2.0, with 5.0.1 in brackets where it differs.
Versions: pestphp/pest-plugin-browser 5.0.1 and 5.2.0, pestphp/pest 5.2.1, phpunit/phpunit 13.3.4, PHP 8.5.8, Playwright 1.63.0 (Chromium), laravel/framework 13.39.0, macOS.
Routes used by the tests:
Route::get('/ok', fn () => 'Hello');
Route::get('/fail', fn () => throw new RuntimeException('Thrown by /fail.'));
1. A test that fails on its first browser check is also risky
it('fails on its first check', function () {
visit('/ok')->assertSee('Goodbye');
});
Run: vendor/bin/pest tests/Browser/FirstCheckFailsTest.php --log-junit junit.xml --log-events-text events.txt
Expected: 1 failed test with 1 assertion and no risky event, as for a failed PHPUnit, Pest or Laravel assertion.
Actual: Tests: 1 failed (0 assertions), assertions="0" in junit.xml, and in events.txt:
Test Considered Risky (P\Tests\Browser\FirstCheckFailsTest::__pest_evaluable_it_fails_on_its_first_check)
This test did not perform any assertions
laravel/pao 1.1.5 reports "assertions":0,"failed":1,"risky":1.
A failed first action does the same: visit('/ok')->click('No such button') fails with Timeout 5000ms exceeded., 0 assertions, risky.
A test that passes one check before it fails is not affected.
Cause: these calls throw ExpectationFailedException directly, with no assertion counted: assertSee() and assertDontSee() (src/Api/Concerns/MakesElementAssertions.php:61, :79), which count one only when they pass, a failed action in Client::execute() (src/Playwright/Client.php:127 [106]), GuessLocator::for() (src/Support/GuessLocator.php:75 [65]) and ArrayablePendingAwaitablePage::ensureOnWebpage() (src/Api/ArrayablePendingAwaitablePage.php:151).
PHPUnit's own assertions count before they throw, and PHPUnit marks a test with 0 assertions as risky unless it errored, was skipped or is incomplete (src/Framework/TestRunner/TestRunner.php:224-234), so a failed test is not exempt.
Suggested fix: count one assertion before each of these throws, as assertPortIs() already does (src/Api/Concerns/MakesUrlAssertions.php:109-110).
With expect(true)->toBeTrue(); before the throw in assertSee(), the test above gives 1 failed (1 assertions) and no risky event.
2. The server exception of an earlier page reaches later pages of the same test
it('does not throw the exception of an earlier page again', function () {
$this->withoutExceptionHandling();
expect(fn () => visit('/fail')->assertSee('Server Error'))
->toThrow(RuntimeException::class, 'Thrown by /fail.');
visit('/ok')->assertSee('Hello');
});
it('names only the exception of the page that failed', function () {
visit('/no-such-page')->assertSee('Not Found');
visit('/ok')->assertSee('Goodbye');
});
Expected: the first test passes, and the second fails on Goodbye with no server exception in its message, because /ok throws nothing.
Actual: the first test fails with RuntimeException: Thrown by /fail., thrown again by visit('/ok')->assertSee('Hello'); the same call passes in a test of its own.
The failure message of the second test contains:
The following exception was thrown by the HTTP server:
Symfony\Component\HttpKernel\Exception\NotFoundHttpException: The route no-such-page could not be found.
Cause: LaravelHttpServer keeps the last request exception in $lastThrowable (src/Drivers/LaravelHttpServer.php:352, :368 [281, 293]) and clears it only in flush(), which runs in the plugin's afterEach (src/Plugin.php:55 [53]).
Until then, AwaitableWebpage::__call() throws it again after each page call under withoutExceptionHandling() (src/Api/AwaitableWebpage.php:89, :100 [61, 72]), and each failed check adds it to its message (src/Exceptions/BrowserExpectationFailedException.php:60 [51]).
Suggested fix: clear $lastThrowable when throwLastThrowableIfNeeded() throws it, and when Page::goto() starts a new page load.
With both changes, both tests behave as expected, and visit('/fail')->assertSee('Goodbye') still names RuntimeException: Thrown by /fail. in its message.
3. Request: keep the plugin's frames out of failure traces
The failure of the test in case 1 is located in the plugin: the trace in junit.xml starts at vendor/pestphp/pest-plugin-browser/src/Exceptions/BrowserExpectationFailedException.php:69 [60], and laravel/pao reports that file and line.
Pest puts its own src on PHPUnit's exclude list (src/Bootstrappers/BootExcludeList.php), but the plugin does not.
With PHPUnit\Util\ExcludeList::addDirectory() for the plugin's src, the trace is only at tests/Browser/FirstCheckFailsTest.php:4.
PHPUnit then also ignores the warnings that the plugin suppresses with @, such as the rmdir() warnings that pestphp/pest-plugin-browser#250 and pestphp/pest-plugin-browser#278 fixed in 5.2.0.
- Lingua principale
- PHP
- Stelle
- 11.8k
- Fork
- 559
- Merge medio
- 5g 23h
- PR unite (30g)
- 24
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di pestphp/pest
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: --mutate hides unreadable files, so mutants behind a failed read can never be killedForse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: [mutate] Windows: @pest-mutate-ignore comments land on line 1 in files with LF line endingsForse già presa @webdevsamran l’ha presa 57 giorni fa. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 5 giorni
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 80/100
I maintainer di solito rispondono entro 5 giorni
-
[pest-plugin-browser] Page::waitForLoadState() never sends its message — waitForEvent() is a silent no-opForse già presa @asllanmaciel l’ha presa 70 giorni fa. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
pestphp/pest#1892 · 2 commenti ·
I maintainer di solito rispondono entro 5 giorni
Tutte le issue di pestphp/pest
Issue simili
-
nginx update required due to NGINX2RootForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 65/100
mailcow/mailcow-dockerized#7668 · 5 commenti · 2 reazioni ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
lara-zeus/spatie-translatable#80 ·
I maintainer di solito rispondono entro 1 giorno
-
small
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
uiowa/uiowa#10296 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
asyncapi/modelina#2709 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
My-Little-Forum/mylittleforum#937 ·