[Bug]: Pest 5 concurrency — parallel coverage loses a covered line, and --mutate --parallel reports survivors as killed
I maintainer di solito rispondono entro 5 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 38/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- php
- Ambito
- testing-qa
Direzione di ricerca
Start by running the listed serial and parallel coverage and mutation commands on a representative suite, then diff their reports across repeated runs. Focus on the two unresolved findings, using the stated Pest, php-code-coverage, PHP, and PCOV versions; done means parallel results agree with serial results or the unsupported combination is clearly handled.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
What Happened
Three independent findings landed on the same day across a 46-repository fleet on Pest 5. Individually each looked like bad luck. Together they are one shape: a Pest 5 run that forks produces a different, quieter answer than the same run serial — and in two of the three the wrong answer is the more flattering one.
I am filing them together deliberately, because the pattern is the point. Each is separable and carries its own evidence.
One of the three has since been confirmed and fixed by a maintainer, which is why I think the other two are worth a look rather than a shrug.
1 — A parallel coverage run loses a line that is demonstrably covered
Status: reproducible in place, deterministic, not yet minimised.
| serial | pest --coverage --min=100 --exclude-group=browser → 100.0% |
| parallel | pest --coverage --min=100 --exclude-group=browser --parallel --processes=4 → 99.9% |
| the line | EmailTwoFactorChallengeController line 197, reported uncovered |
| repeats | same line, every parallel run |
| control | on Pest 4, this exact parallel command was green |
Same tree, same tests, same driver settings (PCOV, XDEBUG_MODE=off exported as an env var so workers inherit it — not -d). The only variable is --parallel, and php-code-coverage moving 12.5.7 → 14.2.3 with the major.
It is not a threshold artefact: serial says the line is covered, parallel says it is not, and it picks the same line each time. That reads like a coverage merge losing one worker's contribution rather than a flaky collection.
The workaround in place is to run coverage serially, which costs wall-clock and nothing else — but it means every repository that trusts --parallel --coverage on Pest 5 may be enforcing a floor against a number that is slightly too low, and would fail a --min=100 gate for a line that is fine.
I have not minimised this to a standalone package yet. If a smaller reproducer is what unblocks it, say so and I will build one — I did not want to sit on the field evidence while doing that.
2 — --mutate --parallel reports surviving mutants as killed
Status: hand-verified, and the wrong number is the higher one.
Same class, same tests, same command apart from --parallel:
| MSI | |
|---|---|
pest --mutate (serial) |
89% |
pest --mutate --parallel |
100% |
The survivors that vanish under --parallel are real: verified by hand against the serial run. So the parallel score is not noisier — it is inflated, in the direction nobody double-checks. A score that reads 100% is the one least likely to be questioned.
This is now a standing rule across the fleet: mutation runs serially, always, and parallelism is expressed as sharding at the container level (one serial --mutate per shard) rather than --parallel. That is a workaround for consumers; upstream it seems worth either fixing or refusing the flag combination outright, because the failure mode is a fabricated pass.
This is adjacent to #1316 but not the same direction: there, tested code is reported UNTESTED; here, untested code is reported killed.
3 — The type-coverage cache rewrote itself (already fixed — included as the confirmation)
pest --type-coverage --min=100, no --parallel, died with:
ParseError: syntax error, unexpected single-quoted string " => "
at vendor/pestphp/pest-plugin-type-coverage/.temp/v3.php:14
12▕ );baef33c5698f9926572068f087c' =>
13▕ array (
➜ 14▕ 0 => '/…/app/Models/SystemFile.php',
The cache file is torn mid array-literal: two writes interleaved. Stack trace named Pokio\Runtime\Fork\ThrowableCapsule::__unserialize(). Turning concurrency off at the command line does not escape it, because the plugin forks internally. And it is a race — the next run over the identical tree was green.
This one is already fixed, in pest-plugin-type-coverage v5.0.1 (fix: parallel corruption file). The old withinLock() tried flock(LOCK_EX|LOCK_NB) a hundred times over 100 ms and then, on giving up, ran the callback anyway — an unlocked write. Making the lock blocking closed it.
I include it because it is the same family and it was real, so it is some evidence that 1 and 2 are worth measuring rather than dismissing as environment.
How to reproduce
For 1 and 2: any suite large enough to have a coverage merge worth doing. Run each command twice, once with --parallel and once without, and diff the reports — the difference is stable, not flaky.
Sample Code / Versions
Pest 5, php-code-coverage 14.2.3, PCOV, PHP 8.4/8.5, Linux containers (4 processes) and macOS both. Finding 3 on pest-plugin-type-coverage v5.0.0, fixed in v5.0.1.
- Lingua principale
- PHP
- Stelle
- 11.8k
- Fork
- 559
- Merge medio
- 5g 23h
- PR unite (30g)
- 24
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di pestphp/pest
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: --mutate hides unreadable files, so mutants behind a failed read can never be killedForse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: [mutate] Windows: @pest-mutate-ignore comments land on line 1 in files with LF line endingsForse già presa @webdevsamran l’ha presa 57 giorni fa. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 5 giorni
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 80/100
I maintainer di solito rispondono entro 5 giorni
-
[pest-plugin-browser] Page::waitForLoadState() never sends its message — waitForEvent() is a silent no-opForse già presa @asllanmaciel l’ha presa 70 giorni fa. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
pestphp/pest#1892 · 2 commenti ·
I maintainer di solito rispondono entro 5 giorni
Tutte le issue di pestphp/pest
Issue simili
-
Group 2 - Data Storage | Tripal DBX | Chado Group 5 - Search | ElasticSearch | Views Priority - Medium
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
-
nginx update required due to NGINX2RootForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 65/100
mailcow/mailcow-dockerized#7668 · 5 commenti · 2 reazioni ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
lara-zeus/spatie-translatable#80 ·
I maintainer di solito rispondono entro 1 giorno
-
small
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
uiowa/uiowa#10296 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
asyncapi/modelina#2709 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno