[5.x] TIA: replayed tests report stale/incomplete coverage, breaking --min
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 45/100
Direzione di ricerca
Start by tracing Pest\Plugins\Tia coverage handling, especially KEY_COVERAGE_CACHE (coverage.bin.gz) and the mergePerTestFiles() call that combines collector and recorder data. Reproduce with --coverage --tia --min=100 using a cold and warm ~/.pest/tia cache, then verify that replayed tests produce the same coverage totals and pass the minimum gate as a fresh run.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
ENVIRONMENT
pestphp/pest v5.0.2
phpunit/phpunit 13.2.6
phpunit/php-code-coverage 14.2.4
laravel/framework v13.23.0
PHP 8.4.20
Coverage driver pcov 1.0.13-dev (Xdebug not installed)
OS Linux 6.17.0 (Docker, SLES-based image)
Suite size 666 tests / 1687 assertions
SUMMARY
When --coverage and --tia are combined, coverage numbers depend on whether the
TIA cache is warm. With a cold cache the suite reports 100.0%. Running the exact
same command again — no source or test changes in between — reports 99.9% and
fails --min=100.
The drop is deterministic and always hits the same two files. Those files are
fully covered: running --coverage without --tia reports them at 100.0%.
This makes --coverage --tia unusable as a CI gate, and it is misleading during
development: it points at files as under-covered when their tests exist and pass.
STEPS TO REPRODUCE
-
Have a suite that reaches 100% line coverage.
-
Clear the TIA cache and run with coverage:
rm -rf ~/.pest/tia
php -d pcov.enabled=1 vendor/bin/pest --coverage --tia --min=100-> Tests: 666 passed (1687 assertions)
-> Total: 100.0 %
-> exit code 0 -
Run the exact same command again, changing nothing:
php -d pcov.enabled=1 vendor/bin/pest --coverage --tia --min=100
-> Tests: 666 passed (1687 assertions, 666 replayed)
-> Domain/B2B/Actions/UpdateProductAction ............... 94 / 97.4%
-> Livewire/B2B/Produtos/AbstractProdutosTable ... 306..348 / 98.9%
-> Total: 99.9 %
-> FAIL Code coverage below expected 100.0 %, currently 99.9 %
-> exit code 1 -
Confirm the files are actually covered, without TIA:
php -d pcov.enabled=1 vendor/bin/pest --coverage
-> Domain/B2B/Actions/UpdateProductAction ............... 100.0%
-> Livewire/B2B/Produtos/AbstractProdutosTable .......... 100.0%
-> Total: 100.0 %
Steps 2-4 reproduce every time, in this order.
EXPECTED BEHAVIOUR
Coverage totals should be identical whether tests are executed or replayed from
the TIA cache. A replayed test should contribute the same coverage data it
contributed when it last ran.
ACTUAL BEHAVIOUR
Replayed tests contribute incomplete coverage data. The reported total drops from
100.0% to 99.9%, and --min=100 fails on a suite that has not changed.
NOTES THAT MIGHT HELP NARROW IT DOWN
-
The uncovered ranges reported under TIA are not random. They are always the same
two files, and always the same line ranges:- UpdateProductAction line 94 — a
match ($segmento)expression whose two arms
are each exercised by a different test file.
Covered by 2 test files. - AbstractProdutosTable lines 306..348 — a
match ($result)with four arms plus
two public methods.
Covered by 3 test files (one per concrete subclass, plus a permissions test).
Both regions happen to be covered by more than one test file, and both are
matchexpressions whose arms are split across those files. We could not turn
that into a conclusion, though: plenty of other classes in this suite are also
touched by many test files (SegmentoEnum by 40) and stayed at 100.0% under TIA.
So multi-file coverage alone is not sufficient to trigger it — reporting the
observation as-is rather than a diagnosis. - UpdateProductAction line 94 — a
-
The tests covering those regions were not affected by the change that warmed the
cache, so they were replayed rather than re-executed. -
Relevant-looking internals: Pest\Plugins\Tia has
piggybackCoverage,
KEY_COVERAGE_CACHE ('coverage.bin.gz') and amergePerTestFiles()call that
merges the collector's per-test files with the recorder's. If the merge keeps
only one contributor per file/line, a region covered by several test files would
lose the contributions of the replayed ones — which matches what we observe. -
Cache location in our setup: ~/.pest/tia/-/
coverage.bin.gz (1.0 MB)
graph.json (358 KB) -
No custom coverage config: no --coverage-filter, no --path-coverage, plain
phpunit.xml source include of app/. -
--tiaon its own (no coverage) is correct: it replays properly and the suite
passes in 0.86s vs 61s. Only the coverage numbers are affected.
IMPACT / CURRENT WORKAROUND
We can't use --coverage --tia as a gate. We split the two uses in composer
scripts, wiping the TIA cache before any coverage run:
"test:tia": "pest --tia"
"test:coverage": wipe ~/.pest/tia, then "pest --coverage --min=100"
That restores a trustworthy number, at the cost of losing TIA's speed for any run
that also measures coverage.
- Lingua principale
- Nessun dato sulla lingua
- Stelle
- 56
- Fork
- 180
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di pestphp/docs
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
-
Add light-mode Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 30/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 45/100
-
documentation versioning Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
Tutte le issue di pestphp/docs
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
jaegertracing/jaeger-ui#4506 ·
-
[indexer] [QA] Add a focused test for the new NonRetryableError / assertSocketAlive() behavior. Apertabot:ai-assisted component:indexer QA-roadmap status:untriaged
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
midnightntwrk/midnight-indexer#1557 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
Luminous-Dynamics/mycelix#3069 ·
-
bug ci good first issue needs testing
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
-
agent/quality hive/hosted-available-lke648397-260827-5n31 quality testing
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100