TIA: a suite absent from the graph can never be recorded while a graph exists — replay only refreshes tests the graph already knows
I maintainer di solito rispondono entro 5 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 55/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Tranquilla
- Stack tecnologico
- php
- Ambito
- testing-qa
Direzione di ricerca
Start in src/Plugins/Tia.php around the record/replay gate at lines 905-910, then trace enterReplayMode() near lines 1054 and 1076. Reproduce with two PHPUnit configurations and inspect graph.json; done means an initially unknown test suite has its edges and results persisted during replay instead of remaining uncached.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
Once a TIA graph exists, a test suite (or any set of test files) the graph does not know about can never be recorded into it — no matter how many times it runs with --tia. Replay mode is entered solely because a graph exists, and inside replay the recorder is only activated for tests the graph already knows. The unknown tests execute (reported as uncached), but neither their edges nor their results are persisted — so every subsequent run pays the full uncached cost again, forever.
This bites hardest in the common Laravel setup where two PHPUnit configs share one project: the default phpunit.xml (Unit/Feature) and a second config for browser tests (phpunit.browser.xml, different testsuite directory). Whichever suite records first wins the graph; the other one is permanently locked out of it.
Reproduction
Any project with Pest v5, a coverage driver (pcov), and git. No browser plugin needed — the point is a test directory outside the recorded graph.
# 1. Two configs, two test directories
# phpunit.xml -> <testsuite name="Unit"><directory>tests/Unit</directory></testsuite>
# phpunit.browser.xml -> same file, but <testsuite name="Browser"><directory>tests/Browser</directory></testsuite>
mkdir -p tests/Browser
cp phpunit.xml phpunit.browser.xml # then edit the testsuite as above
# 2. One trivial test in each directory
# tests/Unit/ExampleTest.php -> test('unit', fn () => expect(true)->toBeTrue());
# tests/Browser/ExampleTest.php -> test('browser', fn () => expect(true)->toBeTrue());
# 3. Commit everything so the working tree is clean (important: $changed must be [])
git add -A && git commit -m repro
# 4. Record the graph with the default config
vendor/bin/pest --tia
# -> "Experimental TIA mode enabled / fresh graph."
# -> .pest/tia/graph.json (or ~/.pest default) now has edges for tests/Unit only
# 5. Run the second suite — any number of times
vendor/bin/pest -c phpunit.browser.xml --tia
# -> banner says "Experimental TIA mode enabled." (replay — no "fresh graph")
# -> the Browser test RUNS, reported as "1 uncached"
# 6. Check the graph
grep -c 'tests/Browser' .pest/tia/graph.json
# -> 0. Repeat step 5 forever: still 0. The suite is never cached.
Observed at scale (real project)
Laravel app, Pest v5.1.1, pcov, 8.3k Feature tests + 446 browser tests (Pest browser plugin), both configs sharing the default TIA directory:
- Feature leg records first (445 s,
fresh graph), writesgraph.json. - Browser leg then runs
--tia: enters replay, executes all 446 tests asuncachedin 950 s, and adds zero entries to the graph — no edges, no results (baselines[branch].resultsgained nothing for those files). - Ran it twice to confirm (950 s and 922 s):
grep -c 'tests/Browser' graph.json→ 0 both times. - Net effect: the browser suite pays full price on every single run while believing it is cached.
Mechanism (v5.1.1 sources)
src/Plugins/Tia.php:905-910— the record/replay gate is graph existence, nothing else:
if ($graph instanceof Graph) {
return $this->enterReplayMode($graph, $projectRoot, $arguments);
}
return $this->enterRecordMode($arguments);
- Inside
enterReplayMode(), the affected set is derived from the graph's own edges (Tia.php:1054):
$affectedFromChanges = $changed === [] ? [] : $graph->testFilesOnDisk($graph->affected($changed));
A test file the graph has never seen cannot be produced by $graph->affected() — so it is never part of $affected, regardless of working-tree state. With a clean tree it is even simpler: $changed === [] → $affected === [].
- Recording during replay is gated on that same set (
Tia.php:1076):
$canRefreshReplayEdges = $affected !== [] && $coverageAvailable;
With $affected === [], RECORDING_GLOBAL is never set, workers never activate the recorder, and snapshotTestResults() has nothing marked to persist for the unknown files. The unknown tests still execute (they are not replayable), but the run is write-only-for-known-tests: edges and results for the unknown suite are dropped on the floor.
So the system has exactly two stable states — "no graph: record everything" and "graph: refresh only what I already know" — and there is no path from the second state to "learn something new", except deleting the graph (--fresh), which throws away the first suite's cache to record the second's.
Expected
Either of these would fix it:
- Treat test files unknown to the graph as affected. During replay, scan the selected testsuite for files absent from
graph.jsonand include them in$affected(recorder active for them). Unknown tests already run anyway — the only change is that their edges/results get persisted instead of discarded. - Or give each resolved configuration its own graph. This is the same input-set gap as #1858 (the fingerprint ignores
-c/--configuration): two configs with disjoint testsuites currently share one graph and one fingerprint, so they can neither coexist nor invalidate each other.
Workaround
Per-suite graph directories, switched by an env var:
// tests/Pest.php
pest()->tia()->locally()->directory(env('PEST_TIA_DIR', '.pest/tia'));
# browser suite runs
PEST_TIA_DIR=.pest/tia-browser vendor/bin/pest -c phpunit.browser.xml --tia
Each suite then records fresh once and replays afterwards. Works, but every consumer script must remember to pass the env var — forgetting it silently re-creates the bug.
Environment
- pestphp/pest v5.1.1 (also inspected
5.xup to 2026-08-18 —9b36f44b,2e589182don't touch this logic) - PHP 8.4, pcov (xdebug also present), Linux, git 2.43
- Related: #1858 (fingerprint ignores the config actually in use — same root cause family: the graph/fingerprint don't account for which configuration produced them)
- Lingua principale
- PHP
- Stelle
- 11.8k
- Fork
- 559
- Merge medio
- 5g 23h
- PR unite (30g)
- 24
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di pestphp/pest
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: --mutate hides unreadable files, so mutants behind a failed read can never be killedForse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
I maintainer di solito rispondono entro 5 giorni
-
[Bug]: [mutate] Windows: @pest-mutate-ignore comments land on line 1 in files with LF line endingsForse già presa @webdevsamran l’ha presa 57 giorni fa. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 5 giorni
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 80/100
I maintainer di solito rispondono entro 5 giorni
-
[pest-plugin-browser] Page::waitForLoadState() never sends its message — waitForEvent() is a silent no-opForse già presa @asllanmaciel l’ha presa 70 giorni fa. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
pestphp/pest#1892 · 2 commenti ·
I maintainer di solito rispondono entro 5 giorni
Tutte le issue di pestphp/pest
Issue simili
-
Group 2 - Data Storage | Tripal DBX | Chado Group 5 - Search | ElasticSearch | Views Priority - Medium
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
-
nginx update required due to NGINX2RootForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 65/100
mailcow/mailcow-dockerized#7668 · 5 commenti · 2 reazioni ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
lara-zeus/spatie-translatable#80 ·
I maintainer di solito rispondono entro 1 giorno
-
small
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
uiowa/uiowa#10296 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
asyncapi/modelina#2709 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno