ci: Coverage job fails on a random package's test:coverage task and hides the failing test
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 35/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- github-actions, typescript
Direzione di ricerca
Start with .github/workflows/pr.yml lines 76 and 105 and inspect the Coverage job output settings. Reproduce coverage failures under CPU load in ai-harness and ai-react, using the listed vitest and Nx commands; use the failing test output to identify timing dependent tests. Done when failed task output identifies the test and the intermittent failures are fixed, with parallelism capped only if failures persist.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
TanStack AI version
main (8731d75)
Framework/Library version
CI only (.github/workflows/pr.yml, Coverage job)
Describe the bug and the steps to reproduce it
The PR Coverage job fails at random on PRs that change @tanstack/ai. No coverage metric drops. One package's test:coverage task fails, and it is a different package each time. A re-run passes.
Seen so far:
| Run | PR | Failed task |
|---|---|---|
| 40271697755 | #1894 | @tanstack/ai-harness:test:coverage (passed on re-run) |
| 40270604983 | #1887 | @tanstack/ai-harness:test:coverage |
| 40277024581 | #1894 | @tanstack/ai-react:test:coverage |
The same tests pass locally:
packages/ai-harness:vitest run, 2469 passed.packages/ai-react:vitest runandnx run @tanstack/ai-react:test:coverage --skip-nx-cache, 251 passed.
Likely cause: a change to @tanstack/ai makes the job measure about 76 packages, twice (PR head and merge-base). nx run-many runs them in parallel on one runner. Tests that depend on timing then time out under the load.
Second problem: the job log does not show the output of the failed task. pr.yml:76 and pr.yml:105 run nx run-many --outputStyle=static, and the log only has the Nx summary line ✖ @tanstack/ai-harness:test:coverage. So nobody can name the flaky test from CI.
Fix
- Make the Coverage job print the full output of a failed task, so the failing test name and error are in the job log.
- Find the timing-sensitive tests in
ai-harnessandai-react. To do this, run their coverage under CPU load locally, or use the log from step 1. Then fix them: remove real-time waits, use fake timers, or wait for events instead of a fixed time. - If step 2 does not remove all failures, cap the parallel tasks for the coverage step (
--parallel=N). Do not add retries that hide failures.
Your Minimal, Reproducible Example - (Sandbox Highly Recommended)
CI runs listed above. No local reproduction yet.
Do you intend to try to help solve this bug with your own PR?
Yes, I am also opening a PR to fix this issue.
- Lingua principale
- TypeScript
- Stelle
- 3.2k
- Fork
- 372
- Merge medio
- 1g 5h
- PR unite (30g)
- 233
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di TanStack/ai
-
openai-base: a Responses max_output_tokens cutoff ends as RUN_ERROR without usage instead of finishReason 'length'Forse già presa @tombeckenham l’ha presa 1 giorno fa. Apertahas-pr waiting-on: maintainer
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
TanStack/ai#1916 · 1 commento · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 1-2 giorni Idoneità per principianti 58/100
I maintainer di solito rispondono entro 1 giorno
-
`ChatEngine.handleTextMessageContentEvent` is O(n²) in answer length: `startsWith(before)` copies the whole accumulated text on every chunkForse già presa @tombeckenham l’ha presa oggi. Apertawaiting-on: maintainer
TanStack/ai#1951 · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
-
openai-base: streamed `apply_patch` calls reach the app with an empty `diff`Forse già presa @tombeckenham l’ha presa oggi. Apertahas-pr waiting-on: maintainer
TanStack/ai#1947 · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
-
needsApproval client tools can't be approved without a server (custom connection adapters)Forse già presa @jherr l’ha presa oggi. Apertawaiting-on: maintainer
TanStack/ai#1945 · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
area:desktop area:ui bug platform:macos
Difficoltà 2/5 1-3 ore Idoneità per principianti 67/100
anthropics/claude-code#106183 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
-
Difficoltà 1/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 1 giorno
-
reproduced
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
anomalyco/opencode#60558 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno