unit-test: with --coverage, a setup file's hooks reach only the first spec file of each worker

Abierto
#34,137 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
72/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
angular, typescript

Línea de trabajo

Start with @angular/build/src/builders/unit-test/runners/vitest/build-options.js and plugins.js, tracing how setup entry points are created and served when coverage is enabled. Run the linked reproduction with and without coverage, then verify that the builder's setup hooks run for every spec file in the coverage case while the existing runner-config workaround remains understood.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

area: @angular/build gemini-triaged
Command

test

Is this a regression?
  • Yes, this behavior used to work in the previous version
The previous version in which this bug was not present was

Not known to be. It has been present for as long as the vitest runner has served coverage stubs.

Description

@angular/build:unit-test with the vitest runner turns every test entry point into a one-line stub
when coverage is enabled, so that the real file can be excluded from the coverage report. Setup
files declared in the builder's setupFiles option are among those entry points.

Vitest re-imports each setup file before each spec file and invalidates the setup module first, so
that the module body runs again and the hooks it registers are registered onto the suite of the file
about to run. Invalidation does not cascade through the stub's import: the stub is invalidated and
re-run, and the module behind it — the one that holds the setup code — is never invalidated and so
is evaluated only once.

The effect is that a setup file's beforeEach/afterEach runs for the first spec file each worker
picks up and for no other file
, whenever coverage is on. With coverage off the same setup file
behaves as documented.

Nothing reports this. Every test still passes; the hooks simply do not run.

Expected behaviour

A setup file's hooks are registered for every spec file, with or without --coverage — that is what
vitest's per-file setup invalidation exists to guarantee, and what the builder's setupFiles option
is for.

Minimal Reproduction

https://github.com/BigMichi1/ng-coverage-setup-repro

A workspace with the builder's setupFiles pointing at a setup file that counts its own module
evaluations and the spec files its afterEach runs for, three trivial spec files, and a
runnerConfig that pins the run to one worker so all three files share one environment:

npm install

rm -f hook-log.txt && npx ng test --no-watch --coverage && cat hook-log.txt
rm -f hook-log.txt && npx ng test --no-watch              && cat hook-log.txt

The setup file is roughly:

import { appendFileSync } from 'node:fs';
import { join } from 'node:path';
import { afterEach, expect } from 'vitest';

const logFile = join(process.cwd(), 'hook-log.txt');
const g = globalThis as typeof globalThis & { seen?: Set<string> };
const seen = (g.seen ??= new Set<string>());

appendFileSync(logFile, 'evaluation\n');

afterEach(() => {
  const file = (expect.getState().testPath ?? '?').split('/').pop() ?? '?';
  if (!seen.has(file)) {
    seen.add(file);
    appendFileSync(logFile, `hook ${file}\n`);
  }
});
Exception or Error
None. Every test passes; the hooks simply do not run.
Your Environment
Angular CLI       : 22.1.8
Angular           : 22.1.6
Node.js           : 26.8.2
Package Manager   : bun 1.4.2
Operating System  : linux x64

@angular/build            22.1.8
@angular/cli              22.1.8
@angular/common           22.1.6
@angular/compiler         22.1.6
@angular/compiler-cli     22.1.6
@angular/core             22.1.6
@angular/platform-browser 22.1.6
rxjs                      7.8.2
typescript                6.0.3
vitest                    4.1.11
@vitest/coverage-v8       4.1.11
jsdom                     30.0.1
Anything else relevant?

Measurement

Both runs report Test Files 3 passed (3). Stable over three runs each way; which spec file the
hook reaches is whichever the single worker runs first.

With --coverage — the module is evaluated once and the hook runs for 1 of 3 spec files:

evaluation 1
hook three.spec.ts

Without coverage — 3 evaluations, and the hook runs for 3 of 3:

evaluation 1
hook three.spec.ts
evaluation 2
hook two.spec.ts
evaluation 3
hook one.spec.ts

A second setup file declared in the runner config's own test.setupFiles instead of the builder's
setupFiles option is not an entry point, is therefore served as itself, and does reach all three
files in the same coverage run — which is the workaround, and also the evidence that the stub is the
cause rather than anything about coverage instrumentation:

evaluation 1               <- the builder's setup file, once
runner-evaluation          <- the runner config's setup file
runner-hook three.spec.ts
hook three.spec.ts
runner-evaluation
runner-hook two.spec.ts
runner-evaluation
runner-hook one.spec.ts

The two mechanisms

@angular/build/src/builders/unit-test/runners/vitest/build-options.js adds the setup files to the
build's entry points:

if (options.setupFiles?.length) {
  const setupEntryPoints = getTestEntrypoints(options.setupFiles, {
    projectSourceRoot,
    workspaceRoot,
    removeTestExtension: false,
    prefix: 'setup',
  });
  for (const [entryPoint, setupFile] of setupEntryPoints) {
    entryPoints.set(entryPoint, setupFile);
  }
}

@angular/build/src/builders/unit-test/runners/vitest/plugins.js serves every entry point as a stub
when coverage is enabled:

if (vitestConfig?.coverage?.enabled) {
  // To support coverage exclusion of the actual test file, the virtual
  // test entry point only references the built and bundled intermediate file.
  // If vitest supported an "excludeOnlyAfterRemap" option, this could be removed completely.
  return {
    code: `import "./${outputPath}";`,
  };
}

@vitest/runner re-runs the setup files for each spec file:

clearCollectorContext(file, runner);
const setupFiles = toArray(config.setupFiles);
if (setupFiles.length) {
  await runSetupFiles(config, setupFiles, runner);
}

and vitest's TestRunner.importFile invalidates the setup module so that the re-import re-evaluates
it:

importFile(filepath, source) {
  if (source === "setup") {
    const moduleNode = this.workerState.evaluatedModules.getModuleById(filepath);
    if (moduleNode) this.workerState.evaluatedModules.invalidateModule(moduleNode);
  }
  ...
}

The module vitest invalidates is the stub. Its import is a separate module node, is not invalidated,
and is not evaluated again.

Suggested direction

The comment on the stub says it exists so the test file itself can be excluded from the coverage
report. Setup files are excluded from coverage by ordinary coverageExclude patterns anyway, so the
simplest fix looks like not applying the stub to entry points whose source is a setup file — the
prefix: 'setup' entry points above are already distinguishable at the point where the stub is
produced. Alternatively the setup files could be kept out of entryPoints entirely and served as
themselves, which is how a setup file declared in the runner config already behaves.

What it costs a consumer

  • Any per-test cleanup written in a setup file — restoring fake timers, clearing Web Storage,
    resetting a global — silently stops happening after the first spec file of each worker, but only
    when coverage is on. Since coverage is typically on in CI and off locally, the same suite is green
    on a developer machine and red on CI, and the failure surfaces in an innocent spec file that the
    poisoned environment was handed to, not in the one that caused it.
  • Nothing warns. The setup file is imported, the run is green, and the hooks are simply registered
    onto one file's suite.
  • Diagnosing it costs a lot: it took two separate debugging passes here, and the eventual
    explanation needed reading the builder's and vitest's built output, because the behaviour
    contradicts vitest's documented setup-file semantics.
  • The workaround — moving every per-file hook out of the builder's setupFiles and into a setup
    file declared in the runner config — is a file the consumer maintains to undo a builder rewrite,
    and it is invisible to anyone who has not read this.
Lenguaje dominante
TypeScript
Estrellas
27k
Forks
11.8k
Merge medio
16 h 35 min
PR fusionados (30 d)
176

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de angular/angular-cli

Todos los issues de angular/angular-cli

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.