Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

test.fail() in after hooks relabels an already-executed test and can make a failing run exit 0

Abierto
  • ¿Algo no está bien?
#43,738 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
72/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
playwright, typescript
Área
testing

Línea de trabajo

Start by reading _modifier and the existing skip handling changed in #43713, then trace how expectedStatus affects test results after after hooks and fixture teardown. Reproduce the issue with the provided test.afterEach() example and a failing assertion. Done means a late test.fail() is reported as a failure without changing the completed test’s expected status, while a declared test.fail('title', fn) still behaves as expected.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

test.fail() in after hooks relabels an already-executed test and can make a failing run exit 0

Found while qualifying Playwright's modifier handling for 9l (9lives), as a sibling of #43676 (fixed by #43713). 9l assesses Playwright suites and consumes their execution results, so a modifier that changes a completed test's expectedStatus after execution directly corrupts execution evidence.

Reproduction (verified on 1.61.1, 1.64.1 and 1.65.0-alpha-2026-10-28)
import {test, expect} from '@playwright/test';

test.afterEach(() => {
  test.fail();
});

test('genuinely failing', () => {
  expect(1).toBe(2);
});

$ npx playwright test a.spec.ts

  ✘  1 a.spec.ts:7:5 › genuinely failing (3ms)

  1 passed (0.3s)

$ echo $?
0

The test visibly fails, yet the summary says 1 passed and the process exit code is 0.

What happens
  • test.fail() called in an afterEach (also afterAll, also an auto-fixture teardown) runs after the test body finished.
  • _modifier sets testInfo.expectedStatus = 'failed' directly — no after-hook guard, and unlike skip/fixme it does not throw.
  • A failing test: status = failed now equals expectedStatus = failed → outcome expected → green exit code. A real failure is indistinguishable from a declared expected failure.
  • A passing test: relabeled unexpected → the run fails retroactively.
Scenario Result
failing body + test.fail() in afterEach expected, exit 0
failing body + test.fail() in afterAll expected, exit 0
failing body + test.fail() in fixture teardown expected, exit 0
same with --retries=1 exit 0
passing body + test.fail() in afterEach unexpected, exit 1
control: declared test.fail('title', fn) that fails expected, exit 0 (correct)
Why #43713 does not cover it

#43713 sets testInfo._allowSkips = false before after hooks, which only affects the TestSkipError path (skip/fixme). I verified this by applying the same one-line patch locally: test.skip(true, ...) in afterEach is then correctly rejected (exit 1), while test.fail() in afterEach still relabels and exits 0.

The fail branch of _modifier needs the same treatment: after the test has finished executing, it must not mutate expectedStatus — it should be rejected (reported as a failure), as #43713 does for skips.

Impact for downstream tooling

Any consumer of Playwright results (reporters, CI gates, evidence tools) reads expected as "intentional failure". With one stray test.fail() in a shared after hook — or any code reaching it through a fixture teardown — genuine failures pass CI silently. The relabeled annotation is also attached to the finished test, leaving no trace that the expectation was set after execution.

Lenguaje dominante
TypeScript
Estrellas
98.5k
Forks
6.7k
Merge medio
16 h 10 min
PR fusionados (30 d)
305

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de microsoft/playwright

Todos los issues de microsoft/playwright

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.