Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

test.fail() in after hooks relabels an already-executed test and can make a failing run exit 0

クローズ
  • 何か問題がありますか？
#43,738 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
72/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
playwright, typescript
領域
testing

調査の方向性

Start by reading _modifier and the existing skip handling changed in #43713, then trace how expectedStatus affects test results after after hooks and fixture teardown. Reproduce the issue with the provided test.afterEach() example and a failing assertion. Done means a late test.fail() is reported as a failure without changing the completed test’s expected status, while a declared test.fail('title', fn) still behaves as expected.

索引モデルが issue の本文から書いたものです。

説明

test.fail() in after hooks relabels an already-executed test and can make a failing run exit 0

Found while qualifying Playwright's modifier handling for 9l (9lives), as a sibling of #43676 (fixed by #43713). 9l assesses Playwright suites and consumes their execution results, so a modifier that changes a completed test's expectedStatus after execution directly corrupts execution evidence.

Reproduction (verified on 1.61.1, 1.64.1 and 1.65.0-alpha-2026-10-28)
import {test, expect} from '@playwright/test';

test.afterEach(() => {
  test.fail();
});

test('genuinely failing', () => {
  expect(1).toBe(2);
});

$ npx playwright test a.spec.ts

  ✘  1 a.spec.ts:7:5 › genuinely failing (3ms)

  1 passed (0.3s)

$ echo $?
0

The test visibly fails, yet the summary says 1 passed and the process exit code is 0.

What happens
  • test.fail() called in an afterEach (also afterAll, also an auto-fixture teardown) runs after the test body finished.
  • _modifier sets testInfo.expectedStatus = 'failed' directly — no after-hook guard, and unlike skip/fixme it does not throw.
  • A failing test: status = failed now equals expectedStatus = failed → outcome expected → green exit code. A real failure is indistinguishable from a declared expected failure.
  • A passing test: relabeled unexpected → the run fails retroactively.
Scenario Result
failing body + test.fail() in afterEach expected, exit 0
failing body + test.fail() in afterAll expected, exit 0
failing body + test.fail() in fixture teardown expected, exit 0
same with --retries=1 exit 0
passing body + test.fail() in afterEach unexpected, exit 1
control: declared test.fail('title', fn) that fails expected, exit 0 (correct)
Why #43713 does not cover it

#43713 sets testInfo._allowSkips = false before after hooks, which only affects the TestSkipError path (skip/fixme). I verified this by applying the same one-line patch locally: test.skip(true, ...) in afterEach is then correctly rejected (exit 1), while test.fail() in afterEach still relabels and exits 0.

The fail branch of _modifier needs the same treatment: after the test has finished executing, it must not mutate expectedStatus — it should be rejected (reported as a failure), as #43713 does for skips.

Impact for downstream tooling

Any consumer of Playwright results (reporters, CI gates, evidence tools) reads expected as "intentional failure". With one stray test.fail() in a shared after hook — or any code reaching it through a fixture teardown — genuine failures pass CI silently. The relabeled annotation is also attached to the finished test, leaving no trace that the expectation was set after execution.

主要言語
TypeScript
スター
98.5k
フォーク
6.7k
平均マージ
17時間 5分
マージ済み PR（30日）
326

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

microsoft/playwright のほかの issue

microsoft/playwright の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。