Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Full-suite test time: ReDoS/contract sweeps are 32% of the suite; gate them on PRs, add a slow-test budget, trim the top 10

オープン
  • 何か問題がありますか？
#5,166 コメント 6 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
10/100
issue の種類
リファクタリング
明瞭さ
おおむね明確
活発さ
活発
技術スタック
github-actions, python
領域
ci-cd, testing-qa

調査の方向性

Start with tests/tools/pr_gates.py for the existing ratchet pattern, smoke-test.yml for the full-suite parts, and the sweep tests under tests/extraction/. Parts A and B change CI gating and need maintainer design decisions first. Part C is the most separable: profile one slow test such as test_galaxyscope and keep its assertions. Done means the test passes with a shorter wall time and no weakened check.

索引モデルが issue の本文から書いたものです。

説明

ci-cd github-actions performance scope:infra testing

Context (2026-10-14 CI audit, measured from origin/main .test_durations refreshed by #5151): the full suite (smoke-test.yml parts, required check full-suite) is the largest PR-path CI cost, 27.7% of runner time 21:00–24:00Z (squid-bot dashboard / #5164 table). Its time is concentrated:

Tests Serial time Share
Full suite (minus test_evidence_components.py, now Ground Truth's per #5163) 17,463 56.9 min 100%
Extraction ReDoS / contract sweeps (tests/extraction/**, *_redos_immune, *_redos_sweep, contract rules) 1,386 18.0 min 32%
Tests over 10 s 32 10.7 min 19%
Tests under 0.1 s 15,845 ~1.2 min 2%

Heaviest sweep functions (summed over params): test_ownership_contract_rules_are_redos_immune 3.3 min, test_high_risk_execution_contract_rules_are_redos_immune 3.0, globals 1.8, branch 1.6, import 1.5, state_mutation 0.9, safety 0.6, cleanup_contract_redos_sweep 0.6.

Already handled elsewhere (don't redo): evidence tests and the duplicate test_evidence_components run (#5162/#5163), part balancing and durations weight (#5133/#5162), Ground Truth (#5164).

Rule for every part: coverage never drops. Main pushes, nightly and dispatch keep running every test. Only when a test runs on a PR, or how fast, may change. When in doubt, run.

A. Run the extraction ReDoS/contract sweeps on PRs only when their inputs change
  • Mark the sweep tests (one pytest marker, e.g. redos_sweep) and define their input set: the extraction rule/regex files and contract specs they read, the sweep harness, and pyproject/requirements. In doubt, include the file.
  • On pull_request, the full-suite parts deselect the marker when the PR changes no input. The decision comes from the PR's changed files, computed the same way as Det Sweep's (#4825), and the required check still always reports. Push to main, nightly and dispatch run them always.
  • Tests for the classifier in both directions. PR body: the share of the last ~50 merged PRs that would have skipped, and the minutes saved.
B. Test-time ratchet: the suite must not get slower over time (owner, 2026-10-14)

Register it as a Ratchet in tests/tools/pr_gates.py (check command, update command, committed baseline), like the existing ratchets. CI runner timings vary ±20–30%, so it has two levels and only the robust one gates PRs:

  • Per-test gate (hard, every PR): a test whose CI duration exceeds 10 s fails unless it carries an explicit slow marker with a reason, or is listed in tests/test_time_baseline.json. Seed the baseline with today's 32 tests over 10 s and their times. A baseline test fails if it exceeds 2× its recorded time (wide enough for runner noise). The baseline only shrinks or gets faster: the update command lowers entries and drops ones under 10 s, and never raises one.
  • Whole-suite budget (hard on nightly main, report-only on PRs): total serial test seconds of the full suite, measured by the nightly run on main from its own junit/--durations (not the contended durations cache, see #5162). Nightly fails, and files or refreshes one issue, when the total exceeds the committed budget by >10%. When the suite gets faster, the update command lowers the budget to the new level (shrink-only). PRs print their suite total and the delta against the budget in the job summary, without gating.
  • Tests for the ratchet logic (threshold, 2× rule, shrink-only update, budget delta). Document it in .claude/rules/ (test-writing) and docs/ci.md.
C. Speed up the top 10 by hand (4.9 min, 8.6%), keeping the same assertions

Profile each and fix the cause (fixture scope, repeated scans, sampling size), or split it so -n auto can parallelise it. Don't weaken the check:

  • cultural_gauntlet::test_the_sampled_plan_passes_its_ratchet 60.8 s
  • test_unicode_gauntlet::test_the_oracle_is_sound_on_the_mainframe_seed 46.4 s
  • test_scan_path_independence_4058::test_the_same_tree_scans_the_same_under_any_parent[tmp] 42.2 s
  • test_cleanup_contract_2888::test_cleanup_contract_redos_sweep 33.2 s (or covered by A)
  • cobol_mainframe/test_tool_regex_redos::test_no_new_offenders 21.9 s
  • test_immutability_locks_contract_2772::test_immutability_locks_redos_sweep 21.0 s (or covered by A)
  • test_galaxyscope (3 tests: typosquat 19.5 s, z-score backfill 14.8 s, sarif purge 14.1 s): likely a shared expensive fixture
  • test_inline_extraction::test_a_spawned_pool_is_the_forked_pools_output 18.1 s

Acceptance: before/after full-suite part wall times on a typical ready PR (code-only and docs-only), and the runner-minutes per run, in each PR body. Test count on main and nightly unchanged.

主要言語
Python
スター
57
フォーク
11
平均マージ
1時間 8分
マージ済み PR（30日）
978

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

squid-protocol/gitgalaxy のほかの issue

squid-protocol/gitgalaxy の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。