Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Full-suite test time: ReDoS/contract sweeps are 32% of the suite; gate them on PRs, add a slow-test budget, trim the top 10

未关闭
  • 哪里不对？
#5,166 6 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
10/100
Issue 类型
重构
描述清晰度
基本清楚
活跃度
活跃
技术栈
github-actions, python
领域
ci-cd, testing-qa

调研方向

Start with tests/tools/pr_gates.py for the existing ratchet pattern, smoke-test.yml for the full-suite parts, and the sweep tests under tests/extraction/. Parts A and B change CI gating and need maintainer design decisions first. Part C is the most separable: profile one slow test such as test_galaxyscope and keep its assertions. Done means the test passes with a shorter wall time and no weakened check.

由索引模型根据 Issue 内容生成。

描述

ci-cd github-actions performance scope:infra testing

Context (2026-10-14 CI audit, measured from origin/main .test_durations refreshed by #5151): the full suite (smoke-test.yml parts, required check full-suite) is the largest PR-path CI cost, 27.7% of runner time 21:00–24:00Z (squid-bot dashboard / #5164 table). Its time is concentrated:

Tests Serial time Share
Full suite (minus test_evidence_components.py, now Ground Truth's per #5163) 17,463 56.9 min 100%
Extraction ReDoS / contract sweeps (tests/extraction/**, *_redos_immune, *_redos_sweep, contract rules) 1,386 18.0 min 32%
Tests over 10 s 32 10.7 min 19%
Tests under 0.1 s 15,845 ~1.2 min 2%

Heaviest sweep functions (summed over params): test_ownership_contract_rules_are_redos_immune 3.3 min, test_high_risk_execution_contract_rules_are_redos_immune 3.0, globals 1.8, branch 1.6, import 1.5, state_mutation 0.9, safety 0.6, cleanup_contract_redos_sweep 0.6.

Already handled elsewhere (don't redo): evidence tests and the duplicate test_evidence_components run (#5162/#5163), part balancing and durations weight (#5133/#5162), Ground Truth (#5164).

Rule for every part: coverage never drops. Main pushes, nightly and dispatch keep running every test. Only when a test runs on a PR, or how fast, may change. When in doubt, run.

A. Run the extraction ReDoS/contract sweeps on PRs only when their inputs change
  • Mark the sweep tests (one pytest marker, e.g. redos_sweep) and define their input set: the extraction rule/regex files and contract specs they read, the sweep harness, and pyproject/requirements. In doubt, include the file.
  • On pull_request, the full-suite parts deselect the marker when the PR changes no input. The decision comes from the PR's changed files, computed the same way as Det Sweep's (#4825), and the required check still always reports. Push to main, nightly and dispatch run them always.
  • Tests for the classifier in both directions. PR body: the share of the last ~50 merged PRs that would have skipped, and the minutes saved.
B. Test-time ratchet: the suite must not get slower over time (owner, 2026-10-14)

Register it as a Ratchet in tests/tools/pr_gates.py (check command, update command, committed baseline), like the existing ratchets. CI runner timings vary ±20–30%, so it has two levels and only the robust one gates PRs:

  • Per-test gate (hard, every PR): a test whose CI duration exceeds 10 s fails unless it carries an explicit slow marker with a reason, or is listed in tests/test_time_baseline.json. Seed the baseline with today's 32 tests over 10 s and their times. A baseline test fails if it exceeds 2× its recorded time (wide enough for runner noise). The baseline only shrinks or gets faster: the update command lowers entries and drops ones under 10 s, and never raises one.
  • Whole-suite budget (hard on nightly main, report-only on PRs): total serial test seconds of the full suite, measured by the nightly run on main from its own junit/--durations (not the contended durations cache, see #5162). Nightly fails, and files or refreshes one issue, when the total exceeds the committed budget by >10%. When the suite gets faster, the update command lowers the budget to the new level (shrink-only). PRs print their suite total and the delta against the budget in the job summary, without gating.
  • Tests for the ratchet logic (threshold, 2× rule, shrink-only update, budget delta). Document it in .claude/rules/ (test-writing) and docs/ci.md.
C. Speed up the top 10 by hand (4.9 min, 8.6%), keeping the same assertions

Profile each and fix the cause (fixture scope, repeated scans, sampling size), or split it so -n auto can parallelise it. Don't weaken the check:

  • cultural_gauntlet::test_the_sampled_plan_passes_its_ratchet 60.8 s
  • test_unicode_gauntlet::test_the_oracle_is_sound_on_the_mainframe_seed 46.4 s
  • test_scan_path_independence_4058::test_the_same_tree_scans_the_same_under_any_parent[tmp] 42.2 s
  • test_cleanup_contract_2888::test_cleanup_contract_redos_sweep 33.2 s (or covered by A)
  • cobol_mainframe/test_tool_regex_redos::test_no_new_offenders 21.9 s
  • test_immutability_locks_contract_2772::test_immutability_locks_redos_sweep 21.0 s (or covered by A)
  • test_galaxyscope (3 tests: typosquat 19.5 s, z-score backfill 14.8 s, sarif purge 14.1 s): likely a shared expensive fixture
  • test_inline_extraction::test_a_spawned_pool_is_the_forked_pools_output 18.1 s

Acceptance: before/after full-suite part wall times on a typical ready PR (code-only and docs-only), and the runner-minutes per run, in each PR body. Test count on main and nightly unchanged.

主要语言
Python
星标
57
派生
11
平均合并
1 小时 20 分钟
30 天内合并 PR
985

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

squid-protocol/gitgalaxy 的其他 Issue

查看 squid-protocol/gitgalaxy 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。