Report and enforce coverage non-regression for src Python changes

未关闭
  • 哪里不对？
#4,786 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
50/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
github-actions, python
领域
ci-cd, documentation, testing-qa

调研方向

Read the existing pytest CI configuration first, then inspect the coverage settings and the documented guidance in AGENTS.md and CONTRIBUTING.md. Trace how pull requests identify changes under src/**/*.py and how main pushes are handled. Done means comparable base-versus-PR reports, an unrounded non-regression check with human-controlled exceptions, retained artifacts, and updated documentation without removing the cross-platform test matrix.

由索引模型根据 Issue 内容生成。

描述

triage-can-wait

Problem

Spec Kit requires tests for behavioral changes and regression evidence for bug fixes, but CI currently runs pytest without collecting or comparing code coverage. pytest-cov and coverage configuration already exist. Without a comparable base-versus-PR measurement, coverage regressions can pass unnoticed.

Proposed policy

For pull requests that change src/**/*.py, Python code coverage must not decrease relative to the base revision. A decrease requires human intervention: restore coverage before merging, or have a maintainer explicitly review and document an exceptional decision. Agents must not waive a decrease on their own.

Pull requests that do not change src/**/*.py are exempt from the coverage comparison. Existing behavioral and regression test requirements continue to apply to executable changes elsewhere.

Implementation

  • Add one canonical coverage CI job with a fixed OS and Python version, while retaining the existing cross-platform test matrix.
  • On relevant PRs, run coverage on the base and PR revisions under comparable conditions. Report both coverage values and their covered/coverable line counts in the Actions summary; retain the reports as artifacts.
  • Compare unrounded coverage ratios rather than displayed percentages, and fail the check on a decrease. Make any exception path explicitly human-controlled and documented; do not let an agent silently skip or override the check.
  • On pushes to main, publish the coverage report without attempting a PR comparison.
  • Document the policy in AGENTS.md and the measurement and intervention procedure in CONTRIBUTING.md.

Acceptance criteria

  • In-scope PRs receive a base-versus-PR coverage report and a passing or failing non-regression check.
  • PRs without changes to src/**/*.py do not need a coverage comparison and do not fail due to a skipped comparison.
  • Coverage decreases cannot be silently accepted or waived by an agent; human intervention is required.
  • The existing cross-platform tests and positive/negative and bug-regression test requirements remain in place.

AI disclosure

Drafted with GitHub Copilot (GPT-6 Sol) in an interactive, human-supervised conversation; the requester directed the scope and revised the policy. Reasoning effort was not specified. AI assistance was limited to drafting this issue; no code changes were made.

主要语言
Python
星标
149k
派生
13.4k
平均合并
1 天 5 小时
30 天内合并 PR
223

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

github/spec-kit 的其他 Issue

查看 github/spec-kit 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。