Report and enforce coverage non-regression for src Python changes

Aberta
  • Algo errado?
#4,786 2 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
50/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
github-actions, python
Domínio
ci-cd, documentation, testing-qa

Direção de pesquisa

Read the existing pytest CI configuration first, then inspect the coverage settings and the documented guidance in AGENTS.md and CONTRIBUTING.md. Trace how pull requests identify changes under src/**/*.py and how main pushes are handled. Done means comparable base-versus-PR reports, an unrounded non-regression check with human-controlled exceptions, retained artifacts, and updated documentation without removing the cross-platform test matrix.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

triage-can-wait

Problem

Spec Kit requires tests for behavioral changes and regression evidence for bug fixes, but CI currently runs pytest without collecting or comparing code coverage. pytest-cov and coverage configuration already exist. Without a comparable base-versus-PR measurement, coverage regressions can pass unnoticed.

Proposed policy

For pull requests that change src/**/*.py, Python code coverage must not decrease relative to the base revision. A decrease requires human intervention: restore coverage before merging, or have a maintainer explicitly review and document an exceptional decision. Agents must not waive a decrease on their own.

Pull requests that do not change src/**/*.py are exempt from the coverage comparison. Existing behavioral and regression test requirements continue to apply to executable changes elsewhere.

Implementation

  • Add one canonical coverage CI job with a fixed OS and Python version, while retaining the existing cross-platform test matrix.
  • On relevant PRs, run coverage on the base and PR revisions under comparable conditions. Report both coverage values and their covered/coverable line counts in the Actions summary; retain the reports as artifacts.
  • Compare unrounded coverage ratios rather than displayed percentages, and fail the check on a decrease. Make any exception path explicitly human-controlled and documented; do not let an agent silently skip or override the check.
  • On pushes to main, publish the coverage report without attempting a PR comparison.
  • Document the policy in AGENTS.md and the measurement and intervention procedure in CONTRIBUTING.md.

Acceptance criteria

  • In-scope PRs receive a base-versus-PR coverage report and a passing or failing non-regression check.
  • PRs without changes to src/**/*.py do not need a coverage comparison and do not fail due to a skipped comparison.
  • Coverage decreases cannot be silently accepted or waived by an agent; human intervention is required.
  • The existing cross-platform tests and positive/negative and bug-regression test requirements remain in place.

AI disclosure

Drafted with GitHub Copilot (GPT-6 Sol) in an interactive, human-supervised conversation; the requester directed the scope and revised the policy. Reasoning effort was not specified. AI assistance was limited to drafting this issue; no code changes were made.

Linguagem predominante
Python
Estrelas
149k
Forks
13.4k
Merge médio
1d 5h
PRs com merge (30d)
223

Preparar o ambiente

Abrir no Codespaces

Inicia o contêiner de desenvolvimento do projeto no navegador, com a sua própria conta do GitHub.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de github/spec-kit

Todas as issues de github/spec-kit

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.