Report and enforce coverage non-regression for src Python changes

Aperta
  • Qualcosa non va?
#4,786 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
50/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
github-actions, python
Ambito
ci-cd, documentation, testing-qa

Direzione di ricerca

Read the existing pytest CI configuration first, then inspect the coverage settings and the documented guidance in AGENTS.md and CONTRIBUTING.md. Trace how pull requests identify changes under src/**/*.py and how main pushes are handled. Done means comparable base-versus-PR reports, an unrounded non-regression check with human-controlled exceptions, retained artifacts, and updated documentation without removing the cross-platform test matrix.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

triage-can-wait

Problem

Spec Kit requires tests for behavioral changes and regression evidence for bug fixes, but CI currently runs pytest without collecting or comparing code coverage. pytest-cov and coverage configuration already exist. Without a comparable base-versus-PR measurement, coverage regressions can pass unnoticed.

Proposed policy

For pull requests that change src/**/*.py, Python code coverage must not decrease relative to the base revision. A decrease requires human intervention: restore coverage before merging, or have a maintainer explicitly review and document an exceptional decision. Agents must not waive a decrease on their own.

Pull requests that do not change src/**/*.py are exempt from the coverage comparison. Existing behavioral and regression test requirements continue to apply to executable changes elsewhere.

Implementation

  • Add one canonical coverage CI job with a fixed OS and Python version, while retaining the existing cross-platform test matrix.
  • On relevant PRs, run coverage on the base and PR revisions under comparable conditions. Report both coverage values and their covered/coverable line counts in the Actions summary; retain the reports as artifacts.
  • Compare unrounded coverage ratios rather than displayed percentages, and fail the check on a decrease. Make any exception path explicitly human-controlled and documented; do not let an agent silently skip or override the check.
  • On pushes to main, publish the coverage report without attempting a PR comparison.
  • Document the policy in AGENTS.md and the measurement and intervention procedure in CONTRIBUTING.md.

Acceptance criteria

  • In-scope PRs receive a base-versus-PR coverage report and a passing or failing non-regression check.
  • PRs without changes to src/**/*.py do not need a coverage comparison and do not fail due to a skipped comparison.
  • Coverage decreases cannot be silently accepted or waived by an agent; human intervention is required.
  • The existing cross-platform tests and positive/negative and bug-regression test requirements remain in place.

AI disclosure

Drafted with GitHub Copilot (GPT-6 Sol) in an interactive, human-supervised conversation; the requester directed the scope and revised the policy. Reasoning effort was not specified. AI assistance was limited to drafting this issue; no code changes were made.

Lingua principale
Python
Stelle
149k
Fork
13.4k
Merge medio
1g 5h
PR unite (30g)
223

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di github/spec-kit

Tutte le issue di github/spec-kit

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.