Report and enforce coverage non-regression for src Python changes

Ouverte
  • Quelque chose ne va pas ?
#4,786 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
50/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
github-actions, python
Domaine
ci-cd, documentation, testing-qa

Piste de recherche

Read the existing pytest CI configuration first, then inspect the coverage settings and the documented guidance in AGENTS.md and CONTRIBUTING.md. Trace how pull requests identify changes under src/**/*.py and how main pushes are handled. Done means comparable base-versus-PR reports, an unrounded non-regression check with human-controlled exceptions, retained artifacts, and updated documentation without removing the cross-platform test matrix.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

triage-can-wait

Problem

Spec Kit requires tests for behavioral changes and regression evidence for bug fixes, but CI currently runs pytest without collecting or comparing code coverage. pytest-cov and coverage configuration already exist. Without a comparable base-versus-PR measurement, coverage regressions can pass unnoticed.

Proposed policy

For pull requests that change src/**/*.py, Python code coverage must not decrease relative to the base revision. A decrease requires human intervention: restore coverage before merging, or have a maintainer explicitly review and document an exceptional decision. Agents must not waive a decrease on their own.

Pull requests that do not change src/**/*.py are exempt from the coverage comparison. Existing behavioral and regression test requirements continue to apply to executable changes elsewhere.

Implementation

  • Add one canonical coverage CI job with a fixed OS and Python version, while retaining the existing cross-platform test matrix.
  • On relevant PRs, run coverage on the base and PR revisions under comparable conditions. Report both coverage values and their covered/coverable line counts in the Actions summary; retain the reports as artifacts.
  • Compare unrounded coverage ratios rather than displayed percentages, and fail the check on a decrease. Make any exception path explicitly human-controlled and documented; do not let an agent silently skip or override the check.
  • On pushes to main, publish the coverage report without attempting a PR comparison.
  • Document the policy in AGENTS.md and the measurement and intervention procedure in CONTRIBUTING.md.

Acceptance criteria

  • In-scope PRs receive a base-versus-PR coverage report and a passing or failing non-regression check.
  • PRs without changes to src/**/*.py do not need a coverage comparison and do not fail due to a skipped comparison.
  • Coverage decreases cannot be silently accepted or waived by an agent; human intervention is required.
  • The existing cross-platform tests and positive/negative and bug-regression test requirements remain in place.

AI disclosure

Drafted with GitHub Copilot (GPT-6 Sol) in an interactive, human-supervised conversation; the requester directed the scope and revised the policy. Reasoning effort was not specified. AI assistance was limited to drafting this issue; no code changes were made.

Langage dominant
Python
Étoiles
149k
Forks
13.4k
Merge moyen
1 j 5 h
PR mergées (30 j)
223

Préparer son environnement

Ouvrir dans Codespaces

Lance le conteneur de développement du projet dans votre navigateur, avec votre propre compte GitHub.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de github/spec-kit

Toutes les issues de github/spec-kit

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.