Prove a green suite was earned: changed-line coverage and mutation checks

Inizia con i payload Edit della trascrizione e i relativi dati oldString/newString e structuredPatch, quindi traccia come lo Stop hook può ispezionare i report di coverage senza chiamate al modello. Implementa un formato end-to-end, ad esempio coverage.xml, e verifica che le righe modificate vengano mappate solo quando il report è più recente dell’ultima modifica. Il lavoro è completo quando viene segnalato un unsupported result se le righe modificate non erano coperte, mentre la mutazione rimane opt-in tramite backcheck --mutate.

Today backcheck can tell you the suite passed and that nobody disabled a test to get there.
It cannot tell you the passing suite actually exercises the code that changed — the most
common way a green bar means nothing.

An agent edits src/billing.py, runs the suite, everything passes. If no test touches the new
branch, the green is real and meaningless.

What would close the gap

Changed-line coverage. The agent edited these files; the coverage report says these lines
ran. Were any of the changed lines covered?

✗ unsupported  tests pass
    `pytest` passed, but none of the 14 lines changed in src/billing.py were executed
    by the suite (coverage.xml)

This is tractable because coverage output is a file on disk with a stable format, and
backcheck already knows exactly which lines were edited — the Edit payloads in the
transcript carry oldString/newString and a structuredPatch.

• parse coverage.xml / .coverage (Python)
• parse lcov.info (JS, Rust via tarpaulin/llvm-cov)
• parse cobertura.xml (widely emitted)
• map changed lines from the transcript's patches onto the coverage report
• report only when a coverage file exists and is newer than the last edit

Mutation signal (harder, later). A test that runs a line but asserts nothing about it still
passes when the line is broken. Scoped mutation testing on changed lines only is the rigorous
version of "was this green earned", and is
discussed
as the real answer to agents gaming test suites. It is slow and invasive, so it would have to be
opt-in (backcheck --mutate) and probably delegated to mutmut/cargo-mutants/stryker
rather than reimplemented.

Design constraint

Coverage parsing keeps the no-model-calls rule intact and stays deterministic, which is why it
is the right first step. Anything that needs to run code (mutation) must be explicitly opt-in
and must never happen inside the Stop hook — the hook has to stay fast enough to be invisible.

Start with one coverage format end-to-end rather than a general framework.