Diagnostic category comment disagrees with its tests

Aperta Adatta ai principianti
  • Qualcosa non va?
#666 1 commento 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@woohsin ci sta già lavorando.

Dal 13/11/2026.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
66/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
rust
Ambito
backend, testing

Direzione di ricerca

Leggi il commento del sanitizer in src/agent/integrity.rs:150-157 insieme a diagnostic() in src/agent/evidence/run_signals.rs:53-68 e ai tre test bloccati in tests/unit/agent/evidence.rs. Decidi se modificare il commento o il comportamento; l’issue dice che correggere il commento è l’opzione più piccola. Il lavoro è concluso quando il comportamento scelto e i relativi test concordano con il commento.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Observed behavior

The comment above the diagnostic sanitizer in src/agent/integrity.rs:150-157 says a category is counted as other only when the run carries a setup error, and that a category on a run that did not fail to start is counted as nothing. The code does otherwise.

diagnostic() in src/agent/evidence/run_signals.rs:53-68 returns a result whenever the runner names a valid category, with or without setupError, using structured:<Category> as its material. a_structured_category_is_recorded_without_setup_text (tests/unit/agent/evidence.rs:1470) and a_runner_naming_other_without_prose_is_still_a_diagnostic (tests/unit/agent/evidence.rs:2257) pin that.

So a run that executed and passed everything still counts as a runner failure when it names a category, and the prompt can print 1 runs failed to start (src/agent/evidence.rs:580) beside 5 of 5 passing.

Expected behavior

The comment and the behavior agree. Either of these resolves it:

  • Correct the comment to say a runner-named category is counted without prose, and keep the code and tests.
  • Stop counting a category on a run that executed, and change the two tests above with it.
Reproduction
  1. Call record_test with {"passed":5,"total":5,"diagnostic":{"category":"syntax"}}.
  2. Read the ledger's diagnostics: the syntax counter is 1 and the compile or runner failure count is non-zero.

Read from the code and the pinned tests, not run against this exact payload.

CodeTrial version

desktop-latest-25-g55597864

Environment

Not environment specific.

Sanitized evidence

The line is labelled as browser-reported and unverified, so the impact on the report is small.

Dropping the fallback also fails test_claims_keep_browser_provenance_in_every_prompt_view (tests/unit/agent/evidence.rs:3150) in addition to the two tests above, so the intended behavior should be decided first. Deciding which of the two is intended is the whole task; the first option is the smaller change.

Public data confirmation
  • I removed credentials, session tokens, candidate recordings, and personal interview content from this public report.
Lingua principale
Rust
Stelle
228
Fork
150
Merge medio
1g 3h
PR unite (30g)
226

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di sysprog21/codetrial

Tutte le issue di sysprog21/codetrial

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.