Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

issue-backlog-stats: STALE-CANDIDATE is listed as a triage class but defined as orthogonal

Chiusa
  • Qualcosa non va?
#1,678 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@sharpca ci sta già lavorando.

Dal 21/10/2026.

  • #1753 di @sharpca — aperta
  • #1754 di @HarshRajSinghania — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
markdown, python
Ambito
devtools, testing-qa

Direzione di ricerca

Start with the Step 2 class table and the Step 3 bucket definition in skills/issue-backlog-stats/SKILL.md, then the output spec at tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/output-spec.md. Check whether the Step 5 render section or the spec-loop spec also lists five classes. Done means the skill and spec treat staleness only as the is_stale_candidate flag, and the case-4 fixture passes in the eval runner.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

capability:fix capability:reassess capability:resolve capability:stats capability:triage family:docs family:issue family:tools good first issue substrate:framework-dev
What's broken

The issue-backlog-stats skill contradicts itself about what STALE-CANDIDATE is.

Step 2 of the skill tells the model to give each issue exactly one triage class. The table of classes has five rows, and STALE-CANDIDATE is one of them. But the same row says staleness is "orthogonal" and that a stale issue should be counted under its real class as well.

Those two instructions cannot both be followed. Sonnet follows the first one. Opus and Fable follow the second one. The eval fixture expects the second one, so Sonnet fails it.

Which layer

Skill: skills/issue-backlog-stats/SKILL.md.

Two places treat STALE-CANDIDATE as a class:

  • The Step 2 class table, in the STALE-CANDIDATE row.
  • The Step 3 bucket definition, which says "issues with class STALE-CANDIDATE".

The eval output spec has the same problem. It lists STALE-CANDIDATE as an allowed value of triage_class, and then says staleness belongs in the separate is_stale_candidate boolean. File: tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/output-spec.md.

How to reproduce

Run the suite with Sonnet on main at 399ab1bc:

PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner \
  --cli "claude -p --model claude-sonnet-5-5" tools/skill-evals/evals/issue-backlog-stats/

The case step-2-classify/case-4-stale-candidate fails every time. I ran it 10 times and got the same diff on all 10:

$.triage_class: expected='UNTRIAGED', actual='STALE-CANDIDATE'

The fixture is an untriaged issue that was last updated 216 days ago, with warn_days set to 90.

For comparison, Opus and Fable each pass this case on 10 of 10 runs. The full runs are in the suite's Run history table, added in #1626.

Expected vs actual

Expected: triage_class is UNTRIAGED and is_stale_candidate is true.

Actual from Sonnet: triage_class is STALE-CANDIDATE.

The fixture is right and the Step 2 table is wrong. The rest of the skill already treats staleness as a flag, not a class:

  • Step 3 counts stale candidates using is_stale_candidate.
  • The untriaged-funnel rule in Step 3 selects issues that are UNTRIAGED and not stale. That only makes sense if an issue can be both.
Suggested fix
  1. Remove the STALE-CANDIDATE row from the Step 2 class table, leaving four classes that do not overlap.
  2. Under the table, add one line saying is_stale_candidate is a separate flag set when days_since_updated >= warn_days.
  3. Reword the Step 3 bucket to count issues where that flag is set.
  4. Remove STALE-CANDIDATE from the triage_class values in the step-2 output spec.
  5. Leave the case-4 fixture as it is.
Surface area (optional)
  • I have not checked Step 5 (render) or the spec-loop spec for this skill. If either assumes five classes, the fix needs to touch them too.
  • This edits SKILL.md, so it needs a measured_tokens restamp and an eval rerun. Sonnet is the model that needs to go from failing to passing.
  • Same kind of problem as #1615: the eval failure points at the prompt, not the fixture.
Lingua principale
Python
Stelle
124
Fork
114
Merge medio
14h 39m
PR unite (30g)
370

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/magpie

Tutte le issue di apache/magpie

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.