calibrate-judge build: a sample holding `---` makes its case fail to load and go untested
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start with calibrate-judge.py and inspect yaml_quoted() and the build output path; then read validate-cases.py to understand the YAML subset it accepts. Add a fixture sample containing a Markdown table rule and test that built output loads and round-trips to the original sample. Done when no --- run survives in the emitted line and the regression test passes.
Written by the indexing model from the issue text.
Description
Observed
In the code-tidying pilot (#6539), calibrate-judge.py build wrote 5 calibration cases that claude plugin eval refused with invalid YAML frontmatter: Unexpected EOF. Each sample held a markdown table rule (---). prompt_md() puts the sample in one double-quoted append_system_prompt scalar via yaml_quoted(), which escapes backslash, quote, newline, tab and carriage return, but not -. The CLI's frontmatter reader ended the frontmatter at the --- inside the quoted value.
Those 5 cases were silently untested: score lists a sample with no reproduced run as untested, but a case that never loads is easy to miss in a long run. Observed at Claude Code 2.1.289 on 2026-10-07.
Workaround used
Escaping each - in the sample (a YAML double-quoted escape that parses to the same string) let the cases load and calibrate.
Fix wanted
yaml_quoted()escapes so that no---run survives in the emitted line, within the subsetvalidate-cases.pyreads.- A fixture sample holding a table rule, with a test that
build's output loads invalidate-cases.pyand round-trips to the original sample.
Refs: #6476
Generated by Claude Code
- Dominant language
- Shell
- Stars
- 22
- Forks
- 2
- Avg merge
- 5h 18m
- Merged PRs (30d)
- 869
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from melodic-software/claude-code-plugins
-
good first issue needs-triage priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
melodic-software/claude-code-plugins#6699 · 1 comment ·
Maintainers usually reply within 1 day
-
good first issue needs-triage priority: medium
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
melodic-software/claude-code-plugins#6631 · 1 comment ·
Maintainers usually reply within 1 day
-
needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
melodic-software/claude-code-plugins#6535 ·
Maintainers usually reply within 1 day
-
test_comment_census.py: SccArgv flag-shaped-filename test errors on Windows (#!/bin/sh scc shim)Opengood first issue needs-triage priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
melodic-software/claude-code-plugins#6532 · 1 comment ·
Maintainers usually reply within 1 day
-
good first issue needs-triage priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
melodic-software/claude-code-plugins#6390 · 1 comment ·
Maintainers usually reply within 1 day
All issues in melodic-software/claude-code-plugins
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
zerocracy/judges-action#2733 ·
Maintainers usually reply within 8 days
-
dependencies feature github_actions good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
wemake-services/wemake-django-template#3149 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
crazy-goat/rabbit-stream#830 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
severity: low
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
luainkernel/lunatik#1853 ·
Maintainers usually reply within 1 day