[curriculum-eval] curriculum_assessment.py: checkpoint_quality — regex misses :white_check_mark: headings, 92 pages falsely scored 0
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 77/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- github-actions, python
- Domain
- testing-qa
Research direction
Start with .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, especially CHECKPOINT_RE and the prerequisite detection regex. Compare those patterns with the headings quoted in the issue and inspect any related tests or fixtures. Done means the detector recognizes shortcode headings and the cited workshop page reports its checkpoint correctly; the issue also asks to rerun assessment across workshop/*.md.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (affects every workshop/*.md)
Overall Score: n/a — rubric defect (corpus mean: 6.06)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 92/94 learning pages | ≥ 7.5 (≥ 3 items) | −7.5 or more |
| scaffolding | 5.0 on pages that have "Before You Start" (e.g. side-quest-01-02-environment-reference.md) |
10 | −5.0 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) only matches the literal emoji, but workshop pages use the :white_check_mark: shortcode per repo conventions. The scorer therefore reports has_checkpoint: false and missing_checkpoint for pages that do have checkpoints, depressing the corpus mean by up to ~2 points (weight 2.0).
Evidence (quoted from the file):
## :white_check_mark: Checkpoint—workshop/04-github-actions-intro.mdline 138, which lists five checklist items yet is reported ashas_checkpoint: false, checkpoint_quality: 0.0.
Learning Science Rationale:
Valid measurement is a precondition for acting on rubric output. Formative-assessment research (Black & Wiliam) stresses that checkpoints give learners retrieval practice and feedback; a metric that cannot see them sends authors to add redundant checkpoints, or to ignore the signal entirely. Fixing the detector restores the rubric's ability to discriminate genuinely weak checkpoints (few or vague items) from present ones, and also exposes that history deltas such as the −2.00 drops on side-quest-11-09-agent-session-phases.md and side-quest-12-01-iterate-agent-output.md are likely heading-format changes rather than real content regressions.
Improvement Prompt (for an agent):
Edit .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py:
1. Change CHECKPOINT_RE to r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint" (case-insensitive).
2. Verify the prerequisite/scaffolding regex also matches "## :clipboard: Before You Start" and "Prerequisites" with the :clipboard: shortcode; fix if not (side-quest-01-02-environment-reference.md has this heading but has_prereq_section is false).
3. Update any unit tests/fixtures and re-run the assessment over workshop/*.md; confirm has_checkpoint is true for workshop/04-github-actions-intro.md and that the number of pages flagged missing_checkpoint drops from 92 to only pages genuinely lacking one.
4. Regenerate score-history baselines if they are cached so trend deltas stay meaningful.
Expected Score After Fix: 7.5 / 10.0 (corpus mean)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~43 AIC · ⌖ ~2 AIC · ⊞ 5K · ◷
- expires on Nov 6, 2026, 11:04 AM UTC
- Dominant language
- JavaScript
- Stars
- 58
- Forks
- 40
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 8
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
githubnext/gh-aw-workshop#5360 ·
Maintainers usually reply within 3 days
-
[curriculum-eval] 05-agentic-workflows-intro.md: active_learning — 1,107 words with density 0.72Opencurriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
githubnext/gh-aw-workshop#5361 ·
Maintainers usually reply within 3 days
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
githubnext/gh-aw-workshop#5359 ·
Maintainers usually reply within 3 days
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
githubnext/gh-aw-workshop#5358 ·
Maintainers usually reply within 3 days
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
githubnext/gh-aw-workshop#5356 ·
Maintainers usually reply within 3 days
All issues in githubnext/gh-aw-workshop
Similar issues
-
502 on Node 26: ERR_INVALID_ARG_VALUE: Setting the TLS ServerName to an IP address is not permittedOpen
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
github/paste-markdown#129 · 2 reactions ·
-
bug
Difficulty 1/5 1-3 hours Newbie friendliness 83/100
obsidianmd/obsidian-help#1164 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
richardcase/skills#26 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day