[curriculum-eval] curriculum_assessment.py: checkpoint_quality — regex misses :white_check_mark: shortcode, 92/94 pages falsely flagged
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 85/100
Research direction
Read .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and locate CHECKPOINT_RE and the prereq regex. Update them to match both emoji and GitHub shortcode heading forms as described, then re-run the assessment against workshop/ and confirm has_checkpoint is true for 04-github-actions-intro.md with rising corpus mean. Add a unit test for both heading forms.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: corpus mean 6.06 / 10.0 (depressed by scorer defect)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 92/94 pages | ≥ 7.5 | −7.5 |
| scaffolding | 5.0 on most pages | 10 | −5.0 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE matches ##\s+✅\s*Checkpoint and the prerequisite check matches ##\s+📋\s*Before You Start, but workshop files write these headings with GitHub shortcodes (## :white_check_mark: Checkpoint, ## :clipboard: Before You Start). Every page is therefore scored as missing both, which drags scores by about 2–3 points and triggers missing_checkpoint on 92 of 94 pages.
Evidence (quoted from the file):
workshop/04-github-actions-intro.mdline 138:## :white_check_mark: Checkpointfollowed by 5 checklist items — yet its metrics reporthas_checkpoint: false,checkpoint_quality: 0.0.
Scorer: CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)
Learning Science Rationale:
Valid measurement underpins any evidence-based review: a rubric with a systematic false negative produces misleading prioritisation (for example, the −2.00 "declines" on side-quest-11-09 and side-quest-12-01 likely reflect heading-format changes rather than learning quality). Fixing the detector lets Cognitive Load, Mayer's segmenting and retrieval-practice signals (checkpoints) be evaluated on real differences.
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both the emoji and shortcode forms, e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", and update the has_prereq_section regex to match r"##\s+(?:📋|:clipboard:)\s*Before You Start" (also accept a "Prerequisites" heading). Also check CHECKLIST_RE and any other emoji-dependent patterns for shortcode equivalents. Re-run the assessment against workshop/ and confirm has_checkpoint is true for 04-github-actions-intro.md and that corpus mean rises. Add a small unit test covering both heading forms.
Expected Score After Fix: ~8.3 / 10.0 corpus mean (checkpoint +2.0 weight × ~8 and scaffolding +1.5 × 5 on most pages)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~42 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷
- expires on Oct 26, 2026, 5:07 AM UTC
- Dominant language
- JavaScript
- Stars
- 56
- Forks
- 40
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 16
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
githubnext/gh-aw-workshop#5077 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
githubnext/gh-aw-workshop#5068 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
githubnext/gh-aw-workshop#5067 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
githubnext/gh-aw-workshop#5065 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
githubnext/gh-aw-workshop#5059 ·
Maintainers usually reply within 1 day
All issues in githubnext/gh-aw-workshop
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
-
Difficulty 1/5 1-3 hours Newbie friendliness 91/100
Maintainers usually reply within 1 day
-
enhancement ui-dashboard
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
security
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
remotestorage/remotestorage.js#1405 ·
-
🐞 bug 🧩 ai-context
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
owasp-noir/noir#3295 ·
Maintainers usually reply within 1 day