[curriculum-eval] curriculum_assessment.py: checkpoint_quality — emoji-only regex scores shortcode checkpoints as missing
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
The fix is in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, in the CHECKPOINT_RE and Before-You-Start regex definitions, which only match literal emoji. Start by reading those two patterns and one page such as workshop/04-github-actions-intro.md that uses the :white_check_mark: shortcode. Done means both regexes accept the emoji and shortcode forms, new unit tests cover both, and the re-run reports has_checkpoint true for that page.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: corpus mean 6.09 / 10.0 (all pages affected)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 90/93 pages | ≥ 7.5 (checkpoint with ≥ 3–4 items) | up to −10 |
| scaffolding | 5.0 on 93/93 pages | 10 | −5 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE (##\s+✅\s*Checkpoint) and the Before-You-Start regex (##\s+📋\s*Before You Start) require literal emoji, but workshop pages use GitHub shortcodes (## :white_check_mark: Checkpoint, ## :clipboard: Before You Start), so detection always fails. Result: every non-dispatcher page is penalised ~2.5–3 points of weighted score and findings such as missing_checkpoint are false positives.
Evidence (quoted from the file):
workshop/04-github-actions-intro.md: "## ✅ Checkpoint" followed by 5 checklist items — scoredhas_checkpoint: false,checkpoint_quality: 0.0.
Learning Science Rationale:
Valid measurement is a precondition for acting on Cognitive Load Theory and retrieval-practice findings: checkpoints are a retrieval-practice mechanism (Roediger & Karpicke), so a rubric that reports them absent misdirects effort toward pages that already comply and hides real gaps (e.g. only 3 pages truly lack checkpoints: 00-welcome, 01-prerequisites, 06-install-gh-aw).
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both the emoji and shortcode forms: r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", and update the "Before You Start" regex to r"##\s+(?:📋|:clipboard:)\s*Before You Start". Add unit tests covering both forms, rerun the assessment over workshop/*.md, and confirm has_checkpoint is true for pages like 04-github-actions-intro.md and has_prereq_section is true where "## :clipboard: Before You Start" exists. Also check other emoji-based regexes in the file for the same shortcode mismatch.
Expected Score After Fix: ~7.8 / 10.0 corpus mean (from 6.09)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 25.5 AIC · ⌖ 0.831 AIC · ⊞ 10.4K · ◷
- expires on Oct 15, 2026, 6:54 AM UTC
- Dominant language
- JavaScript
- Stars
- 53
- Forks
- 35
- Avg merge
- 13h 50m
- Merged PRs (30d)
- 14
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 88/100
githubnext/gh-aw-workshop#4569 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 85/100
githubnext/gh-aw-workshop#4568 ·
Maintainers usually reply within 1 day
-
accessibility bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
githubnext/gh-aw-workshop#4555 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
githubnext/gh-aw-workshop#4548 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 66/100
githubnext/gh-aw-workshop#4549 ·
Maintainers usually reply within 1 day
All issues in githubnext/gh-aw-workshop
Similar issues
-
docs hacktoberfest help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
pacocartones/free-llm-api-hub#347 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
cytoscape/cytoscape.js#3536 ·
Maintainers usually reply within 15 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
cuttle-cards/cuttle#1440 ·
Maintainers usually reply within 1 day
-
URL-encoded parser builds nested objects nothing readsPossibly taken @Senthil455 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Janani-bn/CivicFix#168 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
code-yeongyu/oh-my-openagent#10172 ·
Maintainers usually reply within 1 day