[curriculum-eval] curriculum_assessment.py: checkpoint_quality — CHECKPOINT_RE misses `:white_check_mark:` headings, scoring 92 pages as 0
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start with .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, especially CHECKPOINT_RE and the has_prereq_section detection; compare those patterns with the headings in workshop/04-github-actions-intro.md. Check other emoji based regexes in the same file, then update relevant unit tests. Done when the checkpoint and prerequisite headings using shortcodes are detected and the assessment reports a positive checkpoint_quality for that workshop page.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (affects all workshop/*.md)
Overall Score: n/a — rubric defect (corpus mean: 6.06)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 92 of 92 learning pages | ≥ 4 items → 10 | −10 |
| scaffolding | 5.0 on pages that do have :clipboard: Before You Start |
10 | −5 (likely same cause) |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) (line 10) matches only the literal emoji, but workshop files use the GitHub shortcode ## :white_check_mark: Checkpoint, as the repo conventions require. All 92 learning pages therefore score has_checkpoint: false and checkpoint_quality: 0, which also drives the false missing_checkpoint flag.
Evidence (quoted from the file):
workshop/04-github-actions-intro.mdline 138:## :white_check_mark: Checkpointfollowed by 5 checklist items. The rubric reportshas_checkpoint: false,checkpoint_quality: 0.0. A count of## :white_check_mark: Checkpointacrossworkshop/returns 92, which equals the number of pages flagged as missing.
Learning Science Rationale:
Formative assessment checkpoints support retrieval practice and give learners feedback (the testing effect, Roediger & Karpicke). A metric that reports zero checkpoints stops the evaluator from distinguishing pages that really lack them, and it hides real quality differences. The reported 2.0-weight penalty compresses all scores into roughly 4.9–8.6 and generates spurious "declining" trends (for example -2.00 on two pages).
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE so it matches both the literal emoji heading and the GitHub shortcode heading, e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint". Also check how the has_prereq_section / scaffolding detection matches "Before You Start" (the workshop uses "## :clipboard: Before You Start") and accept the ":clipboard:" shortcode as well as the emoji. Check any other emoji-based regexes in the file for the same shortcode mismatch. Update any unit tests, then re-run the assessment and confirm that pages such as workshop/04-github-actions-intro.md report has_checkpoint=true and checkpoint_quality > 0. Do not edit workshop content.
Expected Score After Fix: ~7.5 / 10.0 corpus mean (up from 6.06; the rubric's checkpoint weight is 2.0 of 12.0 and most pages have ≥ 4 items)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~48 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷
- expires on Oct 30, 2026, 4:05 AM UTC
- Dominant language
- JavaScript
- Stars
- 56
- Forks
- 40
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
githubnext/gh-aw-workshop#5193 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-workshop#5194 ·
Maintainers usually reply within 1 day
-
[workshop-sync] workshop/side-quest-17-05-supply-chain-mcp.md: verify `network.allowed` link anchorsOpendocumentation
Difficulty 1/5 Under an hour Newbie friendliness 82/100
githubnext/gh-aw-workshop#5192 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
githubnext/gh-aw-workshop#5184 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
githubnext/gh-aw-workshop#5182 ·
Maintainers usually reply within 1 day
All issues in githubnext/gh-aw-workshop
Similar issues
-
Let TypeScript accept an initial object in ReactiveDictPossibly taken @yograj-vijay-yadav claimed this today. Openbug good first issue help wanted workshop
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
meteor/meteor#14919 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
akitaonrails/FrankMD#268 ·
Maintainers usually reply within 1 day
-
account: profile
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 5 days
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
mfreed7/xslt_polyfill#65 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day