[curriculum-eval] curriculum_assessment.py: checkpoint_quality — shortcode headings not detected, 91/93 pages falsely score 0
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- github-actions, markdown, python
- Domain
- developer-experience, testing
Research direction
Start with .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and inspect CHECKPOINT_RE, has_prereq_section, and the checklist counting logic. Compare their behavior on the literal emoji and shortcode headings shown in the issue, then run the assessment and relevant unit tests. Done means those headings are recognized and pages with checkpoint sections no longer get false missing_checkpoint findings.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (affects every workshop/*.md)
Overall Score: 5.4 / 10.0 for workshop/04-github-actions-intro.md (corpus mean: 6.1)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 | ≥ 4 items → 10.0 | -10.0 |
| scaffolding | 5.0 | 10.0 | -5.0 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE matches only a literal ✅ and has_prereq_section only a literal 📋, but workshop files use :white_check_mark: / :clipboard: shortcodes (per AGENTS.md). As a result 91 of 93 pages report has_checkpoint: false although they have 4+ checklist items.
Evidence (quoted from the file):
## :white_check_mark: Checkpointfollowed by five- [ ]items (workshop/04-github-actions-intro.md, lines 138–144) is scoredhas_checkpoint: false,checkpoint_quality: 0.0.
Learning Science Rationale:
Valid measurement is a precondition for acting on formative-assessment signals (Black & Wiliam). Checkpoints support retrieval practice and self-monitoring; a rubric that scores them 0 for nearly all pages produces false missing_checkpoint findings and hides genuine differences such as length (Cognitive Load Theory) and low activity density.
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both "## ✅ Checkpoint" and "## :white_check_mark: Checkpoint", and update the has_prereq_section regex to match both "## 📋 Before You Start" and "## :clipboard: Before You Start" (plus "## Prerequisites"). Ensure checkpoint checklist-item counting works with the shortcode form. Add/adjust unit tests, re-run the assessment, and confirm the corpus no longer reports missing_checkpoint for pages that have a checkpoint section.
Expected Score After Fix: 7.4 / 10.0 for 04-github-actions-intro.md (checkpoint +2.0 weight, scaffolding +0.75 effective); corpus mean rises to roughly 7.5.
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 25.6 AIC · ⌖ 0.781 AIC · ⊞ 10.4K · ◷
- expires on Oct 23, 2026, 5:38 AM UTC
- Dominant language
- JavaScript
- Stars
- 55
- Forks
- 36
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 17
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
githubnext/gh-aw-workshop#4897 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
githubnext/gh-aw-workshop#4896 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 88/100
githubnext/gh-aw-workshop#4885 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 1-3 hours Newbie friendliness 84/100
githubnext/gh-aw-workshop#4878 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 90/100
githubnext/gh-aw-workshop#4879 ·
Maintainers usually reply within 1 day
All issues in githubnext/gh-aw-workshop
Similar issues
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
fossasia/visdom-dev#69 ·
Maintainers usually reply within 3 days
-
Difficulty 1/5 Under an hour Newbie friendliness 74/100
-
建议写成 1949-Open
Difficulty 1/5 Under an hour Newbie friendliness 62/100
DeanChensj/jiangshan-map#133 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
ew
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day