[curriculum-eval] curriculum_assessment.py: checkpoint_quality and scaffolding — regexes miss :white_check_mark: / :clipboard: shortcodes
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
Research direction
Start with .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and inspect CHECKPOINT_RE and has_prereq_section; then check for existing tests. Compare the regex matches against the shortcode headings shown in the issue and run the assessment on workshop/*.md. Done means both shortcode headings are detected while ## Prerequisites still matches.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: corpus mean 6.06 / 10.0 (threshold 5.52) — scores are systematically understated
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 92/94 pages | ≥ 4 items → 10.0 | up to −10.0 (weight 2.0) |
| scaffolding | 5.0 on 94/94 pages | 10.0 when section present | −5.0 (weight 1.5) |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE requires the literal ✅ emoji and has_prereq_section requires the literal 📋 emoji, but workshop pages use the GitHub shortcodes :white_check_mark: and :clipboard:. Every page therefore loses up to ~2.0 + 0.75 weighted points, and missing_checkpoint flags are false positives.
Evidence (quoted from the file):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)...re.search(r"##\s+📋\s*Before You Start", raw, re.IGNORECASE)
while workshop/04-github-actions-intro.md contains ## :clipboard: Before You Start and ## :white_check_mark: Checkpoint with 5 checklist items.
Learning Science Rationale:
Metrics used to steer instructional design must be valid measures of the construct they claim to measure. Retrieval-practice checkpoints (Roediger & Karpicke) and advance-organizer scaffolding (Ausubel) are both present in the workshop, but the rubric reports them as absent, which would push agents to "fix" content that is not broken and hides real issues (length, concept load).
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both the emoji and shortcode forms, e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", and update has_prereq_section so "Before You Start" matches both "📋" and ":clipboard:" (and still matches "## Prerequisites"). Add/extend tests if any exist. Re-run the assessment over workshop/*.md and confirm has_checkpoint is true for pages containing "## :white_check_mark: Checkpoint" and has_prereq_section is true for pages containing "## :clipboard: Before You Start". Do not edit workshop content.
Expected Score After Fix: corpus mean ≈ 7.5–8.0 / 10.0 (checkpoint and scaffolding dimensions recover for nearly all pages)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~47 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷
- expires on Oct 30, 2026, 3:35 AM UTC
- Dominant language
- JavaScript
- Stars
- 56
- Forks
- 40
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
feedback simulation workshop
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
githubnext/gh-aw-workshop#5157 ·
Maintainers usually reply within 1 day
-
feedback simulation workshop
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
githubnext/gh-aw-workshop#5156 ·
Maintainers usually reply within 1 day
-
[workshop-sync] workshop/side-quest-13-02-pr-summary-pattern.md: agent job uses write permissionOpendocumentation
Difficulty 1/5 Under an hour Newbie friendliness 85/100
githubnext/gh-aw-workshop#5153 ·
Maintainers usually reply within 1 day
-
[workshop-sync] workshop/side-quest-13-01-pr-labeler-pattern.md: agent job uses write permissionOpendocumentation
Difficulty 1/5 Under an hour Newbie friendliness 88/100
githubnext/gh-aw-workshop#5154 ·
Maintainers usually reply within 1 day
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
githubnext/gh-aw-workshop#5146 ·
Maintainers usually reply within 1 day
All issues in githubnext/gh-aw-workshop
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/monaco-editor#5563 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
account: profile bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 5 days
-
ai-driven-fuzzing bug claude Minor
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
linagora/cozy-client#1777 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day