[curriculum-eval] curriculum_assessment.py: checkpoint and scaffolding regexes miss :white_check_mark: / :clipboard: shortcodes
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Start with .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, especially CHECKPOINT_RE, the prerequisite section check, CHECKLIST_RE, and other emoji based patterns. Read the relevant unit tests and fixtures, then run the assessment against workshop/*.md. Done when checkpoint matches rise to about 92 files and prerequisite matches to about 87, with tests and fixtures updated and workshop content unchanged.
Written by the indexing model from the issue text.
Description
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: n/a (tooling defect affecting all 94 files; corpus mean 6.06)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 92/94 files | ≥ 7.5 (≥ 3 items) | ≈ −7.5 |
| scaffolding | 5.0 on 94/94 files | 10 | −5.0 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE requires a literal ✅ and the prereq regex requires a literal 📋, but workshop headings use ## :white_check_mark: Checkpoint and ## :clipboard: Before You Start. As a result has_checkpoint is true for only 2/94 files and has_prereq_section for 0/94, even though 92 files have a Checkpoint and 87 have "Before You Start".
Evidence (quoted from the file):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)...re.search(r"##\s+📋\s*Before You Start", raw, re.IGNORECASE)
Workshop file: ## :white_check_mark: Checkpoint, ## :clipboard: Before You Start.
Learning Science Rationale:
Scores are only useful if they measure the construct they claim to. Retrieval practice and formative checks (Roediger & Karpicke) are a core design feature of this workshop; a scorer that reports them as absent produces false missing_checkpoint findings, hides true regressions (the −2.00 "declines" on side-quest-11-09 and side-quest-12-01 are artefacts) and sends improvement effort to the wrong pages.
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both "## ✅ Checkpoint" and "## :white_check_mark: Checkpoint" (e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint"), and update the has_prereq_section regex to match "## 📋 Before You Start", "## :clipboard: Before You Start" and "## Prerequisites". Also check CHECKLIST_RE and any other emoji-based patterns (e.g. :dart:, :pencil2:) for the same problem. Re-run the assessment script against workshop/*.md and confirm has_checkpoint is true for ~92 files, has_prereq_section for ~87, then update any unit tests/fixtures. Do not modify workshop content.
Expected Score After Fix: corpus mean ≈ 8.5 / 10.0 (checkpoint and scaffolding dimensions recovered)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~45 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷
- expires on Nov 6, 2026, 3:32 AM UTC
- Dominant language
- JavaScript
- Stars
- 58
- Forks
- 40
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 8
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
githubnext/gh-aw-workshop#5400 ·
Maintainers usually reply within 3 days
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
githubnext/gh-aw-workshop#5401 ·
Maintainers usually reply within 3 days
-
curriculum documentation quality
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
githubnext/gh-aw-workshop#5392 ·
Maintainers usually reply within 3 days
-
documentation
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
githubnext/gh-aw-workshop#5379 ·
Maintainers usually reply within 3 days
-
[workshop-sync] workshop/side-quest-11-02-yaml-frontmatter.md: `emoji: :bar_chart:` is invalid YAMLOpendocumentation
Difficulty 1/5 Under an hour Newbie friendliness 88/100
githubnext/gh-aw-workshop#5368 ·
Maintainers usually reply within 3 days
All issues in githubnext/gh-aw-workshop
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
obsidianmd/obsidian-developer-docs#281 ·
Maintainers usually reply within 4 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
FriendsOfFlarum/synopsis#16 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Fei-Away/Codex-Dream-Skin#449 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
sysprog21/codetrial#604 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
outline/outline#14489 · 1 comment ·
Maintainers usually reply within 1 day