[workshop-sim] Repair: Fix checkpoint_quality rubric detection to recognize :white_check_mark: emoji shortcode
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 73/100
Research direction
Read .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and inspect CHECKPOINT_RE. Update the detection to recognize both the Unicode check mark and :white_check_mark: shortcode, then run the assessment script against the corpus. Done when affected steps have non-zero checkpoint quality scores and workshop markdown is unchanged.
Written by the indexing model from the issue text.
Description
Problem statement
Every one of the 30 workshop steps scores checkpoint_quality: 0.0 in curriculum-quality-metrics.json, despite ~90 occurrences of well-formed ## :white_check_mark: Checkpoint sections with checklist items across the workshop. The shared rubric's detection regex (CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py) only matches the literal Unicode ✅ character, but every workshop file uses the GitHub emoji shortcode :white_check_mark: instead. This is a learning barrier classification issue at the tooling level: it does not change actual instructional content, but it silently zeroes out the checkpoint_quality dimension for the entire corpus, which distorts every downstream repair-prioritization decision (including this very simulation run) that relies on checkpoint_quality or the learning KPI index.
Proposed change
Update CHECKPOINT_RE to also match the :white_check_mark: shortcode form, e.g. re.compile(r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", re.IGNORECASE), and re-run the assessment script against the corpus to confirm checkpoint_quality scores reflect the real checklist content (checklist_items * 2.5, capped at 10).
Failure mode classification
Learning barrier (measurement/tooling gap that currently masks true checkpoint scaffolding quality across the corpus — fixing it does not change content, it corrects the signal used to evaluate content).
Quantitative guardrail
This is a rubric/tooling fix, not a content edit, so there is no single affected step's overall_score to hold flat — the expected effect is that checkpoint_quality scores rise from 0.0 toward values reflecting each step's actual checklist_items count (observed range in the corpus suggests meaningful positive scores once fixed), which will raise overall_score for every step that already has a ## :white_check_mark: Checkpoint section with checklist items, and raise the cohort-wide learning KPI index. No content, scaffolding, or active-learning material is removed or altered by this change.
Acceptance criteria
- The shared curriculum quantitative assessment
overall_scorefor every affected step (all 30 main steps) stays flat or improves after the regex fix is applied and the assessment is re-run - The learning KPI index
(2.0 × active_learning + 2.0 × checkpoint_quality + 1.5 × scaffolding) / 5.5improves cohort-wide once checkpoint_quality reflects real checklist content (current cohort mean is artificially capped at 2.87/10 by the 0.0 checkpoint_quality floor) - No workshop markdown content is modified as part of this fix — only the detection regex in
curriculum_assessment.pychanges, verified by runninggit diffagainstworkshop/ - Re-running
curriculum_assessment.pyagainst the corpus produces non-zerocheckpoint_qualityvalues for the ~90 files containing## :white_check_mark: Checkpointsections
Suggested owner
copilot coding agent (small, well-scoped regex fix plus re-running the existing assessment script; no judgment calls about content needed)
Related to #4453
Generated by 🔬 Workshop Student Simulator · copilot · auto · 236.2 AIC · ⌖ 7.07 AIC · ⊞ 15.1K · ◷
- expires on Oct 12, 2026, 8:03 AM UTC
- Dominant language
- JavaScript
- Stars
- 52
- Forks
- 26
- Avg merge
- 12h 58m
- Merged PRs (30d)
- 15
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from githubnext/gh-aw-workshop
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 88/100
githubnext/gh-aw-workshop#4496 ·
Maintainers usually reply within 1 day
-
feedback simulation workshop
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
githubnext/gh-aw-workshop#4495 ·
Maintainers usually reply within 1 day
-
feedback simulation workshop
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
githubnext/gh-aw-workshop#4494 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
githubnext/gh-aw-workshop#4491 ·
Maintainers usually reply within 1 day
-
[workshop-sync] workshop/06-install-gh-aw.md: inaccurate gh aw init output and stale version exampleOpendocumentation
Difficulty 1/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-workshop#4490 ·
Maintainers usually reply within 1 day
All issues in githubnext/gh-aw-workshop
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
enhancement
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
holepunchto/hrpc-test#28 ·
-
good first issue
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
anoopcodehack/DevBoard#616 ·
Maintainers usually reply within 2 days
-
idnits3
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
ietf-tools/idnits#319 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 76/100
521xueweihan/HelloGitHub#3939 ·