[curriculum-eval] curriculum_assessment.py: checkpoint_quality/scaffolding — regex ignores :shortcode: headings, 92/94 pages scored 0

Closed Beginner friendly
  • Is something wrong?
#5,255 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, markdown, python
Domain
documentation, tooling

Research direction

Start in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and locate CHECKPOINT_RE and the has_prereq_section regex. Check a workshop page such as workshop/04-github-actions-intro.md to see the :white_check_mark: and :clipboard: shortcode headings. Done when both emoji and shortcode forms match, unit tests cover each form, and checkpoint_quality is no longer uniformly 0.0 across the corpus.

Written by the indexing model from the issue text.

Description

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: n/a (tooling defect; corpus mean 6.06)

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 on 92/94 pages ≥ 10 with ≥ 4 items −10
scaffolding 5.0 on 94/94 pages 10 when Before You Start present −5

Root Cause (≤ 2 sentences):
CHECKPOINT_RE is ##\s+✅\s*Checkpoint and the prereq check uses ##\s+📋\s*Before You Start, but workshop files use the GitHub shortcodes :white_check_mark: and :clipboard:. Only side-quest-13-01 and 13-02 use literal emoji, so they are the only pages scored correctly (8.59 and 8.39 vs corpus mean 6.06).

Evidence (quoted from the file):

## :white_check_mark: Checkpoint — followed by 5–7 - [ ] items (e.g. workshop/04-github-actions-intro.md), yet flagged missing_checkpoint.

Learning Science Rationale:
Valid measurement is a precondition for acting on any rubric: Kirkpatrick-style evaluation and formative assessment both rely on instruments that detect the construct they claim to measure. A systematic false negative on checkpoints (the retrieval-practice mechanism, Roediger & Karpicke) hides real gaps and drives authors toward needless edits.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py update CHECKPOINT_RE to match both "## ✅ Checkpoint" and "## :white_check_mark: Checkpoint" (e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint"), and update the has_prereq_section regex to match both "## 📋 Before You Start" and "## :clipboard: Before You Start" as well as "## Prerequisites". Add a unit test for each form. Re-run the assessment and confirm checkpoint_quality and scaffolding are no longer uniformly 0.0/5.0 and that the `missing_checkpoint` flag only fires on pages genuinely lacking a checkpoint.

Expected Score After Fix: ~8.0 / 10.0 corpus mean

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~44 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷

  • expires on Nov 3, 2026, 3:33 AM UTC
Dominant language
JavaScript
Stars
56
Forks
40
Avg merge
2d 14h
Merged PRs (30d)
13

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

  • No Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-aw-workshop

All issues in githubnext/gh-aw-workshop

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.