[curriculum-eval] curriculum_assessment.py: checkpoint_quality & scaffolding — regexes miss :white_check_mark: / :clipboard: headings

Open Beginner friendly
#5,966 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, markdown, python
Domain
tooling

Research direction

Start in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and look at CHECKPOINT_RE and the prerequisite regex (has_prereq_section). Confirm the miss by running the assessment over workshop/04-github-actions-intro.md, which uses ## :white_check_mark: Checkpoint. Done when has_checkpoint is true for that page, the shortcode forms are matched alongside the emoji ones, and the corpus mean rises. Update any tests or docs in the same skill directory.

Written by the indexing model from the issue text.

Description

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: 6.1 / 10.0 (corpus mean: 6.06) — affects 93 of 95 pages

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 (on 93 pages) ≥ 4 items → 10 -10.0
scaffolding 5.0 (on ~88 pages) section present → 10 -5.0

Root Cause (≤ 2 sentences):
CHECKPOINT_RE matches the literal ✅ glyph and the prerequisite regex matches 📋, but the workshop headings use the GitHub shortcodes :white_check_mark: and :clipboard:. Every page is therefore scored has_checkpoint: false and without scaffolding, so the dashboards report false missing_checkpoint flags and understate scores by ~2–3.5 points.

Evidence (quoted from the file):

CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) ... re.search(r"##\s+📋\s*Before You Start", raw, re.IGNORECASE)

versus workshop/04-github-actions-intro.md: ## :white_check_mark: Checkpoint (it has 5 checklist items, yet scored checkpoint_quality: 0.0).

Learning Science Rationale:
Valid measurement is a precondition for acting on rubric output. Checkpoints support retrieval practice and formative feedback (Roediger & Karpicke's testing effect), and scaffolding supports Vygotsky-style guided progression. When the tool reports these as absent everywhere, genuine gaps cannot be separated from artefacts, and improvement agents waste effort adding duplicate checkpoints.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both the emoji and shortcode forms, e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", and update the prerequisite/scaffolding regex (has_prereq_section) to r"##\s+(?:📋|:clipboard:)\s*(?:Before You Start|Prerequisites)". Also check any other emoji-based regexes in the file (e.g. :pencil2:, :dart:) for the same mismatch. Then re-run the assessment over workshop/*.md and confirm has_checkpoint is true for pages containing "## :white_check_mark: Checkpoint" and that corpus mean_score rises. Update any tests or docs under .github/skills/curriculum-quantitative-assessment/ accordingly.

Expected Score After Fix: 8.3 / 10.0 (estimated corpus mean once checkpoint and scaffolding are detected)

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~39 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷

  • expires on Nov 22, 2026, 3:31 AM UTC
Dominant language
JavaScript
Stars
61
Forks
43
Avg merge
10h 38m
Merged PRs (30d)
12

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

  • No Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-aw-workshop

All issues in githubnext/gh-aw-workshop

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.