[curriculum-eval] curriculum_assessment.py: checkpoint_quality and scaffolding — regexes miss :shortcode: headings

Open Beginner friendly
  • Is something wrong?
#5,297 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, markdown, python
Domain
tooling

Research direction

Start in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and find CHECKPOINT_RE and the Before-You-Start regex, which match only literal emoji. Extend both to accept the ✅ and 📋 shortcodes, including the bloom 'activity:checkpoint' detection. Done when workshop/04-github-actions-intro.md scores checkpoint_quality ≥ 10 and scaffolding 10 on a rerun against workshop/.

Written by the indexing model from the issue text.

Description

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: n/a (rubric tooling; corpus mean 6.06)

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 (92 of 94 files) ≥ 7.5 ≈ −7.5
scaffolding 5.0 (understated) 10 when present −5

Root Cause (≤ 2 sentences):
CHECKPOINT_RE = r"##\s+✅\s*Checkpoint" and the Before-You-Start regex ##\s+📋\s*Before You Start match literal emoji, but workshop files use :white_check_mark: and :clipboard: shortcodes. Every checkpoint and scaffolding section is therefore reported as missing, which drags down scores and produces false missing_checkpoint findings.

Evidence (quoted from the file):

workshop/04-github-actions-intro.md: ## :white_check_mark: Checkpoint followed by 5 checklist items, and ## :clipboard: Before You Start — yet scored checkpoint_quality: 0.0, scaffolding: 5.0.

Learning Science Rationale:
Valid measurement is a precondition for acting on scores. Formative-assessment research (Black & Wiliam) shows checkpoints support retrieval practice and self-monitoring. A rubric that reports them as absent misdirects authors toward adding content that already exists and hides real gaps.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE and the has_prereq_section regex so they match both the literal emoji (✅, 📋) and GitHub shortcodes (:white_check_mark:, :clipboard:). Example: r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint". Also update the bloom 'activity:checkpoint' detection that uses CHECKPOINT_RE. Add/adjust tests if present, rerun the assessment on workshop/, and confirm 04-github-actions-intro.md now gets checkpoint_quality ≥ 10 and scaffolding = 10.

Expected Score After Fix: ~7.0 / 10.0 corpus mean (from 6.06)

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~31 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷

  • expires on Nov 4, 2026, 6:03 AM UTC
Dominant language
JavaScript
Stars
56
Forks
40
Avg merge
2d 14h
Merged PRs (30d)
13

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

  • No Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-aw-workshop

All issues in githubnext/gh-aw-workshop

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.