[curriculum-eval] curriculum_assessment.py: checkpoint_quality — regexes miss :white_check_mark: and :clipboard: headings (93/95 pages scored 0)

Open Beginner friendly
#5,779 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
tooling

Research direction

Start with .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and inspect CHECKPOINT_RE and the prerequisite section checks; compare their patterns with the heading examples in workshop/04-github-actions-intro.md. Run the assessment over workshop/*.md and check that checkpoint and prerequisite headings are detected as described. Done when shortcode and emoji heading forms are recognized and a focused test covers the shortcode variants; the issue says not to edit workshop files.

Written by the indexing model from the issue text.

Description

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: corpus mean 6.06 / 10.0 (measurement artifact)

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 on 93/95 pages ≥ 10 with ≥ 4 items −10
scaffolding 5.0 on 95/95 pages 10 −5

Root Cause (≤ 2 sentences):
CHECKPOINT_RE matches ##\s+✅\s*Checkpoint and the prerequisites check matches ##\s+📋\s*Before You Start, but workshop headings are written ## :white_check_mark: Checkpoint and ## :clipboard: Before You Start. Every page therefore reports has_checkpoint: false and has_prereq_section: false, costing roughly 2.75 points per page.

Evidence (quoted from the file):

CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) versus ## :white_check_mark: Checkpoint in workshop/04-github-actions-intro.md

Learning Science Rationale:
Metrics that do not discriminate real quality undermine feedback loops. Retrieval practice (Roediger & Karpicke) depends on checkpoints actually existing, and the evaluator cannot reward or detect them. A valid rubric is a precondition for any Bloom or Cognitive Load analysis, since the current uniform ~6.0 scores hide true differences between pages.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both the emoji and the GitHub shortcode form (e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint"), and update the has_prereq_section regexes to match "## 📋 Before You Start", "## :clipboard: Before You Start", and "## Prerequisites". Re-run the assessment over workshop/*.md and confirm has_checkpoint is true for all pages that contain a checkpoint heading (expected 93/95) and has_prereq_section is true for pages with a Before You Start heading. Check any existing tests still pass and add a small test for the shortcode variants. Do not edit workshop files.

Expected Score After Fix: corpus mean rises from 6.06 to approximately 8.0–8.8 / 10.0; the remaining low scorers will then reflect real length and active-learning gaps.

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~47 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷

  • expires on Nov 17, 2026, 3:38 AM UTC
Dominant language
JavaScript
Stars
62
Forks
43
Avg merge
11h 26m
Merged PRs (30d)
13

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

  • No Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-aw-workshop

All issues in githubnext/gh-aw-workshop

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.