Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[curriculum-eval] .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py: checkpoint_quality/scaffolding — emoji-only regex fa

Open Beginner friendly
  • Is something wrong?
#3,849 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing-qa, tooling

Research direction

Start in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and compare the two heading regexes with .github/workflows/guidelines.md. Check workshop/04-github-actions-intro.md, workshop/05-agentic-workflows-intro.md, and workshop/side-quest-01-02-environment-reference.md, then rerun the corpus metrics. Done means both emoji and shortcode headings are detected and a unit test or assertion covers both forms.

Written by the indexing model from the issue text.

Description

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: N/A — tooling defect, corpus-wide impact (affects 90–93 of 93 files)

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 (90/93 files) items × 2.5, capped at 10 −up to 10.0
scaffolding 5.0 (93/93 files) 10 if present −5.0

Root Cause (≤ 2 sentences):
CHECKPOINT_RE and the prerequisite-section regex in curriculum_assessment.py require the literal emoji glyphs ✅ and 📋, but every workshop file follows the repository's own style guide (.github/workflows/guidelines.md) and uses the GitHub-supported shortcodes :white_check_mark: and :clipboard: instead. As a result, the rubric almost never detects checkpoints or "Before You Start" sections even when they are present and well-formed.

Evidence (quoted from the file):

CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)
"has_prereq_section": bool(re.search(r"##\s+📋\s*Before You Start", raw, re.IGNORECASE) or re.search(r"##\s+Prerequisites", raw, re.IGNORECASE))

Verified against workshop/04-github-actions-intro.md, which contains ## :white_check_mark: Checkpoint and ## :clipboard: Before You Start — both regexes return False/None on this real file, and checkpoint_quality/scaffolding are miscalculated across nearly the whole corpus (90/93 files show has_checkpoint: false; 93/93 show scaffolding: 5.0, the "absent" default).

Learning Science Rationale:
This is not a pedagogical defect but a measurement-validity problem: per classical test theory, a rubric that systematically fails to detect a construct it is designed to measure produces scores with near-zero construct validity for that dimension, and any downstream Bloom's/cognitive-load conclusions drawn from checkpoint_quality or scaffolding are unreliable until the detector is fixed. Correcting the regex is a prerequisite for trusting any future finding tied to these two dimensions.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, fix two regexes that only match literal emoji glyphs instead of the GitHub shortcode form used throughout workshop/*.md per .github/workflows/guidelines.md:

1. CHECKPOINT_RE currently requires a literal "✅" character. Update it to also match the shortcode form ":white_check_mark:", e.g.:
   CHECKPOINT_RE = re.compile(r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", re.IGNORECASE)

2. The has_prereq_section detection currently requires a literal "📋" character. Update it to also match the shortcode form ":clipboard:", e.g.:
   re.search(r"##\s+(?:📋|:clipboard:)\s*Before You Start", raw, re.IGNORECASE)

After the fix, re-run the corpus metrics/rubric generation and confirm has_checkpoint and has_prereq_section are True for files that use the shortcode headings (spot check workshop/04-github-actions-intro.md, workshop/05-agentic-workflows-intro.md, and workshop/side-quest-01-02-environment-reference.md). Add a small unit test or assertion covering both the emoji and shortcode heading forms so this regression cannot reoccur.

Expected Score After Fix: Corpus mean checkpoint_quality and scaffolding scores are expected to rise substantially (simulated: the 5 lowest-scoring flagged files move from ~4.9–5.4 to ~7.7–8.2 / 10.0 once these two dimensions are correctly detected).

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 84.8 AIC · ⌖ 7.78 AIC · ⊞ 9K · ◷

  • expires on Sep 27, 2026, 8:49 AM UTC
Dominant language
JavaScript
Stars
49
Forks
20
Avg merge
7h 22m
Merged PRs (30d)
33

Getting set up

We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-aw-workshop

All issues in githubnext/gh-aw-workshop

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.