Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[curriculum-eval] .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py: checkpoint_quality, scaffolding — regex misses emoji

Open Beginner friendly
  • Is something wrong?
#3,810 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
86/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing-qa, tooling

Research direction

Start in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py by reading CHECKPOINT_RE and the has_prereq_section logic in collect_metrics_from_text. Re-run the rubric against workshop/*.md and spot-check 04-github-actions-intro.md, 05-agentic-workflows-intro.md, and side-quest-17-07-repo-poisoning.md; done means both emoji glyph and shortcode headings are detected without changing other scoring logic.

Written by the indexing model from the issue text.

Description

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (corpus-wide impact: 90–93 of 93 workshop files)
Overall Score: N/A — systemic tooling defect, not a single-page score
Corpus Mean: 6.09 / 10.0 (artificially depressed by this bug)

Flagged Dimensions (aggregate impact):

Dimension Score (current) Benchmark Delta
checkpoint_quality 0.0 for 90/93 files 10.0 if ≥4 checklist items -10.0 (should be ~7.5–10 for most files)
scaffolding 5.0 for 93/93 files 10.0 if prereq/before-you-start section present -5.0 (should be 10.0 for ~86 files)

Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", ...) and the prerequisite-section regex r"##\s+📋\s*Before You Start" only match literal emoji glyphs, but this workshop's actual convention (used on nearly every page) is GitHub-flavored Markdown emoji shortcodes: ## :white_check_mark: Checkpoint and ## :clipboard: Before You Start. Because the regexes never match the shortcode form, has_checkpoint is False for 90 of 93 files and has_prereq_section is False for all 93 files, even though the content is present and well-formed.

Evidence (quoted from the file):

workshop/04-github-actions-intro.md contains ## :clipboard: Before You Start and ## :white_check_mark: Checkpoint with 5 checklist items — yet rubric-results.json reports "has_checkpoint": false, "has_prereq_section": false, "checkpoint_quality": 0.0, "scaffolding": 5.0" for this file. The same pattern repeats for side-quest-17-07-repo-poisoning.md, side-quest-11-06-anthropic-key.md, 05-agentic-workflows-intro.md, and 86 other files.

Learning Science Rationale:
This is not a learning-science issue but a measurement-validity issue: per classical test theory, a rubric that systematically fails to detect a construct (checkpoints, scaffolding) it claims to measure produces scores with poor construct validity, making cross-file and cross-run comparisons (including the trend analysis in score-history.json) unreliable. Two of six weighted dimensions (weight 2.0 and 1.5 of 10.5 total, ≈33% of the composite) are affected, so this single defect can shift overall_score by roughly 0.5–1.1 points per file — enough to flip several files across the "below threshold" line used to select findings.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update the
regex patterns used to detect checkpoints and prerequisite sections so they also match
GitHub emoji-shortcode headings, not just literal emoji glyphs. Specifically:

1. Update CHECKPOINT_RE so it matches both:
   - "## ✅ Checkpoint" (literal emoji)
   - "## :white_check_mark: Checkpoint" (shortcode)
   Example: CHECKPOINT_RE = re.compile(
       r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", re.IGNORECASE)

2. Update the has_prereq_section detection in collect_metrics_from_text so it matches both:
   - "## 📋 Before You Start" (literal emoji)
   - "## :clipboard: Before You Start" (shortcode)
   - "## Prerequisites" (already handled)
   Example:
       re.search(r"##\s+(?:📋|:clipboard:)\s*Before You Start", raw, re.IGNORECASE)
       or re.search(r"##\s+Prerequisites", raw, re.IGNORECASE)

3. Re-run the rubric against workshop/*.md and confirm has_checkpoint and
   has_prereq_section now report True for files that use the :white_check_mark: and
   :clipboard: shortcode headings (spot-check 04-github-actions-intro.md,
   05-agentic-workflows-intro.md, and side-quest-17-07-repo-poisoning.md).

4. Do not change any other scoring logic, weights, or thresholds in this file.

Expected Score After Fix: Corpus mean rises from 6.09 / 10.0 toward an estimated 7.0–7.5 / 10.0, since ~90 files gain full scaffolding credit (5.0 → 10.0) and most gain checkpoint_quality credit (0.0 → 7.5–10.0 based on their existing 4+ checklist items.

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 90.7 AIC · ⌖ 21.1 AIC · ⊞ 9K · ◷

  • expires on Sep 26, 2026, 7:53 AM UTC
Dominant language
JavaScript
Stars
49
Forks
20
Avg merge
7h 29m
Merged PRs (30d)
35

Getting set up

We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-aw-workshop

All issues in githubnext/gh-aw-workshop

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.