[curriculum-eval] curriculum_assessment.py: checkpoint_quality — emoji regex misses :white_check_mark: shortcodes (92/94 false negatives)

已关闭 适合新手
  • 哪里不对？
#5,392 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 3 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
78/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
github-actions, python
领域
testing

调研方向

从 .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py 开始，检查 CHECKPOINT_RE 和先决条件部分的正则表达式。将它们与 workshop/04-github-actions-intro.md 中的 shortcode 标题进行比较；当 emoji 标题和 shortcode 标题都能被检测到（包括列出的“Prerequisites”标题），且该 workshop 页面报告一个包含 5 个清单项的 checkpoint 时，即为完成。为这两种标题形式添加测试，并在 workshop/ 上重新运行评估。

由索引模型根据 Issue 内容生成。

描述

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (affects all workshop/*.md)
Overall Score: 4.9–5.4 / 10.0 for the 5 flagged pages (corpus mean: 6.06)

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 ≥ 10 with ≥ 4 items −10.0
scaffolding 5.0 10 −5.0

Root Cause (≤ 2 sentences):
CHECKPOINT_RE matches the literal ✅ and the prereq regex matches the literal 📋, but workshop files use the GitHub shortcodes :white_check_mark: and :clipboard:. As a result 92/94 pages report has_checkpoint: false and 92/94 report has_prereq_section: false, deflating every score and producing false "missing_checkpoint" findings (e.g. 04-github-actions-intro.md).

Evidence (quoted from the file):

CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) vs. in workshop/04-github-actions-intro.md: ## :white_check_mark: Checkpoint followed by 5 checklist items.

Learning Science Rationale:
Measurement validity: a rubric tied to a rendering detail instead of structure misguides prioritisation. Checkpoints support retrieval practice (Roediger & Karpicke), so the real content gaps must be separated from detector artefacts; otherwise effort is spent on pages that already comply, while true gaps (e.g. side-quest-01-02-environment-reference.md length 1433 words, side-quest-17-07-repo-poisoning.md 23 new concepts) are hidden among noise.

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE to match both "## ✅ Checkpoint" and "## :white_check_mark: Checkpoint", and update the has_prereq_section regex to match both "## 📋 Before You Start" and "## :clipboard: Before You Start" (plus "## Prerequisites"). Add unit tests covering both forms, re-run the assessment over workshop/, and confirm 04-github-actions-intro.md now reports has_checkpoint=true with 5 checklist items.

Expected Score After Fix: ~7.0 / 10.0 for the 5 flagged pages; corpus mean ≈ 7.5 / 10.0. After the fix, re-evaluate real gaps: side-quest-01-02-environment-reference.md (excessive length, low activity density 0.63), side-quest-17-07-repo-poisoning.md (23 concepts, 1234 words), side-quest-11-06-anthropic-key.md (26 concepts).

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~26 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷

  • expires on Nov 7, 2026, 3:30 AM UTC
主要语言
JavaScript
星标
58
派生
40
平均合并
4 天 4 小时
30 天内合并 PR
8

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

githubnext/gh-aw-workshop 的其他 Issue

查看 githubnext/gh-aw-workshop 的全部 Issue

相似的 Issue

更多 JavaScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。