[curriculum-eval] curriculum_assessment.py: checkpoint_quality and scaffolding — regexes miss :white_check_mark: shortcode, 92/94 pages mis-scored
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
首先阅读 .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py 中的 CHECKPOINT_RE 和 has_prereq_section 检测逻辑，然后找到此评估现有的测试。为包含 emoji 和 shortcode 的标题分别添加测试，并针对 workshop/ 运行评估。当两种标题形式都能被检测到、受影响页面的 checkpoint 得分正确，并且报告了更新后的各部分平均分时，即为完成。
由索引模型根据 Issue 内容生成。
描述
File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: n/a (rubric tooling defect; corpus mean 6.06)
Flagged Dimensions:
| Dimension | Score | Benchmark | Delta |
|---|---|---|---|
| checkpoint_quality | 0.0 on 92/94 pages (corpus mean 0.21) | ≥ 2.5 per checklist item, 10 at 4+ items | −10 per affected page |
| scaffolding (suspected) | 5.0 on flagged pages despite a "Before You Start" heading | 10 when present | −5 |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) only matches the literal emoji, but workshop files use the :white_check_mark: shortcode (## :white_check_mark: Checkpoint), so has_checkpoint is false for 92 of 94 pages. The same shortcode-vs-emoji mismatch probably affects the :clipboard: Before You Start scaffolding detection.
Evidence (quoted from the file):
workshop/05-agentic-workflows-intro.mdline 120:## :white_check_mark: Checkpointfollowed by 6 checklist items — yet the report sayshas_checkpoint: false, checkpoint_quality: 0.0.
Learning Science Rationale:
Measurement validity matters for any rubric-driven process: a systematic false negative hides real signal (such as the actual length and concept-load problems) and pushes agents to "fix" checkpoints that already exist. The history also shows side-quest-11-09 and side-quest-12-01 each dropping exactly 2.00 points, which matches losing the checkpoint dimension, and which points to a content-format change rather than a quality regression.
Improvement Prompt (for an agent):
In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE so it matches both the emoji and the GitHub shortcode forms, e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint". Check the regex that detects the "Before You Start" / Prerequisites section (has_prereq_section) and make it accept both "📋" and ":clipboard:" forms. Add unit tests with sample markdown for both forms. Re-run the assessment against workshop/ and confirm has_checkpoint is true for pages with a checkpoint heading, then report the new per-part mean scores.
Expected Score After Fix: corpus mean ≈ 7.5 / 10.0 (up from 6.06; most pages gain roughly +1.5 to +2.0 from the checkpoint dimension)
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~42 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷
- expires on Nov 1, 2026, 5:04 AM UTC
- 主要语言
- JavaScript
- 星标
- 56
- 派生
- 40
- 平均合并
- 2 天 15 小时
- 30 天内合并 PR
- 14
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
githubnext/gh-aw-workshop 的其他 Issue
-
documentation
难度 2/5 1-3 小时 新手友好度 85/100
githubnext/gh-aw-workshop#5193 ·
维护者通常 1 天内回复
-
documentation
难度 2/5 1-3 小时 新手友好度 86/100
githubnext/gh-aw-workshop#5194 ·
维护者通常 1 天内回复
-
[workshop-sync] workshop/side-quest-17-05-supply-chain-mcp.md: verify `network.allowed` link anchors未关闭documentation
难度 1/5 1 小时以内 新手友好度 82/100
githubnext/gh-aw-workshop#5192 ·
维护者通常 1 天内回复
-
curriculum documentation quality
难度 2/5 1-3 小时 新手友好度 82/100
githubnext/gh-aw-workshop#5182 ·
维护者通常 1 天内回复
-
curriculum documentation quality
难度 2/5 1-3 小时 新手友好度 78/100
githubnext/gh-aw-workshop#5183 ·
维护者通常 1 天内回复
查看 githubnext/gh-aw-workshop 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 85/100
pigeon189/zcode-fox-widget#17 · 1 条评论 ·
-
难度 1/5 1 小时以内 新手友好度 88/100
维护者通常 1 天内回复
-
Let TypeScript accept an initial object in ReactiveDict可能已有人在做 @yograj-vijay-yadav 今天认领。 未关闭bug good first issue help wanted workshop
难度 2/5 1-3 小时 新手友好度 92/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
akitaonrails/FrankMD#268 ·
维护者通常 1 天内回复
-
good first issue
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 1 天内回复