[curriculum-eval] curriculum_assessment.py: checkpoint_quality — detector misses `:white_check_mark:` headings, 92/94 false negatives

未关闭 适合新手
  • 哪里不对？
#5,582 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
84/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python
领域
testing-qa

调研方向

从.github/skills/curriculum-quantitative-assessment/curriculum_assessment.py开始，检查CHECKPOINT_RE，以及 issue 中提到的 checkpoint 项计数和activity:checkpoint解析。比较这些处理路径如何识别带 emoji 和 shortcode 的标题，然后对workshop/*.md运行评估。完成标准是能够检测 shortcode 标题，且其清单计数会影响checkpoint_quality；报告语料库平均值以及仍缺少 checkpoint 的页面。

由索引模型根据 Issue 内容生成。

描述

curriculum documentation quality

File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py (affects all workshop/*.md)
Overall Score: 6.06 / 10.0 corpus mean (understated)

Flagged Dimensions:

Dimension Score Benchmark Delta
checkpoint_quality 0.0 on 92 of 94 pages present, ≥ 4 items → 10 −10.0

Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) only matches the literal emoji, while workshop pages (per repo convention) use ## :white_check_mark: Checkpoint. All 92 pages with has_checkpoint: false actually contain that heading, so the missing_checkpoint flag and the 0 score (weight 2.0) are false.

Evidence (quoted from the file):

## :white_check_mark: Checkpoint (e.g. workshop/side-quest-01-02-environment-reference.md line 164, which lists 7 checklist items yet scores checkpoint_quality: 0.0)

Learning Science Rationale:
Valid measurement underpins any formative-assessment loop. Retrieval-practice and feedback research (e.g. Roediger & Karpicke's testing effect) values checkpoints as learner self-checks; a scorer that cannot see them sends authors to "fix" pages that are fine, and hides real checkpoint-quality differences (item count and specificity).

Improvement Prompt (for an agent):

In .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, update CHECKPOINT_RE so it matches both the emoji form and the GitHub shortcode form of the heading, e.g. r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint" (case-insensitive). Check any other place that parses the checkpoint section (checklist item counting, bloom "activity:checkpoint" cue) uses the same pattern. Re-run the assessment over workshop/*.md and confirm has_checkpoint is true for all pages that have a "## :white_check_mark: Checkpoint" heading, and that checkpoint_quality now reflects checklist-item counts. Report the new corpus mean and the pages that still have no checkpoint.

Expected Score After Fix: ~8.0 / 10.0 corpus mean (up to +2.0 per page with ≥ 4 checkpoint items)

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · ~47 AIC · ⌖ ~1 AIC · ⊞ 5K · ◷

  • expires on Nov 12, 2026, 3:32 AM UTC
主要语言
JavaScript
星标
58
派生
40
平均合并
1 天 22 小时
30 天内合并 PR
7

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

githubnext/gh-aw-workshop 的其他 Issue

查看 githubnext/gh-aw-workshop 的全部 Issue

相似的 Issue

更多 JavaScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。