[workshop-sim] Workshop Simulation Report — 2026-10-25 (Run #106000, 1000×Monte Carlo)

已关闭
  • 哪里不对？
#4,974 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
45/100
Issue 类型
文档
描述清晰度
需要澄清
活跃度
活跃
技术栈
github-actions, markdown
领域
documentation

调研方向

Read the step files cited in the report (04-github-actions-intro.md, 05-agentic-workflows-intro.md, 07-your-first-workflow.md) and the template that sets copilot-requests: write. First map the reported drop-off causes to the page content, then apply the listed repairs. Done means the billing contradiction is merged and Steps 04–05 gain hands-on practice with verifiable checkpoints.

由索引模型根据 Issue 内容生成。

描述

feedback simulation workshop
Overview
  • Date: 2026-10-25
  • Students simulated: 46 × 1000 Monte Carlo runs
  • Workshop steps available: 31/31
  • Overall success rate: 22.3% (95% Monte Carlo interval: 21.9%–22.6%)
  • Highest-dropout step: 07-first-workflow (26.6% conditional dropout among 22,399 at-risk runs; 95% interval: 26.1%–27.2%). Part 1.
  • Lowest curriculum quality step: 04-github-actions-intro.md (overall score 5.4/10)
  • Learning KPI index: 2.9/10 (active_learning 4.2 · checkpoint_quality 0.0 · scaffolding 5.0)
  • Model: 2026-07-survival-model-v2 / 2026-07-assumption-model-v2 (parameter hash 2024391902)
  • Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty. Rates are not observed human outcomes.
Part Summary
Part Files Mean Score Std Dev
Part 1 — core path (lessons 00–14) 15 6.69 / 10.0 ±1.74
Part 2 — advanced (lessons 15+) 16 6.04 / 10.0 ±0.36
Overall corpus 31 6.35 / 10.0 —

No pages are classified as other.

Critical Findings
  1. Nearly all attrition is in Part 1 (steps 04–07). Step 07 (Write Your First Workflow, incl. 07d model access) has the highest conditional dropout (26.6%), driven by Copilot access/billing friction (3,693 failures) and workflow-authoring friction (2,273). The most important repair belongs to Part 1.
  2. Step 04 and Step 05 are the largest absolute drop-off sources (7,215 and 6,564 dropouts) from concept overload and concept gaps; they are also the lowest-scoring pages (5.39, 5.43) with active_learning 3.9 and 2.4. Beginners (0.6% success) and CCA-tool learners (8.4%) are hit hardest; UI-preferring learners average 11.3% vs 32.3%.
  3. Learning quality health is weak: KPI index 2.9/10. checkpoint_quality is 0.0 on every step even though pages end with a checkpoint section, so the rubric likely sees checkpoints as unverifiable; learners who stay are not getting measurable skill checks.
  4. Step 08 shows 1.8% conditional dropout after the agent-insight rerun (down from 100% under the baseline's missing-lock-file assumption); the evaluated Step 07 pages confirm create, compile, and push of both files for the terminal path.
Top Repairs to Prioritize

Note: some dropout is expected. Repairs must maintain or improve the learning KPI index.

  1. Resolve the Step 07/07d billing contradiction (template adds copilot-requests: write, personal billing must remove it) and merge the duplicated access check (completion ↑ · learning KPI ↔/↑)
  2. Add practice and verifiable checkpoints to Step 05 with a prerequisite recovery path (completion ↑ · learning KPI ↑)
  3. Add hands-on practice and a self-check to Step 04 to cut concept overload (completion ↑ · learning KPI ↑)
Dropout by step
Step At-risk runs Dropouts Rate 95% interval Failure mode Top reason
07-first-workflow 22,399 5,966 26.6% 26.1–27.2% access barrier Copilot access missing (3,693)
05-agentic-intro 36,929 6,564 17.8% 17.4–18.2% learning barrier Agentic concept gap (4,311)
04-actions-intro 44,144 7,215 16.3% 16.0–16.7% learning barrier Concept overload (7,215)
05c-agentic-practice 30,365 3,749 12.3% 12.0–12.7% learning barrier Cannot classify agentic workflows (3,749)
05b-agentic-security 26,616 2,642 9.9% 9.6–10.3% learning barrier Security concept gap (2,642)
06-install-gh-aw 23,974 1,575 6.6% 6.3–6.9% access barrier Extension install friction (1,575)
Curriculum quality and learning KPIs
Step file Overall active_learning checkpoint_quality scaffolding KPI index Lowest dimension Priority
04-github-actions-intro 5.39 3.9 0.0 5.0 2.78 checkpoint_quality High
05-agentic-workflows-intro 5.43 2.4 0.0 5.0 2.24 checkpoint_quality High
07-your-first-workflow 6.25 6.3 0.0 5.0 3.65 checkpoint_quality High (access)
08-run-your-workflow 5.67 3.0 0.0 5.0 2.45 checkpoint_quality Medium
Cohort mean (31 steps) 6.35 4.15 0.0 5.0 2.87 checkpoint_quality —
Segment breakdowns
Level Mean success n
beginner 0.6% 11
github-basic 14.8% 19
actions-user 45.5% 11
advanced 46.9% 5
Personality Mean success n
curious 19.4% 15
confused 21.5% 6
skeptical 22.7% 7
methodical 24.7% 12
impatient 24.8% 6

Tool: CLI 34.5%, VS Code 28.2%, CCA 8.4%. UI-preferred 11.3% vs terminal-preferred 32.3%.

Notable student journeys (3)
  • Surprising success: Learner 024 (advanced, devops, UI-preferred, vscode) reaches 58.2% despite preferring the UI, showing the Codespace terminal transition is manageable with strong background.
  • Unexpected dropout: Learner 038 (beginner, no-coding, methodical, CLI) succeeds 0% of runs, failing at Step 04 from concept overload; being methodical does not offset the dense refresher with no hands-on practice.
  • Content gap: Learner 043 (github-basic, program-manager, CCA, UI-preferred) fails at Step 04; the page gives no reduced-path or checkpoint for learners without YAML background.

Generated by 🔬 Workshop Student Simulator · copilot · auto · ~41 AIC · ⌖ ~1 AIC · ⊞ 15.4K · ◷

  • expires on Oct 26, 2026, 8:04 AM UTC
主要语言
JavaScript
星标
56
派生
40
平均合并
2 天 14 小时
30 天内合并 PR
13

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

githubnext/gh-aw-workshop 的其他 Issue

查看 githubnext/gh-aw-workshop 的全部 Issue

相似的 Issue

更多 JavaScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。