[workshop-sim] Workshop Simulation Report — 2026-11-21 (Run #252000, 1000×Monte Carlo)

Abierto
#5,975 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
20/100
Tipo de issue
Documentación
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
github, github-actions

Línea de trabajo

This is a generated simulation report, not a task. The lesson pages it cites are 07-your-first-workflow.md, 04-github-actions-intro.md, 05-agentic-workflows-intro.md and 08-run-your-workflow.md, and the checkpoint rubric it blames is in the same repo. Before editing anything, check whether the ## Checkpoint sections are really missing or the rubric is mismatched, since the report itself flags that as unverified. Done would mean a maintainer has confirmed which repair applies and the change is scoped to specific lesson files.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

feedback simulation workshop
Overview
  • Date: 2026-11-21
  • Students simulated: 46 × 1000 Monte Carlo runs
  • Workshop steps available: 32/32
  • Overall success rate: 23.1% (95% Monte Carlo interval: 22.8%–23.5%)
  • Highest-dropout step: 07-first-workflow (22.7% conditional dropout among 22,198 at-risk runs; 95% Monte Carlo interval: 22.2%–23.3%)
  • Lowest curriculum quality step: 04-github-actions-intro.md (overall score 5.39/10; tied with 05-agentic-workflows-intro.md)
  • Learning KPI index: 2.9/10 (active_learning 4.2 · checkpoint_quality 0.0 · scaffolding 5.0)
  • Model: 2026-07-survival-model-v2 / 2026-07-assumption-model-v2 (parameter hash 2024391902)
  • Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Note: the baseline replay showed 08-run-your-workflow at 100% dropout (workflow-not-compiled). After semantic evaluation of Step 7/7d (source, compile, push, and both billing paths are covered with evidence), that artifact disappeared; Step 8 is no longer a top hotspot. Selected steps by semantic risk: 08-run-your-workflow, 07-first-workflow, 05-agentic-intro (all insights verified, hashes match).

Part Summary
Part Files Mean Score Std Dev
Part 1 — core path (lessons 00–14) 15 6.70 / 10.0 ±1.74
Part 2 — advanced (lessons 15+) 17 6.03 / 10.0 ±0.35
Overall corpus 32 6.35 / 10.0 —

No pages are classified as other.

Critical Findings
  1. Part 1 early-concept steps dominate dropout: 04-actions-intro (16.4%), 05-agentic-intro (18.6%), 05c (12.7%), 05b (9.9%) — mostly beginner and github-basic learners (beginner mean success 0.5%, github-basic 15.4%).
  2. Step 7 (22.7%) is the top single-step dropout; 3,897 of 5,048 failures are copilot-access-missing. Part 1 repair.
  3. Learning quality health is weak: KPI index 2.9/10, driven by checkpoint_quality = 0.0 on every step (the rubric detects no checkpoints even where pages have ## Checkpoint sections — likely a rubric/format mismatch worth verifying) and scaffolding flat at 5.0.
  4. UI-preferred learners succeed far less (11.8% vs 33.6%); tool: CCA learners 8.4% vs CLI 35.5% — the Codespace terminal transition is the largest preference-driven gap.
Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index.

  1. Step 7: unify the Copilot access check into a single gate and make the billing decision a guided choice (completion ↑ · learning KPI ↑)
  2. Steps 4–5: split concept density with prediction/practice checkpoints (completion ↑ · learning KPI ↑)
  3. Step 5: remove the "open any .lock.yml" assumption before a lock file exists; add real checkpoints (completion ↑ · learning KPI ↑)
Dropout by step
Step At-risk runs Dropouts Rate 95% interval Mode Top reason
07-first-workflow 22,198 5,048 22.7% 22.2–23.3% access barrier Copilot model access missing
05-agentic-intro 36,948 6,864 18.6% 18.2–19.0% learning barrier Agentic concept gap
04-actions-intro 44,197 7,249 16.4% 16.1–16.7% learning barrier Concept overload
05c-agentic-practice 30,084 3,815 12.7% 12.3–13.1% learning barrier Cannot classify agentic vs standard tasks
05b-agentic-security 26,269 2,604 9.9% 9.6–10.3% learning barrier Security concept gap
06-install-gh-aw 23,665 1,467 6.2% 5.9–6.5% access barrier Extension install friction
17-add-mcp-tools 14,753 694 4.7% 4.4–5.1% learning barrier MCP tooling friction
18-share-and-reuse 14,059 563 4.0% 3.7–4.3% learning barrier Workflow reuse friction
Curriculum quality and learning KPIs
File Overall Active Checkpoint Scaffold KPI Lowest dim Priority
04-github-actions-intro.md 5.39 3.9 0.0 5.0 2.8 checkpoint_quality High
05-agentic-workflows-intro.md 5.39 2.3 0.0 5.0 2.3 checkpoint_quality High
05b-agentic-workflows-security.md 5.75 2.5 0.0 5.0 2.3 checkpoint_quality Medium
05c-agentic-workflows-practice.md 6.22 6.2 0.0 5.0 3.6 checkpoint_quality Medium
07-your-first-workflow.md 6.25 6.3 0.0 5.0 3.7 checkpoint_quality High
07d-confirm-model-access.md 6.51 6.3 0.0 5.0 3.7 checkpoint_quality High
08-run-your-workflow.md 5.67 3.0 0.0 5.0 2.4 checkpoint_quality Medium
06-install-gh-aw.md 10.0 3.5 0.0 5.0 2.6 checkpoint_quality Low
Cohort mean 6.35 4.2 0.0 5.0 2.9 checkpoint_quality —
Segment breakdowns
Level Mean success n
beginner 0.5% 11
github-basic 15.4% 19
actions-user 47.2% 11
advanced 49.5% 5
Personality Mean success n
impatient 26.6% 6
methodical 26.2% 12
skeptical 22.8% 7
confused 21.1% 6
curious 20.2% 15

UI preferred: true 11.8% (22) vs false 33.6% (24). Tool: cli 35.5%, vscode 30.1%, CCA 8.4%.

Notable student journeys (3)
  • Surprising success: Learner 004 (actions-user, backend-dev, vscode, UI-preferred) reaches 62% despite UI preference — Actions familiarity offsets the terminal transition; its main failure is still Step 7.
  • Unexpected dropout: Learner 024 (advanced, devops) tops out at 59.6% and fails at Step 7 — expertise doesn't remove model-access/billing friction.
  • Content gap: Learner 001 (github-basic, UI-preferred, vscode) 0% — drops at Step 4 from concept overload, with no checkpoint to consolidate before Step 5.

Generated by 🔬 Workshop Student Simulator · copilot · auto · ~38 AIC · ⌖ ~1 AIC · ⊞ 15.4K · ◷

  • expires on Nov 22, 2026, 11:03 AM UTC
Lenguaje dominante
JavaScript
Estrellas
61
Forks
43
Merge medio
10 h 38 min
PR fusionados (30 d)
12

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

  • Sin Dockerfile ni archivo de Docker Compose
  • Sin plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de githubnext/gh-aw-workshop

Todos los issues de githubnext/gh-aw-workshop

Issues similares

Más issues de JavaScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.