[workshop-sim] Workshop Simulation Report — 2026-11-09 (Run #187, 1000×Monte Carlo)

Aperta
  • Qualcosa non va?
#5,520 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
github-actions, markdown
Ambito
content, documentation

Direzione di ricerca

The report names 04-github-actions-intro.md, 05-agentic-workflows-intro.md, 07-your-first-workflow.md, 07d-confirm-model-access.md, and 08-run-your-workflow.md as priority areas. Read these workshop lessons and their checkpoints first, then compare them with the reported dropout causes and learning scores. Done means applying the prioritized repairs: inline the billing decision and verification in Step 7d, add predict-then-verify exercises to Steps 4 and 5, and add pre-flight checks to Step 8 while maintaining or improving the learning KPI.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

feedback simulation workshop
Overview
  • Date: 2026-11-09
  • Students simulated: 46 × 1000 Monte Carlo runs
  • Workshop steps available: 31/31
  • Overall success rate: 23.5% (95% Monte Carlo interval: 23.1%–23.9%)
  • Highest-dropout step: 07-first-workflow (21.8% conditional dropout among 22,478 at-risk runs; 95% Monte Carlo interval: 21.3%–22.3%)
  • Lowest curriculum quality step: 04-github-actions-intro.md (overall score 5.39/10)
  • Learning KPI index: 2.9/10 (active_learning 4.15 · checkpoint_quality 0.00 · scaffolding 5.00)
  • Model: 2026-07-survival-model-v2 / 2026-07-assumption-model-v2 (parameter hash 2024391902)
  • Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty
Part Summary
Part Files Mean Score Std Dev
Part 1 — core path (lessons 00–14) 15 6.69 / 10.0 ±1.74
Part 2 — advanced (lessons 15+) 16 6.04 / 10.0 ±0.36
Overall corpus 31 6.35 / 10.0 —
Critical Findings
  1. Step 7 (Part 1) is the top blocker: 21.8% conditional dropout, mostly copilot-access-missing (3,793 of 4,901 failures). It combines the heaviest terminal and auth demand with billing choice deferred to a side quest.
  2. Concept pages 04/05/05b/05c (Part 1) lose 10–18% each, driven by concept overload and agentic concept gaps. These are learning barriers, and 04 and 05 have the lowest active_learning (3.9, 2.4).
  3. Learning quality is weak: checkpoint_quality is 0.0 on every main step, so the KPI index is 2.9/10. Learners who stay get practice but little verifiable feedback.
  4. Beginners (0.4%) and UI-preferring learners (11.8% vs 34.3%) almost never finish. The most important repair belongs to Part 1 (00–14).
  5. Step 8 pre-flight checks cover compile and publish state (the baseline shows 100% workflow-not-compiled for Step 8, which the Step 7 evaluations resolve). Step 8's checkpoint has no pre-flight item.
Top Repairs to Prioritize

Note: some dropout is expected. Repairs must maintain or improve the learning KPI index.

  1. Move the billing-path decision into Step 7d inline, with a verify-it-worked check (completion ↑ · learning KPI ↑)
  2. Add checkpoint quality and a predict-then-verify exercise to the Step 4 and 5 concept pages (completion ↑ · learning KPI ↑)
  3. Add pre-flight verification items to the Step 8 checkpoint (completion ↑ · learning KPI ↑)
Dropout by step
Step At-risk runs Dropouts Rate 95% interval Mode Top reason
07-first-workflow 22,478 4,901 21.8% 21.3%–22.3% access barrier Copilot model access missing
05-agentic-intro 36,969 6,566 17.8% 17.4%–18.2% learning barrier Agentic concept gap
04-actions-intro 44,151 7,182 16.3% 15.9%–16.6% learning barrier Concept overload
05c-agentic-practice 30,403 3,803 12.5% 12.1%–12.9% learning barrier Agentic classification gap
05b-agentic-security 26,600 2,615 9.8% 9.5%–10.2% learning barrier Agentic security gap
06-install-gh-aw 23,985 1,507 6.3% 6.0%–6.6% access barrier Extension install friction
Curriculum quality and learning KPIs
Step file Overall active_learning checkpoint_quality scaffolding KPI index Lowest dim Priority
04-github-actions-intro.md 5.39 3.9 0.0 5.0 2.69 checkpoint_quality High
05-agentic-workflows-intro.md 5.43 2.4 0.0 5.0 2.30 checkpoint_quality High
07-your-first-workflow.md 6.25 6.3 0.0 5.0 3.67 checkpoint_quality High
07d-confirm-model-access.md 6.25 6.3 0.0 5.0 3.67 checkpoint_quality High
08-run-your-workflow.md 5.67 3.0 0.0 5.0 2.43 checkpoint_quality Medium
Cohort mean 6.35 4.15 0.00 5.00 2.87 checkpoint_quality —
Segment breakdowns
Level n Success
beginner 11 0.4%
github-basic 19 15.9%
actions-user 11 47.7%
advanced 5 50.2%
Personality n Success
impatient 6 27.1%
methodical 12 25.6%
skeptical 7 24.1%
confused 6 22.5%
curious 15 20.6%

UI preferred: true (22) 11.8% · false (24) 34.3%.

Notable student journeys (3)
  • Best success: Learner 026 (advanced, devops, confused, CLI) reaches only 63%, and her most common failure is Step 7 despite the strongest profile. This shows that Step 7's access and billing friction hits experts too.
  • Unexpected dropout: Learner 043 (github-basic program manager, CCA, UI-preferring) fails at 04-actions-intro in every run, before any terminal step. This is concept overload, not tooling.
  • Content gap: Learner 038 (beginner, no-coding, methodical) fails at Step 4 every run. Step 4 has 28 new concepts and a 0.0 checkpoint score.

Agent insights were written for 08-run-your-workflow, 07-first-workflow, and 05-agentic-intro, and the simulator was rerun. All three show matching content hashes and semantic scores. Profiles were updated (+1000 runs each).

Generated by 🔬 Workshop Student Simulator · copilot · auto · ~36 AIC · ⌖ ~1 AIC · ⊞ 15.4K · ◷

  • expires on Nov 10, 2026, 9:04 AM UTC
Lingua principale
JavaScript
Stelle
58
Fork
40
Merge medio
1g 22h
PR unite (30g)
7

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

  • Nessun Dockerfile né file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di githubnext/gh-aw-workshop

Tutte le issue di githubnext/gh-aw-workshop

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.