[workshop-sim] Workshop Simulation Report — 2026-08-04 (Run #26, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-08-04
• Students simulated: 46 × 1000 Monte Carlo runs (46,000 virtual completions)
• Workshop steps available: 29 main steps + 61 side quests
• Overall success rate: 22.6% (95% Monte Carlo interval: 22.2%–23.0%)
• Highest-dropout step: 07-first-workflow (27.4% conditional dropout among 23,440 at-risk runs; 95% interval: 26.8%–27.9%)
• Lowest curriculum quality step: 07d-confirm-model-access.md (overall score 5.29/10)
• Learning KPI index: 2.89/10 (active_learning 4.20 · checkpoint_quality 0.00 · scaffolding 5.00)
• Model: 2026-07-survival-model-v2 / 2026-07-assumption-model-v2 (parameter hash 2024391902)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

---

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	15	6.70 / 10.0	±1.75
Part 2 — advanced (lessons 15+)	14	6.10 / 10.0	±0.33
Overall corpus	29	6.41 / 10.0	±1.20

No steps are classified as other. Part 2 scores are tightly clustered (lower variance) but sit below the overall corpus mean.

---

Critical Findings

1. Beginner and UI-preferred learners are effectively blocked. beginner-level students average a 0.5% success rate; ui_preferred: true students average 10.9% vs 33.3% for CLI-comfortable learners. All five top-dropout steps fall in Part 1 (core path, lessons 00–14), meaning most learners drop before reaching Part 2 content at all.

2. Step 07-first-workflow is the steepest single cliff (27.4% conditional dropout, Part 1). The content is well-structured but stacks Copilot CLI access verification, workflow authoring via agent, gh aw compile, billing-path decision, and a two-file commit/push into one linear pass. The top failure cause is copilot-access-missing, an access barrier that fires because many learners arrive at Step 7 without having successfully resolved model access. Agent-insight semantic scores: stateReadiness 62, pathClarity 55, recoverySupport 70.

3. Learning KPI index is 2.89/10 — driven entirely to zero by checkpoint_quality (0.00 corpus-wide). The rubric scores checkpoint_quality: 0.0 for every step, which mechanically pins the KPI regardless of active_learning or scaffolding improvements. This is a systemic measurement signal: every step has a markdown checklist but none satisfy the rubric's formal checkpoint criterion. Learners who persist are building skills (active_learning 4.20, scaffolding 5.00) but the rubric cannot confirm it.

4. The most impactful repairs belong to Part 1. Steps 04-actions-intro, 05-agentic-intro, 05c-agentic-practice, and 07-first-workflow together account for the majority of all dropouts. Fixing the access barrier at Step 7 and the concept-density barrier at Steps 4–5 would improve the overall completion rate more than any Part 2 change.

---

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Step 07: Add an explicit pre-flight gate that verifies Copilot model access before any authoring work begins (completion impact: ↑ · learning KPI impact: ↑). Currently copilot-access-missing is the Add Astro-based 90‑minute agentic workflows workshop kit #1 failure cause; moving the access-check to a blocking checkpoint before the workflow-authoring section removes an access barrier without changing learning content.

2. Step 04-actions-intro: Chunk the 25 new concepts into chunked micro-segments with inline self-check questions (completion impact: ↑ · learning KPI impact: ↑). The page has the second-lowest overall_score (5.42) and sole failure mode concept-overload. Breaking the intro into two or three focused sub-sections with short reveal-answer micro-assessments would raise both active_learning and checkpoint_quality without removing any content.

3. Step 05-agentic-intro: Add a mid-page knowledge consolidation checkpoint after the three-key-terms section (completion impact: ↑ · learning KPI impact: ↑). agentic-concept-gap accounts for 100% of failures here. The page already has three good activities; a consolidation checkpoint between the conceptual section and Activity 2 would anchor understanding before learners attempt the classification tasks.

---

Dropout by step

Step	At-risk runs	Dropouts	Conditional dropout	95% MC interval	Failure mode	Top reason
07-first-workflow	23,440	6,413	27.4%	26.8–27.9%	Access barrier	Copilot model access not resolved before workflow authoring
05-agentic-intro	39,681	6,502	16.4%	16.0–16.8%	Learning barrier	Too many new agentic concepts without sufficient scaffolding
05c-agentic-practice	33,179	4,715	14.2%	13.8–14.6%	Learning barrier	Classification task requires concept solidification not yet achieved
05b-agentic-security	28,464	3,091	10.9%	10.5–11.2%	Learning barrier	Security model gap — learners do not yet have mental model for safe outputs
04-actions-intro	44,120	4,439	10.1%	9.8–10.3%	Learning barrier	25 new concepts in one pass causes concept overload
06-install-gh-aw	25,373	1,933	7.6%	7.3–8.0%	Access barrier	Extension install friction (auth, path, version)
17-add-mcp-tools	14,300	710	5.0%	4.6–5.3%	Learning barrier	MCP tooling configuration complexity
15-conditional-logic	15,210	632	4.2%	3.8–4.5%	Learning barrier	Conditional logic pattern friction
19-research-driven-training-node	13,047	540	4.1%	3.8–4.5%	Learning barrier	Research node pattern friction
02-setup	46,000	1,880	4.1%	3.9–4.3%	Access barrier	Codespace setup friction
18-share-and-reuse	13,590	543	4.0%	3.7–4.3%	Learning barrier	Workflow reuse pattern friction
14b-pr-reviewer-workflow	15,789	579	3.7%	3.4–4.0%	Learning barrier	Event trigger configuration friction
09-agentic-editing	16,345	556	3.4%	3.1–3.7%	Learning barrier	Iterative editing workflow friction
24-self-hosted-runners	11,476	365	3.2%	2.9–3.5%	Access barrier	Self-hosted runner environment friction
08b-interpret-your-run	16,761	416	2.5%	2.3–2.7%	Learning barrier	Output interpretation gap
22-error-handling-and-resilience	11,964	290	2.4%	2.2–2.7%	Learning barrier	Resilience pattern friction
21-inline-sub-agents	12,246	282	2.3%	2.1–2.6%	Learning barrier	Sub-agent decomposition friction
20-persistent-memory	12,507	261	2.1%	1.9–2.4%	Learning barrier	Memory pattern friction
16-connect-data-source	14,578	278	1.9%	1.7–2.1%	Learning barrier	Data source integration friction
25-audit-and-observability	11,111	199	1.8%	1.6–2.1%	Learning barrier	Audit instrumentation friction
27-evaluate-workflow-quality	10,733	191	1.8%	1.5–2.0%	Learning barrier	Evaluation quality friction
23-ab-experiments	11,674	198	1.7%	1.5–1.9%	Learning barrier	Experiment design friction
26-manage-costs-and-budgets	10,912	179	1.6%	1.4–1.9%	Learning barrier	Cost guardrail configuration friction
08-run-your-workflow	17,027	266	1.6%	1.4–1.8%	Access barrier	UI run guidance gap (Actions tab)
28-orchestrate-workflows	10,542	144	1.4%	1.2–1.6%	Learning barrier	Orchestration pattern friction

Curriculum quality and learning KPIs

Step file	Overall	Active learning	Checkpoint quality	Scaffolding	Learning KPI	Lowest dim	Repair priority
07d-confirm-model-access.md	5.29	3.7	0.0	5.0	2.71	checkpoint_quality	High
04-github-actions-intro.md	5.42	4.6	0.0	5.0	3.04	checkpoint_quality	High
05-agentic-workflows-intro.md	5.43	2.4	0.0	5.0	2.24	checkpoint_quality	High
15-conditional-logic.md	5.53	3.8	0.0	5.0	2.75	checkpoint_quality	Medium
05b-agentic-workflows-security.md	5.75	2.5	0.0	5.0	2.27	checkpoint_quality	Medium
08-run-your-workflow.md	5.67	3.0	0.0	5.0	2.45	checkpoint_quality	Medium
14b-pr-reviewer-workflow.md	5.67	4.8	0.0	5.0	3.11	checkpoint_quality	Low
16-connect-data-source.md	5.71	3.8	0.0	5.0	2.75	checkpoint_quality	Low
17-add-mcp-tools.md	5.75	3.4	0.0	5.0	2.60	checkpoint_quality	Medium
20-persistent-memory.md	5.81	3.6	0.0	5.0	2.67	checkpoint_quality	Low
26-manage-costs-and-budgets.md	5.85	4.0	0.0	5.0	2.82	checkpoint_quality	Low
14-next-steps.md	5.91	3.3	0.0	5.0	2.56	checkpoint_quality	Low
21-inline-sub-agents.md	5.99	4.1	0.0	5.0	2.85	checkpoint_quality	Low
09-agentic-editing.md	6.03	4.8	0.0	5.0	3.11	checkpoint_quality	Low
08b-interpret-your-run.md	6.07	4.1	0.0	5.0	2.85	checkpoint_quality	Low
02a-setup-codespace.md	6.09	5.0	0.0	5.0	3.18	checkpoint_quality	Low
18-share-and-reuse.md	6.15	4.5	0.0	5.0	3.00	checkpoint_quality	Low
25-audit-and-observability.md	6.23	4.9	0.0	5.0	3.15	checkpoint_quality	Low
28-orchestrate-workflows.md	6.23	4.9	0.0	5.0	3.15	checkpoint_quality	Low
22-error-handling-and-resilience.md	6.27	5.1	0.0	5.0	3.22	checkpoint_quality	Low
19-research-driven-training-node.md	6.37	5.6	0.0	5.0	3.40	checkpoint_quality	Low
27-evaluate-workflow-quality.md	6.37	5.6	0.0	5.0	3.40	checkpoint_quality	Low
07-your-first-workflow.md	6.59	6.7	0.0	5.0	3.80	checkpoint_quality	High
05c-agentic-workflows-practice.md	6.59	6.7	0.0	5.0	3.80	checkpoint_quality	Medium
23-ab-experiments.md	6.53	6.4	0.0	5.0	3.69	checkpoint_quality	Low
24-self-hosted-runners.md	6.63	6.9	0.0	5.0	3.87	checkpoint_quality	Low
06-install-gh-aw.md	10.00	3.5	0.0	5.0	2.64	checkpoint_quality	Low
00-welcome.md	10.00	0.0	0.0	5.0	1.36	active_learning	N/A
01-prerequisites.md	10.00	0.0	0.0	5.0	1.36	active_learning	N/A
Cohort mean	6.41	4.20	0.00	5.00	2.89	checkpoint_quality	—

Segment breakdowns

Success rate by technical level

Level	Mean success rate	Students
beginner	0.5%	11
github-basic	14.7%	19
actions-user	47.2%	11
advanced	47.1%	5

Success rate by personality

Personality	Mean success rate	Students
impatient	27.0%	6
methodical	24.3%	12
skeptical	23.6%	7
confused	21.4%	6
curious	19.5%	15

Success rate by UI preference

UI preferred	Mean success rate	Students
CLI-comfortable (ui_preferred: false)	33.3%	24
UI-preferred (ui_preferred: true)	10.9%	22

Notable student journeys (3)

Surprising success — Learner 004 (actions-user, methodical, backend-dev, VS Code, ui_preferred: true, goal: work-project): 57.1% success rate. Despite preferring the GitHub UI over the terminal, this learner's methodical personality and prior Actions experience allowed them to navigate the Codespace terminal transition and billing configuration in Step 7 with higher-than-expected consistency. The clear step-by-step structure of 07d-confirm-model-access.md particularly helped a methodical reader who reads all instructions before acting.

Unexpected dropout — Learner 003 (github-basic, skeptical, program-manager, CCA, ui_preferred: true, goal: team-evaluation): 0% success rate across all 1,000 runs. Despite being a team-evaluation persona who should be motivated to persist, this learner hits the concept density of 04-actions-intro (concept-overload) and cannot bridge to the agentic model. The skeptical personality amplifies dropout at conceptual steps: without hands-on confirmation of claims, the learner disengages. The team-evaluation goal predicts early exit when blockers appear, consistent with 0% completion.

Content gap case — Learner 005 (github-basic, curious, backend-dev, VS Code, ui_preferred: true, goal: team-evaluation): 11.1% success rate; mostCommonFailureStep: 07-first-workflow (238/1000 failures). This learner has enough programming background to reach Step 7 relatively often (good concept absorption in Steps 4–5), but consistently fails at the Copilot billing configuration. The billing-path decision tree (centralized vs. personal) in 07d-confirm-model-access.md requires org-admin context that a team evaluator on a personal repository may not have. The content is present but the decision tree adds cognitive load at a moment when the learner is also managing a first-time compile-and-commit.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Workshop Student Simulator · 80.5 AIC · ⌖ 8.21 AIC · ⊞ 10.4K · ◷

• expires on Aug 5, 2026, 3:39 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #2497

View newer issue

---

This action was performed automatically by the Workshop Student Simulator workflow.