mcp-improvements

Author: svincent240

framework/docs/FRAMEWORK_GUIDE.md

@@ -679,7 +679,12 @@ Runs **in Node.js** during build (via `vite.framework-dev.config.js` `closeBundl
 
 ### MCP `coursecode_lint` — Build-Time Only
 
-The MCP `coursecode_lint` tool runs the build linter (config validation, CSS class verification, structure checks). It does NOT include runtime errors. For runtime errors and contrast warnings, use `coursecode_errors` (lightweight — just errors and console logs) or `coursecode_state` (full state snapshot including errors).
+The MCP `coursecode_lint` tool runs the static/build-time linter (config validation, CSS class verification, structure checks). It does **not** inspect the running preview and does **not** include live runtime, browser console, or Vite build-watch diagnostics.
+
+Use:
+- `coursecode_lint` for static preflight validation after source/config edits
+- `coursecode_errors` for the live "what is broken right now?" preview rollup
+- `coursecode_state` when you also need current slide, TOC, engagement, LMS state, and diagnostics
 
 ### Shared Rules (`lib/validation-rules.js`)
 
@@ -861,16 +866,18 @@ The MCP server runs a **persistent headless Chrome** internally via `puppeteer-c
 
 ### Preview Server Ownership
 
-The MCP does **not** start or manage the preview server. The preview must be running before using runtime tools:
+The MCP does **not** start or manage the preview server. Runtime tools connect to an already-running preview server.
 
-- **Human**: run `coursecode preview` (or `npm run preview`) in a terminal
-- **AI agent**: use your terminal/command execution tool to run `npm run preview`
+- If preview is already running for the current project, use it. Do **not** start a second preview server.
+- If preview is not running, start it in a terminal with `coursecode preview`.
+- For framework development from this repo, use `npm run preview`.
+- AI agents may start preview only via their terminal/command execution tool, and only after confirming preview is not already running or after a runtime MCP tool reports that preview is not running.
 
 If the preview is not running, runtime tools fail fast with a clear error message.
 
 ### Setup
 
-1. Start the preview server externally (see above)
+1. Make sure the preview server is running externally (see above)
 2. Add to IDE MCP config:
 
 ```json
@@ -887,8 +894,8 @@ If the preview is not running, runtime tools fail fast with a clear error messag
 
 | Tool | Purpose | Returns |
 |------|---------|--------|
-| `coursecode_state` | Full course snapshot | `{slide, toc, interactions, engagement, lmsState, apiLog, errors, frameworkLogs, consoleLogs}` |
-| `coursecode_errors` | Errors + console logs only | `{errors, consoleLogs, count, clean}` — same error sources as `coursecode_state`, without the state payload |
+| `coursecode_state` | Full course snapshot + live diagnostics | `{slide, toc, interactions, engagement, lmsState, apiLog, diagnostics, issues, errors, frameworkLogs, consoleLogs}` |
+| `coursecode_errors` | Live diagnostic rollup only | `{build, runtime, framework, console, issues, errors, count, clean}` — same diagnostic sources as `coursecode_state`, without the state payload |
 | `coursecode_navigate` | Go to slide by ID | `{slide, interactions, engagement, accessibility}` |
 | `coursecode_interact` | Set response + evaluate | `{interactionId, response}` → `{correct, score, feedback}` |
 | `coursecode_screenshot` | Visual capture (JPEG) | Optional `slideId` to navigate first, `fullPage` for scroll capture |
@@ -933,7 +940,7 @@ MCP Server (IDE) ──puppeteer──▶ Headless Chrome ──HTTP──▶ Pr
                                     └── Course iframe (CourseCodeAutomation API)
 ```
 
-- **Preview not running?** → Tools return clear error: "Start preview server first"
+- **Preview not running?** → Tools return a clear error. Start preview externally in a terminal, then retry.
 - **Chrome not found?** → Install Google Chrome or set `CHROME_PATH` env var
 
 ### Pre-Release Responsive Checks (Framework)

framework/docs/USER_GUIDE.md

@@ -297,7 +297,8 @@ Once connected, your AI assistant gains these capabilities:
 | `coursecode_screenshot` | Take a screenshot of any slide |
 | `coursecode_interact` | Answer an interaction and check if it's correct |
 | `coursecode_reset` | Clear progress and restart the course |
-| `coursecode_lint` | Check for errors (bad CSS classes, missing components, config issues) |
+| `coursecode_errors` | Check live preview diagnostics (build, runtime, framework, and console issues) |
+| `coursecode_lint` | Run static preflight checks (bad CSS classes, missing components, config issues) |
 | `coursecode_component_catalog` | Browse available UI components (tabs, accordion, cards, etc.) |
 | `coursecode_interaction_catalog` | Browse available interaction types (multiple choice, drag-drop, etc.) |
 | `coursecode_css_catalog` | Browse available CSS classes by category |
@@ -306,7 +307,7 @@ Once connected, your AI assistant gains these capabilities:
 | `coursecode_workflow_status` | Get guidance on what to do next based on your project's current state |
 | `coursecode_build` | Build the course for LMS deployment |
 
-> **Note:** The preview server must be running before using runtime tools like `coursecode_state`, `coursecode_screenshot`, or `coursecode_navigate`. Start it with `coursecode preview` in a terminal.
+> **Note:** The preview server must be running before using runtime tools like `coursecode_state`, `coursecode_errors`, `coursecode_screenshot`, or `coursecode_navigate`. If preview is not already running for this project, start it with `coursecode preview` in a terminal. Do not start a second preview server if one is already running.
 
 ### How the Workflow Changes
 

framework/js/automation/api-interactions.js

@@ -6,7 +6,7 @@
 
 import interactionRegistry from '../managers/interaction-registry.js';
 import { courseConfig } from '../../../course/course-config.js';
-import { recordInteractionResult } from '../components/interactions/interaction-base.js';
+import { getInteractionState, recordInteractionResult } from '../components/interactions/interaction-base.js';
 import { logger } from '../utilities/logger.js';
 
 /**
@@ -18,11 +18,26 @@ export function createInteractionMethods(logTrace) {
     return {
         listInteractions() {
             const interactions = interactionRegistry.getAll();
-            const simplifiedList = interactions.map(i => ({
-                id: i.id,
-                type: i.type,
-                description: i.description
-            }));
+            const simplifiedList = interactions.map(i => {
+                const savedState = getInteractionState(i.id);
+                let response = savedState?.response;
+
+                if (response === undefined && typeof i.instance?.getResponse === 'function') {
+                    try {
+                        response = i.instance.getResponse();
+                    } catch {
+                        response = undefined;
+                    }
+                }
+
+                return {
+                    id: i.id,
+                    type: i.type,
+                    description: i.description,
+                    hasResponse: response !== null && response !== undefined && response !== '',
+                    isChecked: savedState?.submitted === true
+                };
+            });
             logTrace('listInteractions', { count: simplifiedList.length });
             return simplifiedList;
         },
@@ -33,10 +48,23 @@ export function createInteractionMethods(logTrace) {
                 throw new Error(`CourseCodeAutomation: Interaction "${interactionId}" not found on the current slide`);
             }
             logTrace('getInteractionMetadata', { interactionId });
+            const savedState = getInteractionState(entry.id);
+            let response = savedState?.response;
+
+            if (response === undefined && typeof entry.instance?.getResponse === 'function') {
+                try {
+                    response = entry.instance.getResponse();
+                } catch {
+                    response = undefined;
+                }
+            }
+
             return {
                 id: entry.id,
                 type: entry.type,
-                description: entry.description
+                description: entry.description,
+                hasResponse: response !== null && response !== undefined && response !== '',
+                isChecked: savedState?.submitted === true
             };
         },
 

lib/headless-browser.js

@@ -358,8 +358,8 @@ class HeadlessBrowser {
 
         // Navigate to specific slide if requested
         if (slideId) {
-            await this.evaluate((id) => {
-                window.CourseCodeAutomation.goToSlide(id);
+            await this.evaluate(async (id) => {
+                await window.CourseCodeAutomation.goToSlide(id);
             }, slideId);
             // Wait for slide transition
             await new Promise(resolve => setTimeout(resolve, 500));

lib/mcp-prompts.js

@@ -20,7 +20,7 @@ export const TOOLS = [
     // --- Runtime Tools (require headless browser) ---
     {
         name: 'coursecode_state',
-        description: `Get course state, runtime errors, and warnings in one call. This is the primary tool for checking errors.
+        description: `Get course state and live preview diagnostics in one call.
 
 Returns:
 - slide: current slide ID (string)
@@ -29,12 +29,15 @@ Returns:
 - engagement: slide engagement {complete, percentage, requirements}
 - lmsState: LMS data {score, completion, success, bookmark, format, objectives, state}
 - apiLog: last 20 LMS API calls [{timestamp, method, args, result}]
-- errors: runtime errors/warnings [{type, message, hint, isWarning}]
+- diagnostics: live issue rollup with build, runtime, framework, console, issues, count, clean
+- issues/errors: flat live issue list [{source, severity, isWarning, type, message, hint?}]
+- runtimeErrors: runtime/debug-panel errors and warnings only
+- buildErrors/buildWarnings: backward-compatible build-watch aliases
 - frameworkLogs: structured framework log events [{level, domain, operation, message, stack?, timestamp}]
 - consoleLogs: browser console warnings/errors [{type, text, time}]
 
 Use this first to understand the course state before taking actions.
-For error checking only (after file edits), prefer coursecode_errors — same error sources, smaller payload.
+For checking what is broken after file edits, prefer coursecode_errors — same live diagnostic sources, smaller payload.
 Requires preview server to be running.`,
         inputSchema: {
             type: 'object',
@@ -48,15 +51,23 @@ Requires preview server to be running.`,
     },
     {
         name: 'coursecode_errors',
-        description: `Get runtime errors and warnings from the live preview. Uses the same error sources as coursecode_state (preview server errors + browser console) but without the heavyweight state payload.
+        description: `Get all current live preview diagnostics without the heavyweight course state payload.
+
+This is the primary "what is broken right now?" tool after edits. It aggregates:
+- build: Vite/build-watch errors and warnings from the running preview server
+- runtime: stub LMS/debug-panel errors and warnings
+- framework: CourseCode logger warnings/errors
+- console: browser console warnings/errors
+
+This is different from coursecode_lint, which is a static/build-time linter and does not inspect the running preview.
 
 Returns:
-- errors: [{type, message, hint?, isWarning?}] — preview server errors and warnings
-- consoleLogs: [{type, text, time}] — browser console warnings/errors
-- count: total number of errors + console logs
-- clean: true if no errors, warnings, or console issues
+- build/runtime/framework/console: grouped diagnostics by source
+- issues/errors: flat list [{source, severity, isWarning, type, message, hint?}] for agents
+- count: total issue count
+- clean: true if no live issues were found
+- runtimeErrors/frameworkLogs/consoleLogs: source-specific aliases
 
-Use after making file changes to check for breakage without the overhead of coursecode_state.
 Requires preview server to be running.`,
         inputSchema: {
             type: 'object',
@@ -378,16 +389,16 @@ Use to discover available interactions before creating assessments.`,
     },
     {
         name: 'coursecode_lint',
-        description: `Run the course linter and get structured results.
+        description: `Run the static course linter and get structured results.
 
-Always runs build-time lint (config, CSS classes, structure). Does NOT include runtime errors — use coursecode_state for runtime errors and contrast warnings when the preview server is running.
+This is a preflight/static validation tool for config, source structure, CSS class names, and schema rules. It does NOT inspect the running preview and does NOT include live runtime, browser console, or Vite build-watch diagnostics. Use coursecode_errors for the live "what is broken right now?" view when preview is running.
 
 Returns:
 - errors: [{slideId?, rule, message, severity, hint?}]
 - warnings: [{slideId?, rule, message, severity, hint?}]
 - passed: boolean
 
-Build-time rules (always checked):
+Static rules (always checked):
 - undefined-css-class: hallucinated or stale class names (with fix suggestions)
 - unknown-component: unregistered data-component types
 - requirement-missing-component: engagement requirement without matching component
@@ -396,7 +407,7 @@ Build-time rules (always checked):
 - assessment-id-mismatch: config ID doesn't match assessment ID
 - invalid-gating: bad gating condition configuration
 
-Use AFTER making changes to validate the course.`,
+Use after making source/config changes as a fast static check. Then use coursecode_errors against the running preview for live diagnostics.`,
         inputSchema: {
             type: 'object',
             properties: {},
@@ -764,12 +775,16 @@ All runtime tools (state, navigate, interact, screenshot, viewport, reset) execu
 
 ### Preview Server Ownership
 - The MCP does NOT start or manage the preview server
-- The preview must be started externally: run \`coursecode preview\` in a terminal (human) or via a terminal/command execution tool (AI agent)
-- If preview is not running, runtime tools will fail with a clear error message
+- Before using runtime tools, check whether preview is already running for this project
+- If preview is already running, use it; do not start a second preview server
+- If preview is not running, start it externally in a terminal: run \`coursecode preview\` (or \`npm run preview\` for framework development)
+- AI agents may start preview only via a terminal/command execution tool, and only after a runtime tool reports that preview is not running
 - The headless browser auto-reconnects when Vite rebuilds (file changes)
 
 ### Navigation API
-- coursecode_state → get slim TOC with slide IDs, current slide, interactions, engagement, lmsState, apiLog, errors, frameworkLogs, consoleLogs
+- coursecode_state → get slim TOC with slide IDs, current slide, interactions, engagement, lmsState, apiLog, diagnostics, frameworkLogs, consoleLogs
+- coursecode_errors → get live diagnostics only (build/runtime/framework/console); use this after edits
+- coursecode_lint → run static preflight lint only; it is not a live preview diagnostic
 - coursecode_navigate(slideId) → go to any slide instantly by ID
 - coursecode_viewport(breakpoint or {width,height}) → set viewport for responsive testing (persists until changed)
 - coursecode_screenshot(slideId) → navigate + capture in one call (quality modes only, never changes viewport)