add quality verification for chain system

chain-system/README.md

@@ -11,12 +11,14 @@ Manage conversation "chains" - structured context across multiple sessions. Each
 ## Commands
 
 ### `/chain-link [chain-name]`
-Saves current conversation chunk as a chain link.
+Saves current conversation chunk as a chain link, then benchmarks it.
 
 **Captures**: User requests, code changes (files/lines), unresolved bugs with solutions attempted, pending tasks, next step.
 
 **Saves to**: `.claude/chains/[chain-name]/[timestamp]-[slug].md`
 
+**Benchmark**: After saving, spawns a clean agent with no prior context that reads ONLY the chain link and answers 10 verification questions. Scores 0-100 per question. If average < 85, the link is iteratively improved (up to 3 attempts) by filling gaps the clean agent couldn't answer. Shows final score breakdown.
+
 ### `/chain-load [chain-name]`
 Loads most recent chain link to continue work.
 
@@ -38,6 +40,30 @@ Lists all available chains with link count and most recent work.
 /plugin install chain-system@deevs-agent-system
 ```
 
+### Verification Hook
+
+Add the following to your project's `.claude/settings.json` to enable automatic chain link verification:
+
+```json
+{
+  "hooks": {
+    "PostToolUse": [
+      {
+        "matcher": "Write|Edit",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "bash chain-system/hooks/verify-chain-link.sh"
+          }
+        ]
+      }
+    ]
+  }
+}
+```
+
+When enabled, every chain link write is verified by an isolated CLI process. If the benchmark score is below 85/100, the write is blocked and gaps are fed back to the agent for correction.
+
 ## File Structure
 
 ```

chain-system/SKILL.md

@@ -29,7 +29,7 @@ Default to `.claude/chains` to remain compatible with the existing Claude chain-
 
 ### `link`
 
-Create a new chain link summarizing the current work.
+Create a new chain link summarizing the current work, then verify its quality.
 
 Steps:
 1. Build a concise but complete summary using this structure:
@@ -47,7 +47,15 @@ Steps:
 4. Generate timestamp in local time: `date '+%Y-%m-%d-%H%M'`.
 5. Write the summary to:
    `.claude/chains/<chain-name>/<timestamp>-<slug>.md`
-6. Write only the summary (no analysis), then tell the user the file path and how to load it.
+6. Write only the summary (no analysis).
+7. **Verify** (automatic — enforced by PostToolUse hook, not by the agent):
+   - When the file is written, the `verify-chain-link.sh` hook fires automatically.
+   - The hook spawns an isolated CLI process (`claude --print --bare`) with only the chain link content.
+   - The isolated process answers 10 verification questions and scores them 0-100.
+   - If average score >= 85: write succeeds, benchmark shown.
+   - If average score < 85: write is **blocked** (exit 2). Gaps feed back to the agent.
+   - Agent must address the gaps and re-save — which re-triggers the hook.
+   - This continues until the chain link passes verification.
 
 Use the same file format each time so `load` can extract summary and next step reliably.
 

chain-system/commands/chain-link.md

@@ -113,11 +113,37 @@ Create a slug (e.g., `implement-auth-handler`) and readable summary (e.g., "Impl
 </plan>
 ```
 
-## Final Step
+## Save Step
 
 Create directory and save file:
 1. Create directory: `mkdir -p .claude/chains/[chain-name]` (substitute actual chain name)
 2. Generate timestamp using local time: `date '+%Y-%m-%d-%H%M'` (e.g., `2025-11-24-0230`)
 3. Write to: `.claude/chains/[chain-name]/[timestamp]-[slug].md`
 4. Save only the content inside `<plan>` tags to the file
-5. Inform user: file path and how to load with `/chain-load [chain-name]`
+
+## Verification (Automatic via Hook)
+
+Verification is **not a manual step** — it is enforced by a `PostToolUse` hook on `Write|Edit`. When you save the chain link file to `.claude/chains/**/*.md`, the hook automatically:
+
+1. Detects the file is a chain link
+2. Runs `chain-system/scripts/chain-verify.sh` — spawns an **isolated CLI process** (`claude --print --bare`) with zero shared context
+3. The isolated process reads ONLY the chain link content and answers 10 verification questions
+4. Scores each answer 0-100
+
+**If score >= 85**: Hook exits 0, write succeeds. Benchmark score is shown.
+
+**If score < 85**: Hook exits 2, **write is blocked**. The gaps are fed back to you as feedback. You must:
+- Read the gap analysis in the hook feedback
+- Re-examine the conversation for the missing information
+- Rewrite the chain link addressing the specific gaps
+- Save again (which re-triggers the hook automatically)
+
+This loop continues until the chain link passes or you've addressed all available information from the conversation.
+
+### Setup
+
+The hook must be registered in `.claude/settings.json` (see Installation section in README). The hook script lives at `chain-system/hooks/verify-chain-link.sh` and calls `chain-system/scripts/chain-verify.sh`.
+
+### Why a hook and not instructions?
+
+Subagents (Agent tool) share the parent conversation context — they already know the answers and would pass every time, making verification meaningless. Instructions in markdown can be skipped or forgotten. The hook is a **hard gate**: the file literally cannot be saved unless the isolated verifier confirms a fresh session could resume from it.

chain-system/hooks/verify-chain-link.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Hook: PostToolUse (Write|Edit)
+# Fires automatically after any file write. Checks if the written file is a
+# chain link (.claude/chains/**/*.md) and runs isolated verification if so.
+#
+# Exit codes:
+#   0 = pass (or not a chain file — skip silently)
+#   2 = fail (blocks the write, feeds gaps back to the agent)
+
+INPUT=$(cat)
+FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // .tool_input.filePath // empty')
+
+# Only trigger on chain link files
+if [[ ! "$FILE_PATH" =~ \.claude/chains/.*\.md$ ]]; then
+  exit 0
+fi
+
+if [[ ! -f "$FILE_PATH" ]]; then
+  exit 0
+fi
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+VERIFY_SCRIPT="$SCRIPT_DIR/../scripts/chain-verify.sh"
+
+if [[ ! -x "$VERIFY_SCRIPT" ]]; then
+  echo "WARNING: chain-verify.sh not found or not executable at $VERIFY_SCRIPT" >&2
+  exit 0
+fi
+
+# Generate verification questions from the chain link content
+# Write them to a temp file for the verification script
+QUESTIONS_FILE="$(mktemp /tmp/chain-verify-questions-XXXXXX.md)"
+CHAIN_CONTENT="$(cat "$FILE_PATH")"
+
+# Use a simple, deterministic question set derived from the 9 required sections.
+# These probe whether each section is present and substantive.
+cat > "$QUESTIONS_FILE" << 'EOF'
+1. What is the user's primary request? Describe the exact goal, not a vague summary.
+2. What specific technologies, frameworks, or patterns are involved, and why were they chosen?
+3. What work has been completed so far? List concrete deliverables with their current status.
+4. What key decisions were made during this work? For each, what was the alternative and why was it rejected?
+5. What files were created or modified? For created files, is the full code included? For modified files, are the before/after changes and line numbers present?
+6. Are there any unresolved issues or blockers? If so, what solutions were attempted and why did they fail?
+7. What tasks remain to be done? Are they specific enough to act on without guessing?
+8. What was actively in progress when this chain link was saved? Include file names, line numbers, and code context.
+9. What is the exact next step? Is it a single, unambiguous action that a new session can execute immediately?
+10. Could you resume this work right now with ONLY the information in this chain link, without asking any clarifying questions? If not, what would you need to ask?
+EOF
+
+echo "Verifying chain link: $(basename "$FILE_PATH")" >&2
+
+# Run verification in isolated process
+VERIFY_OUTPUT=$("$VERIFY_SCRIPT" "$FILE_PATH" "$QUESTIONS_FILE" 2>/dev/null) || true
+rm -f "$QUESTIONS_FILE"
+
+if [[ -z "$VERIFY_OUTPUT" ]]; then
+  echo "WARNING: Verification produced no output (CLI may not be available). Chain link saved without benchmark." >&2
+  exit 0
+fi
+
+# Extract the aggregate score from the verification output
+# Look for "Total Score": followed by a number
+SCORE=$(echo "$VERIFY_OUTPUT" | grep -oP 'Total Score\D*\K[0-9]+' | head -1 || echo "")
+
+if [[ -z "$SCORE" ]]; then
+  # Could not parse score — let it through with the raw output
+  echo "Chain link saved. Verification output (could not parse score):"
+  echo "$VERIFY_OUTPUT"
+  exit 0
+fi
+
+if [[ "$SCORE" -ge 85 ]]; then
+  echo "Chain link verified. Benchmark: ${SCORE}/100"
+  echo ""
+  echo "$VERIFY_OUTPUT"
+  exit 0
+else
+  # Fail — block the write, feed gaps back to the agent
+  cat >&2 << FEEDBACK
+CHAIN LINK VERIFICATION FAILED — Benchmark: ${SCORE}/100 (threshold: 85)
+
+The chain link is not self-sufficient. A fresh session would not be able to resume
+from this chain link alone. Review the gaps below and rewrite the chain link with
+the missing information, then save it again to re-trigger verification.
+
+--- Verification Output ---
+${VERIFY_OUTPUT}
+--- End Verification Output ---
+
+ACTION REQUIRED: Rewrite the chain link addressing the gaps above, then save again.
+FEEDBACK
+  exit 2
+fi

chain-system/scripts/chain-verify.sh

@@ -0,0 +1,78 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Chain Link Verification Script
+# Spawns an isolated CLI process to benchmark a chain link's self-sufficiency.
+# The verifier gets ONLY the chain link content and questions — zero shared context.
+
+CHAIN_LINK_FILE="${1:?Usage: chain-verify.sh <chain-link-file> <questions-file>}"
+QUESTIONS_FILE="${2:?Usage: chain-verify.sh <chain-link-file> <questions-file>}"
+
+if [[ ! -f "$CHAIN_LINK_FILE" ]]; then
+  echo "ERROR: Chain link file not found: $CHAIN_LINK_FILE" >&2
+  exit 1
+fi
+
+if [[ ! -f "$QUESTIONS_FILE" ]]; then
+  echo "ERROR: Questions file not found: $QUESTIONS_FILE" >&2
+  exit 1
+fi
+
+CHAIN_CONTENT="$(cat "$CHAIN_LINK_FILE")"
+QUESTIONS="$(cat "$QUESTIONS_FILE")"
+
+SYSTEM_PROMPT="You are a developer resuming work from a chain link. You have NO other context besides what is provided in this prompt. You must answer each question using ONLY the chain link content given. If you cannot answer a question or the answer is incomplete/ambiguous, say INSUFFICIENT and explain what is missing."
+
+USER_PROMPT="$(cat <<EOF
+## Chain Link Content
+
+${CHAIN_CONTENT}
+
+## Verification Questions
+
+${QUESTIONS}
+
+## Instructions
+
+Answer each question using ONLY the chain link content above. For each question respond with:
+
+### Q[N]: [question text]
+**Answer**: [your answer based solely on the chain link]
+**Score**: [0-100, where 90-100 = fully answerable, 70-89 = missing some detail, 50-69 = significant gaps, 0-49 = cannot answer]
+**Gap**: [what specific information is missing, if anything]
+
+After all questions, provide:
+
+### Aggregate
+**Total Score**: [average of all 10 scores, as integer]
+**Weakest Sections**: [which chain link sections (1-9) had lowest coverage]
+**Critical Gaps**: [specific missing information that would block a resuming developer]
+EOF
+)"
+
+# Detect available CLI
+if command -v claude &>/dev/null; then
+  # --print: non-interactive, output and exit
+  # --system-prompt: override default system prompt for isolation
+  # --no-session-persistence: don't save this throwaway session
+  # --model haiku: fast and cheap for verification scoring
+  #
+  # NOTE: We do NOT use --bare because it disables OAuth/keychain auth.
+  # Isolation is achieved by: separate process (no conversation context),
+  # custom system prompt (overrides default behavior), and --no-session-persistence.
+  # CLAUDE.md may load but it cannot provide conversation context from the parent session.
+  echo "$USER_PROMPT" | claude --print \
+    --system-prompt "$SYSTEM_PROMPT" \
+    --no-session-persistence \
+    --model haiku \
+    --allowedTools ""
+elif command -v codex &>/dev/null; then
+  codex --quiet \
+    --approval-mode full-auto \
+    "$SYSTEM_PROMPT
+
+$USER_PROMPT"
+else
+  echo "ERROR: Neither 'claude' nor 'codex' CLI found. Install one to run verification." >&2
+  exit 1
+fi

scripts/install.sh

@@ -258,6 +258,52 @@ uninstall_codex() {
   printf '\nDone. Restart Codex to reload skills.\n'
 }
 
+install_claude_hooks() {
+  local project_dir="${1:-.}"
+  local settings_file="$project_dir/.claude/settings.json"
+  local hook_cmd="bash chain-system/hooks/verify-chain-link.sh"
+
+  mkdir -p "$project_dir/.claude"
+
+  # Build the hook entry
+  local hook_entry
+  hook_entry=$(cat <<'HOOKJSON'
+{
+  "matcher": "Write|Edit",
+  "hooks": [
+    {
+      "type": "command",
+      "command": "bash \"$CLAUDE_PROJECT_DIR\"/chain-system/hooks/verify-chain-link.sh"
+    }
+  ]
+}
+HOOKJSON
+)
+
+  if [[ -f "$settings_file" ]]; then
+    # Check if hook already registered
+    if jq -e '.hooks.PostToolUse[]? | select(.hooks[]?.command == "bash chain-system/hooks/verify-chain-link.sh")' "$settings_file" >/dev/null 2>&1; then
+      echo "Hook already registered in $settings_file"
+      return 0
+    fi
+    # Merge: append to existing PostToolUse array (or create it)
+    local tmp
+    tmp=$(mktemp)
+    jq --argjson entry "$hook_entry" '
+      .hooks //= {} |
+      .hooks.PostToolUse //= [] |
+      .hooks.PostToolUse += [$entry]
+    ' "$settings_file" > "$tmp" && mv "$tmp" "$settings_file"
+    echo "Hook added to existing $settings_file"
+  else
+    # Create new settings file with just the hook
+    echo '{}' | jq --argjson entry "$hook_entry" '
+      .hooks.PostToolUse = [$entry]
+    ' > "$settings_file"
+    echo "Created $settings_file with verification hook"
+  fi
+}
+
 print_claude_instructions() {
   cat <<'EOC'
 Run the following commands inside Claude Code:
@@ -297,12 +343,21 @@ case "$PLATFORM" in
       uninstall_codex
     else
       install_codex
+      # Also register hook if chain-system is among installed skills
+      for skill in "${SKILLS_LIST[@]}"; do
+        if [[ "$skill" == *chain-system* ]]; then
+          install_claude_hooks "."
+          break
+        fi
+      done
     fi
     ;;
   claude)
     if [[ "$UNINSTALL" == "true" ]]; then
       print_claude_uninstall
     else
+      install_claude_hooks "."
+      printf '\n'
       print_claude_instructions
     fi
     ;;
@@ -313,6 +368,7 @@ case "$PLATFORM" in
       print_claude_uninstall
     else
       install_codex
+      install_claude_hooks "."
       printf '\n'
       print_claude_instructions
     fi