diff --git a/chain-system/README.md b/chain-system/README.md index 8b5483b..49d1c60 100644 --- a/chain-system/README.md +++ b/chain-system/README.md @@ -11,12 +11,14 @@ Manage conversation "chains" - structured context across multiple sessions. Each ## Commands ### `/chain-link [chain-name]` -Saves current conversation chunk as a chain link. +Saves current conversation chunk as a chain link, then benchmarks it. **Captures**: User requests, code changes (files/lines), unresolved bugs with solutions attempted, pending tasks, next step. **Saves to**: `.claude/chains/[chain-name]/[timestamp]-[slug].md` +**Benchmark**: After saving, spawns a clean agent with no prior context that reads ONLY the chain link and answers 10 verification questions. Scores 0-100 per question. If average < 85, the link is iteratively improved (up to 3 attempts) by filling gaps the clean agent couldn't answer. Shows final score breakdown. + ### `/chain-load [chain-name]` Loads most recent chain link to continue work. @@ -38,6 +40,30 @@ Lists all available chains with link count and most recent work. /plugin install chain-system@deevs-agent-system ``` +### Verification Hook + +Add the following to your project's `.claude/settings.json` to enable automatic chain link verification: + +```json +{ + "hooks": { + "PostToolUse": [ + { + "matcher": "Write|Edit", + "hooks": [ + { + "type": "command", + "command": "bash chain-system/hooks/verify-chain-link.sh" + } + ] + } + ] + } +} +``` + +When enabled, every chain link write is verified by an isolated CLI process. If the benchmark score is below 85/100, the write is blocked and gaps are fed back to the agent for correction. + ## File Structure ``` diff --git a/chain-system/SKILL.md b/chain-system/SKILL.md index 0ca393e..920ea40 100644 --- a/chain-system/SKILL.md +++ b/chain-system/SKILL.md @@ -29,7 +29,7 @@ Default to `.claude/chains` to remain compatible with the existing Claude chain- ### `link` -Create a new chain link summarizing the current work. +Create a new chain link summarizing the current work, then verify its quality. Steps: 1. Build a concise but complete summary using this structure: @@ -47,7 +47,15 @@ Steps: 4. Generate timestamp in local time: `date '+%Y-%m-%d-%H%M'`. 5. Write the summary to: `.claude/chains//-.md` -6. Write only the summary (no analysis), then tell the user the file path and how to load it. +6. Write only the summary (no analysis). +7. **Verify** (automatic — enforced by PostToolUse hook, not by the agent): + - When the file is written, the `verify-chain-link.sh` hook fires automatically. + - The hook spawns an isolated CLI process (`claude --print --bare`) with only the chain link content. + - The isolated process answers 10 verification questions and scores them 0-100. + - If average score >= 85: write succeeds, benchmark shown. + - If average score < 85: write is **blocked** (exit 2). Gaps feed back to the agent. + - Agent must address the gaps and re-save — which re-triggers the hook. + - This continues until the chain link passes verification. Use the same file format each time so `load` can extract summary and next step reliably. diff --git a/chain-system/commands/chain-link.md b/chain-system/commands/chain-link.md index 17d7de9..e84119e 100644 --- a/chain-system/commands/chain-link.md +++ b/chain-system/commands/chain-link.md @@ -113,11 +113,37 @@ Create a slug (e.g., `implement-auth-handler`) and readable summary (e.g., "Impl ``` -## Final Step +## Save Step Create directory and save file: 1. Create directory: `mkdir -p .claude/chains/[chain-name]` (substitute actual chain name) 2. Generate timestamp using local time: `date '+%Y-%m-%d-%H%M'` (e.g., `2025-11-24-0230`) 3. Write to: `.claude/chains/[chain-name]/[timestamp]-[slug].md` 4. Save only the content inside `` tags to the file -5. Inform user: file path and how to load with `/chain-load [chain-name]` + +## Verification (Automatic via Hook) + +Verification is **not a manual step** — it is enforced by a `PostToolUse` hook on `Write|Edit`. When you save the chain link file to `.claude/chains/**/*.md`, the hook automatically: + +1. Detects the file is a chain link +2. Runs `chain-system/scripts/chain-verify.sh` — spawns an **isolated CLI process** (`claude --print --bare`) with zero shared context +3. The isolated process reads ONLY the chain link content and answers 10 verification questions +4. Scores each answer 0-100 + +**If score >= 85**: Hook exits 0, write succeeds. Benchmark score is shown. + +**If score < 85**: Hook exits 2, **write is blocked**. The gaps are fed back to you as feedback. You must: +- Read the gap analysis in the hook feedback +- Re-examine the conversation for the missing information +- Rewrite the chain link addressing the specific gaps +- Save again (which re-triggers the hook automatically) + +This loop continues until the chain link passes or you've addressed all available information from the conversation. + +### Setup + +The hook must be registered in `.claude/settings.json` (see Installation section in README). The hook script lives at `chain-system/hooks/verify-chain-link.sh` and calls `chain-system/scripts/chain-verify.sh`. + +### Why a hook and not instructions? + +Subagents (Agent tool) share the parent conversation context — they already know the answers and would pass every time, making verification meaningless. Instructions in markdown can be skipped or forgotten. The hook is a **hard gate**: the file literally cannot be saved unless the isolated verifier confirms a fresh session could resume from it. diff --git a/chain-system/hooks/verify-chain-link.sh b/chain-system/hooks/verify-chain-link.sh new file mode 100755 index 0000000..806fc1a --- /dev/null +++ b/chain-system/hooks/verify-chain-link.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Hook: PostToolUse (Write|Edit) +# Fires automatically after any file write. Checks if the written file is a +# chain link (.claude/chains/**/*.md) and runs isolated verification if so. +# +# Exit codes: +# 0 = pass (or not a chain file — skip silently) +# 2 = fail (blocks the write, feeds gaps back to the agent) + +INPUT=$(cat) +FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // .tool_input.filePath // empty') + +# Only trigger on chain link files +if [[ ! "$FILE_PATH" =~ \.claude/chains/.*\.md$ ]]; then + exit 0 +fi + +if [[ ! -f "$FILE_PATH" ]]; then + exit 0 +fi + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +VERIFY_SCRIPT="$SCRIPT_DIR/../scripts/chain-verify.sh" + +if [[ ! -x "$VERIFY_SCRIPT" ]]; then + echo "WARNING: chain-verify.sh not found or not executable at $VERIFY_SCRIPT" >&2 + exit 0 +fi + +# Generate verification questions from the chain link content +# Write them to a temp file for the verification script +QUESTIONS_FILE="$(mktemp /tmp/chain-verify-questions-XXXXXX.md)" +CHAIN_CONTENT="$(cat "$FILE_PATH")" + +# Use a simple, deterministic question set derived from the 9 required sections. +# These probe whether each section is present and substantive. +cat > "$QUESTIONS_FILE" << 'EOF' +1. What is the user's primary request? Describe the exact goal, not a vague summary. +2. What specific technologies, frameworks, or patterns are involved, and why were they chosen? +3. What work has been completed so far? List concrete deliverables with their current status. +4. What key decisions were made during this work? For each, what was the alternative and why was it rejected? +5. What files were created or modified? For created files, is the full code included? For modified files, are the before/after changes and line numbers present? +6. Are there any unresolved issues or blockers? If so, what solutions were attempted and why did they fail? +7. What tasks remain to be done? Are they specific enough to act on without guessing? +8. What was actively in progress when this chain link was saved? Include file names, line numbers, and code context. +9. What is the exact next step? Is it a single, unambiguous action that a new session can execute immediately? +10. Could you resume this work right now with ONLY the information in this chain link, without asking any clarifying questions? If not, what would you need to ask? +EOF + +echo "Verifying chain link: $(basename "$FILE_PATH")" >&2 + +# Run verification in isolated process +VERIFY_OUTPUT=$("$VERIFY_SCRIPT" "$FILE_PATH" "$QUESTIONS_FILE" 2>/dev/null) || true +rm -f "$QUESTIONS_FILE" + +if [[ -z "$VERIFY_OUTPUT" ]]; then + echo "WARNING: Verification produced no output (CLI may not be available). Chain link saved without benchmark." >&2 + exit 0 +fi + +# Extract the aggregate score from the verification output +# Look for "Total Score": followed by a number +SCORE=$(echo "$VERIFY_OUTPUT" | grep -oP 'Total Score\D*\K[0-9]+' | head -1 || echo "") + +if [[ -z "$SCORE" ]]; then + # Could not parse score — let it through with the raw output + echo "Chain link saved. Verification output (could not parse score):" + echo "$VERIFY_OUTPUT" + exit 0 +fi + +if [[ "$SCORE" -ge 85 ]]; then + echo "Chain link verified. Benchmark: ${SCORE}/100" + echo "" + echo "$VERIFY_OUTPUT" + exit 0 +else + # Fail — block the write, feed gaps back to the agent + cat >&2 << FEEDBACK +CHAIN LINK VERIFICATION FAILED — Benchmark: ${SCORE}/100 (threshold: 85) + +The chain link is not self-sufficient. A fresh session would not be able to resume +from this chain link alone. Review the gaps below and rewrite the chain link with +the missing information, then save it again to re-trigger verification. + +--- Verification Output --- +${VERIFY_OUTPUT} +--- End Verification Output --- + +ACTION REQUIRED: Rewrite the chain link addressing the gaps above, then save again. +FEEDBACK + exit 2 +fi diff --git a/chain-system/scripts/chain-verify.sh b/chain-system/scripts/chain-verify.sh new file mode 100755 index 0000000..e488180 --- /dev/null +++ b/chain-system/scripts/chain-verify.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Chain Link Verification Script +# Spawns an isolated CLI process to benchmark a chain link's self-sufficiency. +# The verifier gets ONLY the chain link content and questions — zero shared context. + +CHAIN_LINK_FILE="${1:?Usage: chain-verify.sh }" +QUESTIONS_FILE="${2:?Usage: chain-verify.sh }" + +if [[ ! -f "$CHAIN_LINK_FILE" ]]; then + echo "ERROR: Chain link file not found: $CHAIN_LINK_FILE" >&2 + exit 1 +fi + +if [[ ! -f "$QUESTIONS_FILE" ]]; then + echo "ERROR: Questions file not found: $QUESTIONS_FILE" >&2 + exit 1 +fi + +CHAIN_CONTENT="$(cat "$CHAIN_LINK_FILE")" +QUESTIONS="$(cat "$QUESTIONS_FILE")" + +SYSTEM_PROMPT="You are a developer resuming work from a chain link. You have NO other context besides what is provided in this prompt. You must answer each question using ONLY the chain link content given. If you cannot answer a question or the answer is incomplete/ambiguous, say INSUFFICIENT and explain what is missing." + +USER_PROMPT="$(cat </dev/null; then + # --print: non-interactive, output and exit + # --system-prompt: override default system prompt for isolation + # --no-session-persistence: don't save this throwaway session + # --model haiku: fast and cheap for verification scoring + # + # NOTE: We do NOT use --bare because it disables OAuth/keychain auth. + # Isolation is achieved by: separate process (no conversation context), + # custom system prompt (overrides default behavior), and --no-session-persistence. + # CLAUDE.md may load but it cannot provide conversation context from the parent session. + echo "$USER_PROMPT" | claude --print \ + --system-prompt "$SYSTEM_PROMPT" \ + --no-session-persistence \ + --model haiku \ + --allowedTools "" +elif command -v codex &>/dev/null; then + codex --quiet \ + --approval-mode full-auto \ + "$SYSTEM_PROMPT + +$USER_PROMPT" +else + echo "ERROR: Neither 'claude' nor 'codex' CLI found. Install one to run verification." >&2 + exit 1 +fi diff --git a/scripts/install.sh b/scripts/install.sh index a2c79a1..349c63c 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -258,6 +258,52 @@ uninstall_codex() { printf '\nDone. Restart Codex to reload skills.\n' } +install_claude_hooks() { + local project_dir="${1:-.}" + local settings_file="$project_dir/.claude/settings.json" + local hook_cmd="bash chain-system/hooks/verify-chain-link.sh" + + mkdir -p "$project_dir/.claude" + + # Build the hook entry + local hook_entry + hook_entry=$(cat <<'HOOKJSON' +{ + "matcher": "Write|Edit", + "hooks": [ + { + "type": "command", + "command": "bash \"$CLAUDE_PROJECT_DIR\"/chain-system/hooks/verify-chain-link.sh" + } + ] +} +HOOKJSON +) + + if [[ -f "$settings_file" ]]; then + # Check if hook already registered + if jq -e '.hooks.PostToolUse[]? | select(.hooks[]?.command == "bash chain-system/hooks/verify-chain-link.sh")' "$settings_file" >/dev/null 2>&1; then + echo "Hook already registered in $settings_file" + return 0 + fi + # Merge: append to existing PostToolUse array (or create it) + local tmp + tmp=$(mktemp) + jq --argjson entry "$hook_entry" ' + .hooks //= {} | + .hooks.PostToolUse //= [] | + .hooks.PostToolUse += [$entry] + ' "$settings_file" > "$tmp" && mv "$tmp" "$settings_file" + echo "Hook added to existing $settings_file" + else + # Create new settings file with just the hook + echo '{}' | jq --argjson entry "$hook_entry" ' + .hooks.PostToolUse = [$entry] + ' > "$settings_file" + echo "Created $settings_file with verification hook" + fi +} + print_claude_instructions() { cat <<'EOC' Run the following commands inside Claude Code: @@ -297,12 +343,21 @@ case "$PLATFORM" in uninstall_codex else install_codex + # Also register hook if chain-system is among installed skills + for skill in "${SKILLS_LIST[@]}"; do + if [[ "$skill" == *chain-system* ]]; then + install_claude_hooks "." + break + fi + done fi ;; claude) if [[ "$UNINSTALL" == "true" ]]; then print_claude_uninstall else + install_claude_hooks "." + printf '\n' print_claude_instructions fi ;; @@ -313,6 +368,7 @@ case "$PLATFORM" in print_claude_uninstall else install_codex + install_claude_hooks "." printf '\n' print_claude_instructions fi