Add router decision logging and config versioning for Addie #437
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add ExecutionPlanBase type with metadata fields (decision_method, latency, tokens) - Log router decisions to unified thread messages (action, reason, timing) - Create config versioning system to track rule/router configuration changes - Link messages to config versions for feedback analysis by configuration - Add database migrations for router_decision column and config_versions table 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Allows testing the Haiku router decision logic without Slack credentials:
- POST /api/admin/addie/test-router with { message, source?, isThread? }
- Returns router decision metadata (action, reason, method, latency, tokens, model, tools)
- Logs to database with router_decision field for verification
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Router improvements: - Add TOOL_DESCRIPTIONS to router prompt so it picks tools based on query intent (not just keywords). Distinguishes "how does X work?" (search_docs) from "validate my X" (validate_adagents) - Add separate expertise area for validation/debugging requests Docs indexing: - Extract markdown headings as separate searchable artifacts - Generate anchor links for deep linking to specific sections - Skip headings inside code blocks to avoid false matches - Build headings index alongside docs index (1659 headings from 100 docs) Search tracking: - Add migration 070 for addie_search_logs table - Log all search queries for pattern analysis - Track results count, latency, and tool used - Add analytics functions for content gap detection Code review fixes: - Use logger.warn instead of logger.debug for search logging failures - Truncate queries to 1000 chars to prevent storage bloat - Handle code blocks when extracting headings 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Router improvements: - Add usage_hints field to AddieTool interface for router-specific guidance - Router now builds tool descriptions from tool definitions (no duplication) - Distinguish "how does X work?" (search_docs) from "validate my X" (validate_adagents) Prompt improvements: - Strengthen GitHub issue drafting instructions - users cannot see tool outputs - Add conversation context maintenance guidance to prevent entity substitution 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Previously, each message in a Slack DM thread was processed without context from previous turns. Claude only received the current message plus member context, causing it to lose track of conversation topics (e.g., confusing Ebiquity with Scope3 when asked follow-up questions). Changes: - Fetch conversation history from database before processing DM messages - Pass previous user/assistant messages to Claude via threadContext param - Limit to last 10 messages to avoid token bloat - Log when conversation history is loaded for debugging Also strengthened prompt instructions: - Clarify that tool outputs are invisible to users (GitHub issue bug) - Add guidance to maintain entity focus throughout conversations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Main branch already has migrations 068-071, so renumber our new migrations: - 068 -> 072: router_decision_metadata - 069 -> 073: config_versioning - 070 -> 074: addie_search_logs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Main had two migrations with version 070: - 070_org_admin_group_dm.sql (from PR #447) - 070_fix_knowledge_source_url_constraint.sql (from PR #448) Renumber to fix: - 070 stays: org_admin_group_dm (first merged) - 070 -> 071: fix_knowledge_source_url_constraint (second merged) - 071 -> 072: join_request_names - 072 -> 073: router_decision_metadata - 073 -> 074: config_versioning - 074 -> 075: addie_search_logs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 24028510 | Triggered | Generic CLI Secret | 6e196bb | scripts/migrate-csv-via-api.ts | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
- Add 30-min cache for member insights with invalidation on writes - Prefetch insights when user opens Addie (before first message) - Add 30-min cache for active insight goals (invalidated on admin changes) - Add 30-min cache for Slack channel info - Add 30-min cache for admin status checks - Parallelize member context and insights fetching - Router now uses member insights for smarter tool selection - Renumber migrations after PR #449 merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Restore 074_notification_channels.sql from main and renumber branch migrations: - 074 → 075 join_request_names - 075 → 076 router_decision_metadata - 076 → 077 config_versioning - 077 → 078 addie_search_logs Also fix 071 → 070 fix_knowledge_source_url_constraint 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Changes
ExecutionPlanBasetype with metadata fields (decision_method, latency_ms, tokens_input, tokens_output, model)router_decisionandconfig_version_idfields to message interfacesbuildRouterDecision()helper, updatedhandleChannelMessage()to log router decisionsconfig_version_idconfig_version_idin web chat messagesTest plan
config_version_id🤖 Generated with Claude Code