0.2.15

.github/workflows/droid-review.yml

@@ -28,4 +28,4 @@ jobs:
         uses: Factory-AI/droid-action@v3
         with:
           factory_api_key: ${{ secrets.FACTORY_API_KEY }}
-          automatic_review: true
+          automatic_review: false

CHANGELOG.md

@@ -5,6 +5,63 @@ All notable changes to the Attocode Python agent will be documented in this file
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.15] - 2026-04-04
+
+### Added
+
+#### Frecency-Boosted Search
+- Frecency tracker (`integrations/context/frecency.py`) — SQLite-backed file access scoring with exponential decay (10-day human / 3-day AI half-life), git modification bonuses, and batch scoring
+- `frecency_search` MCP tool — regex search with frecency-boosted ranking, two-phase file ordering (high-frecency files searched first), trigram pre-filtering
+- `track_file_access`, `get_file_frecency`, `get_frecency_leaderboard`, `get_frecency_stats`, `clear_frecency` MCP tools
+- Frecency leaderboard implementation with ranked output by score
+
+#### Fuzzy Search (Smith-Waterman)
+- Smith-Waterman fuzzy matcher (`integrations/context/fuzzy.py`) — typo-resistant search via local sequence alignment with affine gap penalties
+- `fuzzy_search` MCP tool — line-level fuzzy matching across files with configurable score threshold
+- `fuzzy_filename_search` MCP tool — find files by partial/typo'd filename
+- `fuzzy_score` MCP tool — debug tool for inspecting match quality between pattern and text
+
+#### Cross-Mode Search Suggestions
+- Cross-mode search engine (`integrations/context/cross_mode.py`) — "did you mean" suggestions across file-name and content search modes
+- `suggest_when_file_search_finds_nothing` MCP tool — grep fallback when filename search returns no results
+- `suggest_when_grep_finds_nothing` MCP tool — filename fallback when content search returns no results
+- `cross_mode_search` MCP tool — combined search across both modes
+
+#### Query Constraints (fff-style Filters)
+- Query constraint parser (`integrations/context/query_constraints.py`) — supports `git:modified`, `git:staged`, `!pattern`, `path/`, `*.ext`, `./**/*.py` filters
+- `parse_query_with_constraints`, `filter_files_with_constraints`, `list_modified_files` MCP tools
+- Git porcelain XY parsing — reads both index and worktree columns for accurate status
+
+#### Query History & Combo Boosting
+- Query history tracker (`integrations/context/query_history.py`) — SQLite-backed tracking of query-to-file selections with combo boost scoring (3+ selections trigger boost)
+- `track_query_result`, `get_query_combo_boost`, `get_top_results_for_query`, `get_query_history_stats`, `clear_query_history` MCP tools
+
+#### Code-Intel Testing Infrastructure
+- `code_intel/testing/` package — fixtures, helpers, and mocks for MCP tool tests
+- 9 new tool test modules covering ADR, analysis, dead code, distill, history, learning, LSP, navigation, readiness, search, and server tools
+- Test conftest with shared project fixtures and mock services
+
+### Fixed
+- **Git argument injection** — added `--` separator before file paths in git subprocess calls (`frecency_tools.py`, `query_constraints.py`)
+- **Git status parsing** — `_run_git_status()` and `list_modified_files()` now read both XY columns of porcelain output, fixing invisible working-tree-only modifications
+- **Cross-mode search labels** — renamed methods to `suggest_content_matches()` / `suggest_filename_matches()` and fixed output labels that said "File search results" for grep results
+- **Frecency tracker singleton** — unified to single thread-safe `get_tracker()` via `_shared.py`; removed duplicate non-locking construction
+- **Gitignore-aware file walking** — `frecency_search`, `fuzzy_search`, `fuzzy_filename_search`, and cross-mode suggestions now filter via `IgnoreManager`, skipping `.gitignore`d paths
+- **BM25 cache save on duplicate doc IDs** — `_save_kw_cache()` uses `INSERT OR REPLACE` to handle multiple functions with the same name in one file (e.g. test helpers), fixing silent cache save failures
+
+### Changed
+
+#### Semantic Search Performance: Incremental Cache + Trigram Pre-filter
+- **BM25 keyword index disk cache** — `_build_keyword_index()` now persists the full inverted index to `.attocode/index/kw_index.db` (SQLite/WAL) with file mtime tracking; subsequent searches load from cache and only re-parse changed files instead of rebuilding from scratch (176x speedup on warm cache in smoke test: 5.8s → 33ms)
+- **Trigram pre-filtering for keyword search** — before BM25 scoring, queries the existing trigram index with each query token (UNION semantics) to narrow candidates to files containing at least one term; skips scoring documents from non-matching files with zero accuracy loss (BM25 IDF still uses full corpus stats)
+- **Numpy-accelerated vector search** — `VectorStore.search()` now uses numpy BLAS matrix-vector multiply for batch cosine similarity instead of a pure Python per-vector loop; in-memory vector cache with version-based invalidation on upsert/delete; `np.argpartition` for O(N) top-k selection. 183x speedup at 10K vectors (245ms → 1.3ms), 100K vectors searched in 15ms. Falls back to pure Python if numpy unavailable.
+- `invalidate_file()` now marks entries stale in the keyword cache and clears the trigram index reference
+
+#### Other
+- `frecency_search` uses two-phase file ordering: collects candidate paths, pre-sorts by frecency score, then reads content — ensures high-frecency files appear in results regardless of alphabetical order
+- Extracted `_empty_frecency()` helper to replace repeated inline `FrecencyResult` constructions
+- `_get_frecency_tracker()` moved to `_shared.py` for single source of truth across tool modules
+
 ## [0.2.14] - 2026-04-01
 
 ### Added

docs/guides/evaluation-and-benchmarks.md

@@ -98,24 +98,23 @@ python scripts/benchmark_3way.py --skip-code-intel
 python scripts/benchmark_3way.py --slice published_20 --resume
 ```
 
-### Latest Results (v0.2.11, 20 repos)
+### Latest Results (v0.2.15, 20 repos)
 
 | Metric | grep | ast-grep | code-intel |
 |--------|------|----------|------------|
 | **Avg Quality** | 4.0/5 | 2.8/5 | **4.7/5** |
-| **Avg Bootstrap** | 91ms | 538ms | 1.7s* |
-| **Perfect Scores (5/5)** | 48/120 | 36/120 | **101/120** |
-| **Zero Scores (0/5)** | 0 | 24 | 0 |
+| **Avg Time** | 95ms | 493ms | 2,731ms |
 
-\* Bootstrap time after progressive hydration. Pre-hydration large repo times were 7-25s.
+\* v0.2.15 includes BM25 keyword index caching (8x speedup on large repos), trigram pre-filtering for BM25, and numpy-accelerated vector search (183x at 10K vectors). Overall 35% faster than v0.2.11 (4,182ms → 2,731ms).
 
 **Key findings:**
 - Code-intel delivers the highest quality (4.7/5) with structured, concise output
-- grep is fast (91ms) and surprisingly competitive (4.0/5) for simple lookups
+- grep is fast (95ms) and surprisingly competitive (4.0/5) for simple lookups
 - ast-grep adds limited value — slower than grep with lower quality (2.8/5)
-- Progressive hydration brings all repos under 4s bootstrap (cockroach: 24.5s → 1.2s)
+- Semantic search is the remaining speed bottleneck on large repos, but quality justifies the cost (5/5 vs 3-4/5 for grep/ast-grep)
+- BM25 keyword cache reduces warm-start keyword search from 20s to 2.5s on cockroach-scale repos
 
-Charts and per-repo analysis: `eval/3WAY_BENCHMARK_REPORT.md`
+Charts and per-repo analysis: `eval/3way_comparison_20repos.md`
 
 ## Configured Repos (49)
 

docs/guides/semantic-search.md

@@ -54,13 +54,42 @@ The `semantic_search` tool accepts an optional `mode` parameter:
 | `keyword` | BM25 keyword search only — skips embedding entirely | Speed-critical, large repos, or no embedding model |
 | `vector` | Waits for embedding index to be ready (up to 60s), then uses vector search | Need highest quality results |
 
+## Performance Optimizations (v0.2.15)
+
+Three optimizations reduce search latency by 35% overall (4,182ms → 2,731ms avg across 20 repos).
+
+### BM25 Keyword Index Cache
+
+The BM25 inverted index is now cached to disk at `.attocode/index/kw_index.db` (SQLite). On first search, the index is built from source files and persisted; subsequent searches load from cache. The cache is incremental — only changed files are re-parsed on the next search.
+
+- **Warm cache on cockroach (103K docs):** 2.5s load vs 20s full rebuild (8x speedup)
+- Cache is invalidated per-file based on mtime, so edits are picked up automatically
+
+### Trigram Pre-filtering for BM25
+
+Before BM25 scoring, the existing trigram index is queried for each token in the search query. Files matching any token (UNION semantics) form the candidate set for BM25 scoring.
+
+- Falls back to full corpus scan if the trigram index is not built or all query tokens are shorter than 3 characters
+- Zero accuracy loss: BM25 IDF statistics are still computed over the full corpus, only the scoring pass is narrowed
+
+### Numpy-Accelerated Vector Search
+
+`VectorStore.search()` now uses numpy BLAS matrix multiplication for batch cosine similarity instead of a Python loop. An in-memory vector cache is maintained and auto-invalidated on upsert/delete operations. Top-k selection uses `np.argpartition` for O(N) performance instead of O(N log N) full sort.
+
+- **183x faster** than pure Python at 10K vectors (245ms → 1.3ms)
+- 100K vectors searched in ~15ms
+- Falls back to a pure Python loop if numpy is not installed
+- Server mode is unaffected (already uses pgvector HNSW)
+
 ## Vector Store Backends
 
 | | SQLite (CLI mode) | pgvector (service mode) |
 |---|---|---|
-| **Scale** | ~10K vectors (linear scan) | ~5M vectors (HNSW index) |
-| **Query @ 5K** | ~2ms | ~1ms |
-| **Query @ 500K** | ~200ms (unusable) | ~5ms |
+| **Scale** | ~100K vectors (numpy batch search) | ~5M vectors (HNSW index) |
+| **Query @ 5K** | <1ms | ~1ms |
+| **Query @ 10K** | ~1.3ms (numpy) | ~1ms |
+| **Query @ 100K** | ~15ms (numpy) | ~3ms |
+| **Query @ 500K** | ~75ms (numpy), pure Python ~200ms | ~5ms |
 | **Deployment** | Zero-config, embedded | Same Postgres (already required) |
 | **Consistency** | ACID, in-process | ACID, same DB as app data |
 | **Filtering** | Post-filter in Python | SQL WHERE clause |

docs/roadmap.md

@@ -20,6 +20,19 @@
 7. **Go-specific search improvements** -- Go MRR 0.200 lags Python 0.725; index package docs, use module paths
 8. **ast-grep integration** -- optional structural pattern searches alongside tree-sitter parsing
 
+## v0.2.15 -- Search Performance & New Search Modes (Released 2026-04-04)
+
+1. ~~**BM25 keyword index disk cache**~~ -- DONE: SQLite cache with incremental mtime-based updates; 8x speedup on cockroach-scale repos (20s → 2.5s warm)
+2. ~~**Trigram pre-filtering for BM25**~~ -- DONE: narrows candidate docs before scoring; zero accuracy loss (full corpus IDF preserved)
+3. ~~**Numpy-accelerated vector search**~~ -- DONE: BLAS matmul replaces Python loop; 183x speedup at 10K vectors; in-memory cache with version invalidation; pure Python fallback
+4. ~~**Frecency-boosted search**~~ -- DONE: SQLite-backed file access scoring with exponential decay; `frecency_search` MCP tool with two-phase file ordering
+5. ~~**Fuzzy search (Smith-Waterman)**~~ -- DONE: typo-resistant search via local sequence alignment; `fuzzy_search`, `fuzzy_filename_search`, `fuzzy_score` tools
+6. ~~**Cross-mode search suggestions**~~ -- DONE: "did you mean" fallbacks between filename and content search
+7. ~~**Query constraints (fff-style)**~~ -- DONE: `git:modified`, `!pattern`, `path/`, `*.ext` filters with git porcelain XY parsing
+8. ~~**Query history & combo boosting**~~ -- DONE: SQLite-backed query-to-file tracking; 3+ selections activate combo boost
+9. ~~**Code-intel testing infrastructure**~~ -- DONE: fixtures, mocks, helpers; 9 tool test modules
+10. ~~**Overall benchmark improvement**~~ -- DONE: 35% faster (4,182ms → 2,731ms avg), quality stable at 4.7/5
+
 ## v0.2.x -- Code Intel Infrastructure
 
 1. **Cross-repo search in org** -- aggregate embeddings across repositories, org-scoped vector queries

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "attocode"
-version = "0.2.14"
+version = "0.2.15"
 description = "Production AI coding agent"
 readme = "README.md"
 requires-python = ">=3.12"
@@ -201,7 +201,7 @@ exclude_lines = [
 ]
 
 [tool.bumpversion]
-current_version = "0.2.14"
+current_version = "0.2.15"
 commit = false
 tag = false
 

src/attocode/__init__.py

@@ -1,3 +1,3 @@
 """Attocode - Production AI coding agent."""
 
-__version__ = "0.2.14"
+__version__ = "0.2.15"

src/attocode/code_intel/_shared.py

@@ -222,3 +222,12 @@ def _get_explorer():
         ast_svc = _get_ast_service()
         _explorer = HierarchicalExplorer(ctx, ast_service=ast_svc)
     return _explorer
+
+
+def _get_frecency_tracker():
+    """Get shared frecency tracker (thread-safe via get_tracker lock)."""
+    from attocode.integrations.context.frecency import get_tracker
+
+    project_dir = _get_project_dir()
+    db_path = os.path.join(project_dir, ".attocode", "frecency")
+    return get_tracker(db_path=db_path)

src/attocode/code_intel/server.py

@@ -60,23 +60,24 @@
 import threading
 from pathlib import Path
 
+import attocode.code_intel._shared as _shared  # noqa: F401
+
 # ---------------------------------------------------------------------------
 # Shared deps (mcp instance, lazy singletons, getters) live in _shared.py
 # to break the circular import: server.py → tool modules → server.py.
 # Re-exported here for backward compatibility.
 # ---------------------------------------------------------------------------
 from attocode.code_intel._shared import (  # noqa: F401
-    mcp,
-    clear_remote_service,
-    configure_remote_service,
-    _get_project_dir,
-    _walk_up,
     _get_ast_service,
-    _get_context_mgr,
     _get_code_analyzer,
+    _get_context_mgr,
     _get_explorer,
+    _get_project_dir,
+    _walk_up,
+    clear_remote_service,
+    configure_remote_service,
+    mcp,
 )
-import attocode.code_intel._shared as _shared  # noqa: F401
 
 logger = logging.getLogger(__name__)
 
@@ -556,12 +557,17 @@ def _instrument_all_tools() -> None:
 
 import attocode.code_intel.tools.adr_tools as _adr_tools  # noqa: E402, F401
 import attocode.code_intel.tools.analysis_tools as _analysis_tools  # noqa: E402, F401
+import attocode.code_intel.tools.cross_mode_tools as _cross_mode_tools  # noqa: E402, F401
 import attocode.code_intel.tools.dead_code_tools as _dead_code_tools  # noqa: E402, F401
 import attocode.code_intel.tools.distill_tools as _distill_tools  # noqa: E402, F401
+import attocode.code_intel.tools.frecency_tools as _frecency_tools  # noqa: E402, F401
+import attocode.code_intel.tools.fuzzy_tools as _fuzzy_tools  # noqa: E402, F401
 import attocode.code_intel.tools.history_tools as _history_tools  # noqa: E402, F401
 import attocode.code_intel.tools.learning_tools as _learning_tools  # noqa: E402, F401
 import attocode.code_intel.tools.lsp_tools as _lsp_tools  # noqa: E402, F401
 import attocode.code_intel.tools.navigation_tools as _navigation_tools  # noqa: E402, F401
+import attocode.code_intel.tools.query_constraints_tools as _query_constraints_tools  # noqa: E402, F401
+import attocode.code_intel.tools.query_history_tools as _query_history_tools  # noqa: E402, F401
 import attocode.code_intel.tools.readiness_tools as _readiness_tools  # noqa: E402, F401
 import attocode.code_intel.tools.search_tools as _search_tools  # noqa: E402, F401
 from attocode.code_intel.helpers import (  # noqa: E402, F401

src/attocode/code_intel/testing/__init__.py

@@ -0,0 +1,43 @@
+"""Testing utilities for attocode-code-intel.
+
+This module provides standardized fixtures, mocks, and helpers for
+testing code intelligence tools and MCP integrations.
+
+Example usage in tests::
+
+    import pytest
+    from attocode.code_intel.testing import (
+        code_intel_service,
+        ast_service,
+        sample_project,
+        MockServiceFactory,
+        create_sample_project,
+    )
+
+    class TestSearchTools:
+        @pytest.fixture(autouse=True)
+        def setup(self, code_intel_service, ast_service):
+            self.service = code_intel_service
+            self.ast = ast_service
+
+        def test_symbol_search(self):
+            result = search_tools.symbol_search(symbol_name="foo")
+            assert "foo" in result
+"""
+
+from attocode.code_intel.testing.fixtures import (
+    ast_service,
+    code_intel_service,
+    sample_project,
+)
+from attocode.code_intel.testing.mocks import MockServiceFactory
+from attocode.code_intel.testing.helpers import create_sample_project, get_tool_names
+
+__all__ = [
+    "ast_service",
+    "code_intel_service",
+    "sample_project",
+    "MockServiceFactory",
+    "create_sample_project",
+    "get_tool_names",
+]