Summary
Currently, leann list scans directories to discover indexes, which can be slow for large directory trees. A global registry system would make index discovery O(1) by tracking all index paths centrally.
Proposed Solution
Implement a global registry at ~/.leann/indexes.json that stores the paths of all LEANN indexes:
{
"indexes": [
{
"name": "my-docs",
"path": "/Users/foo/projects/my-project/.leann/indexes/my-docs",
"type": "cli",
"created_at": "2024-12-25T10:00:00Z"
},
{
"name": "app-index",
"path": "/Users/foo/apps/data/app-index.leann",
"type": "app",
"created_at": "2024-12-25T11:00:00Z"
}
]
}
Implementation Steps
- Register on build: When
leann build creates an index, automatically register it in the global registry
- Register on API usage: When apps create indexes via the Python API, register them as well
- Update
leann list: Read from registry instead of scanning directories
- Cleanup stale entries: Validate paths exist and remove stale entries periodically
- Migration: First run should scan existing indexes and populate the registry
Benefits
leann list becomes O(1) instead of O(n) directory scan
- Works correctly even when running from
$HOME or other large directories
- Provides a single source of truth for all LEANN indexes
Related
Summary
Currently,
leann listscans directories to discover indexes, which can be slow for large directory trees. A global registry system would make index discovery O(1) by tracking all index paths centrally.Proposed Solution
Implement a global registry at
~/.leann/indexes.jsonthat stores the paths of all LEANN indexes:{ "indexes": [ { "name": "my-docs", "path": "/Users/foo/projects/my-project/.leann/indexes/my-docs", "type": "cli", "created_at": "2024-12-25T10:00:00Z" }, { "name": "app-index", "path": "/Users/foo/apps/data/app-index.leann", "type": "app", "created_at": "2024-12-25T11:00:00Z" } ] }Implementation Steps
leann buildcreates an index, automatically register it in the global registryleann list: Read from registry instead of scanning directoriesBenefits
leann listbecomes O(1) instead of O(n) directory scan$HOMEor other large directoriesRelated