[EL-34] Add the encrypted local retrieval and evidence-discovery workspace

[sodejm]

Catalog ID: EL-34

Public issue: #130

Reviewed baseline: 71abaf4067d4118f28910a22c20524b4e0282b28

Core objective: Provide local retrieval and evidence discovery across
explicitly indexed, encrypted user data with citation-first results,
living-person safeguards, bounded indexing, and no implicit remote disclosure.

Priority / phase: P3 / Post-MVP retrieval

Publication status: Published as #130.

Dependency mapping

• Blocks: None
• Depends On: #106, #109, #123, Research: evaluate local-first embeddings and retrieval for genealogy notes #17, Feature: add token preflight and local genealogy context materialization #23
• Relates To: #128

UX/UI requirements

• Provide source selection, indexing status, scope filters, evidence snippets,
  citations, confidence explanations, and stale-index warnings.
• Separate exact evidence matches from generated summaries.
• Make living-person filtering and remote-disclosure state visible.
• Preview the effect of delete-index, rebuild, and source removal.

Public interfaces

• Add RetrievalSource, IndexJob, EvidenceQuery, EvidenceHit,
  CitationReference, and IndexDeletionPreview.
• Return bounded snippets and opaque source references, not paths/full records.
• Any future remote augmentation is a separate explicit operation with provider
  and consent fields.

Underlying app changes

• Complete a renewed threat review before implementation.
• Add encrypted index storage, schema/version metadata, source fingerprints,
  bounded ingestion, deletion, rebuild, and stale-source detection.
• Keep deterministic retrieval independent from model generation.
• Filter living-person data before any provider boundary.

Acceptance criteria

• The renewed threat review covers confidentiality, inference leakage,
  deletion, poisoning, living-person data, and remote augmentation.
• Only explicitly selected sources are indexed.
• Index data is encrypted at rest and unusable without key access.
• Results include stable citations and never present generation as evidence.
• Source change/revocation marks results stale.
• Verified index deletion removes source-derived searchable content.
• provider=none retrieval performs no network access.
• Poisoning, oversized sources, hostile queries, and deletion races have
  regression tests.

Security mitigations and rejected anti-patterns

• Applies TM-S01, TM-L01, TM-L02, TM-F01, TM-A02, TM-D01, and TM-O01.
• Reject plaintext vector stores, complete records in renderer, implicit global
  indexing, cloud embeddings without consent, provider inference, network
  fallback, uncited generated answers, and payload logs.

Workspace mapping

• Exclusive owned paths: desktop/src/features/retrieval/**,
  src/ancestryllm/api/routes/retrieval/**,
  src/ancestryllm/services/retrieval/**,
  tests/services/retrieval/**, desktop/tests/retrieval/**, and
  docs/threat-models/retrieval.md.
• Shared-file rule: Encryption, migrations, provider policy, generated
  contracts, dependencies, and lockfiles require coordinator ownership.
• Base rule: Branch after dependencies merge and renewed review approval.
• Branch: feature/electron-el-34-local-retrieval
• Worktree: /Users/justinsoderberg/Development/worktrees/ancestryllm-electron-el-34-local-retrieval
• Validation: make test; make lint; make typecheck; make security;
  pnpm --dir desktop test; pnpm --dir desktop typecheck; encryption and
  network-denial tests

[sodejm]

Recommended Codex configuration

• Planning: gpt-5.6-sol with high reasoning
• Implementation: gpt-5.6-sol with xhigh reasoning
• Rationale: This issue includes security, privacy, concurrency, provider-boundary, or genealogy data-integrity concerns that require deeper implementation reasoning and verification.
• Delivery: Use test-case-driven development: translate the acceptance criteria into failing or characterization tests first, implement incrementally, then run the applicable tests, lint, typecheck, security, integration, and hosted gates.