Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ and testing tools. The quotes prevent the shell from interpreting the brackets.
Rerun the command after changing project metadata or dependencies; editable mode
only makes source-code changes immediately available.

Run `.venv/bin/ancestry` with no arguments for the interactive console. The
Run `.venv/bin/ancestry` with no arguments for the prompt-toolkit/Rich
interactive console. It is the only supported interactive console; the
canonical command reference, examples, offline defaults, and privacy rules are
in [the CLI guide](docs/CLI.md); see [the console guide](docs/CONSOLE.md) for
interactive use.
Expand Down
38 changes: 23 additions & 15 deletions docs/CONSOLE.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,15 @@
# Console guide

Running `ancestry` with no arguments starts the default asynchronous
`prompt_toolkit` REPL installed by the main package. The REPL is an input and
presentation adapter over the same command specifications and application
services used by one-shot execution. One-shot usage is unchanged:
`ancestry MODULE ACTION ...` continues to parse and dispatch the documented
CLI grammar.
Running `ancestry` with no arguments starts the supported interactive console.
The console is the asynchronous prompt-toolkit/Rich REPL installed by the main
package. It is the only interactive console surface; one-shot usage is

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore the legacy-console compatibility note

In environments where operators still need the cmd2 fallback, this statement is inaccurate: src/ancestryllm/cli.py:349-352 still special-cases ancestry --legacy-console and launches AncestryConsole, and the regression test test_main_uses_default_shell_legacy_console_and_preserves_one_shot_dispatch still asserts that path works. Removing the documented compatibility path makes the console guide and migration status misleading until the reachable flag and shipped cmd2 console are actually removed.

Useful? React with 👍 / 👎.

unchanged: `ancestry MODULE ACTION ...` continues to parse and dispatch the
documented CLI grammar.

The legacy `cmd2` console remains available temporarily with:

```text
ancestry --legacy-console
```

This switch is a compatibility fallback during the migration. It does not
change one-shot behavior, and final removal of the `cmd2` implementation is
future work after parity and security validation.
The REPL is an input and presentation adapter over the same transport-neutral
command specifications and application services used by one-shot execution. It
does not provide a shell, Python evaluator, plugin loader, or separate business
logic path.

## Navigation and controls

Expand Down Expand Up @@ -60,6 +54,20 @@ execution, script loading, aliases/macros, command substitution or other
expansion, pipes, redirects, and related shell syntax. There is no command
path that evaluates user input as Python or generated code.

## Command registration

Built-in modules are registered in the explicit module registry as
`ModuleDescriptor` entries with matching transport-neutral `CommandSpec`
metadata. The same command metadata drives one-shot argparse wiring, REPL
routing, help text, completion, and validation. Interactive commands are
therefore not authored as terminal-specific command classes; they are exposed
through descriptors, action specifications, argument specifications, and thin
service dispatchers that return serializable DTOs or stable coded errors.

A module can be disabled through configuration only when it is present in the
registry. Disabled modules are not available through direct root commands,
module context navigation, or completion.

## Tab completion

Completion is context-aware and read-only. At the root it offers commands and
Expand Down
41 changes: 31 additions & 10 deletions docs/MODULE_AUTHORING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,33 @@
# Built-in module authoring

A module declares a `ModuleDescriptor` and a `cmd2.CommandSet` in the explicit
registry. Its console adapter delegates to a service; it must not open storage,
read secrets, call providers, or implement business rules directly. Add the same
argument parser action to the one-shot dispatcher and command set so behavior and
errors remain identical. The user-facing command inventory belongs in
[the CLI reference](CLI.md), rather than module-specific documentation.

Modules are built-in only in v1. Add tests proving disabled modules are not
imported, commands are unavailable, service DTOs serialize, secrets stay out of
history, and offline defaults make no network calls.
A built-in module is registered through the explicit module registry with a
`ModuleDescriptor` and transport-neutral `CommandSpec`. The descriptor records
the module identity, summary, implementation path, and supported action names.
The command specification records action metadata and typed arguments used by
both one-shot CLI execution and the prompt-toolkit/Rich REPL.

Module authors should add or update the command specification first, then wire
the action to a thin dispatcher that delegates to an application service. The
dispatch layer must not open storage directly, read secrets, call providers, or
implement business rules. Services return serializable DTOs, progress events,
or stable `AncestryError` instances so terminal, JSON, and future adapters can
present the same result contract.

Interactive behavior is derived from the shared metadata:

- one-shot `ancestry MODULE ACTION ...` parsing and validation;
- root-level direct module commands in the REPL;
- active-module `run ACTION ...` routing;
- help text, action listings, and option validation; and
- privacy-filtered completion.

Do not author terminal-specific command classes for new modules. A module
should not depend on prompt-toolkit, Rich, or any console input framework. Rich
rendering belongs in presentation adapters only, and JSON output must remain a
serialization of the same service result.

The user-facing command inventory belongs in [the CLI reference](CLI.md),
rather than module-specific documentation. Modules are built-in only in v1. Add
tests proving disabled modules are not imported, commands are unavailable,
service DTOs serialize, secrets stay out of history, and offline defaults make
no network calls.
22 changes: 22 additions & 0 deletions docs/PRIVACY_AND_CONSENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,25 @@ input/output is stored only with explicit retention consent in SQLCipher.
The research workspace is curated supporting data, not the authoritative family
tree. Store provenance and RootsMagic/GEDCOM identifiers so claims can be traced
without copying an entire tree into the workspace.


## Interactive console privacy

The only supported interactive console is the prompt-toolkit/Rich REPL. It uses
the same command specifications, provider policy, and consent checks as one-shot
CLI execution; there is no separate interactive path that can bypass consent or
provider selection. Session options are non-secret, secret-like option names are
rejected, and secret entry must go through no-echo `secrets` commands backed by
the OS-keyring service.

Completion is privacy-filtered and read-only. It may use command metadata,
static enum values, enabled module names, startup snapshots of configured
profile and consent names, static secret-reference types, and bounded local file
listings for file-valued arguments. It must not query databases, keyrings,
providers, networks, people, trees, prompts, workspaces, prompt names, or secret
values.

Interactive history is stored with owner-only permissions. Secret entry and
secret-like commands are excluded from history and defensively redacted from
persisted history. Do not paste credentials, private genealogy records, or
prompt/response payloads into ordinary commands.
63 changes: 37 additions & 26 deletions docs/REPL_ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,23 @@

The interactive console is a local UI over the same application services used
by the one-shot CLI. The consolidated Issues #46–#49 implementation makes the
asynchronous `prompt_toolkit` REPL, installed by the main package, the default
no-argument interface while preserving the one-shot interface and a temporary
legacy fallback.
asynchronous prompt-toolkit/Rich REPL, installed by the main package, the only
supported no-argument interactive interface while preserving the one-shot
interface.

## Decision record

**Status:** Implemented for the default shell and context-aware completion;
background jobs/cooperative cancellation and final `cmd2` removal remain
future migration work.
**Status:** Implemented for the prompt-toolkit/Rich REPL and context-aware
completion. Interactive console consolidation is complete; remaining future
work is limited to features such as structured background jobs and cooperative
cancellation.

The application uses transport-neutral command specifications and
invocation/result contracts between input adapters and application services.
`ancestry MODULE ACTION ...` remains the one-shot compatibility authority. A
no-argument `ancestry` invocation starts the asynchronous `prompt_toolkit`
REPL, while `ancestry --legacy-console` starts the temporary `cmd2` fallback.
No API, WebUI, multi-user server, autonomous agent, Python execution, or LLM
tool-execution capability is part of this decision.
no-argument `ancestry` invocation starts the asynchronous prompt-toolkit/Rich
REPL. No API, WebUI, multi-user server, autonomous agent, Python execution, or
LLM tool-execution capability is part of this decision.

## Target layers

Expand Down Expand Up @@ -83,35 +83,45 @@ dependency.
history, and interactive history is stored with owner-only permissions.
- Provider selection and consent stay explicit. `provider=none` remains
network-free even when keys or provider SDKs are installed.
- Background jobs, progress management, and cooperative cancellation are not
yet part of the completed migration. Final removal of `cmd2` is also future
work; the `--legacy-console` switch remains until parity and security gates
are satisfied.
- Structured background jobs, progress management, and cooperative cancellation
are not part of the completed console consolidation. They require their own
lifecycle and shutdown design before implementation.

## Migration status

| Area | Status | Boundary |
|---|---|---|
| Shared command specifications and typed invocation contracts | Implemented | One-shot and REPL use the same command metadata |
| Session routing and root/active-module controls | Implemented | State is non-secret and UI-independent |
| Default asynchronous `prompt_toolkit` REPL | Implemented | Starts with no arguments and is installed by the main package |
| Asynchronous prompt-toolkit/Rich REPL | Implemented | Starts with no arguments and is installed by the main package |
| Context-aware completion | Implemented | Static/snapshot-driven, privacy-filtered, CWD-bounded |
| Secure history and no-echo secret entry | Implemented | Owner-only history; secrets excluded and redacted |
| Background jobs and cooperative cancellation | Future work | Requires structured job lifecycle and atomic shutdown behavior |
| Final `cmd2` removal | Future work | Retain `ancestry --legacy-console` until parity/security validation |

## Compatibility paths

```text
ancestry MODULE ACTION ... -> unchanged one-shot parser and dispatcher
ancestry -> asynchronous prompt_toolkit REPL
ancestry --legacy-console -> temporary cmd2 compatibility console
ancestry -> asynchronous prompt-toolkit/Rich REPL
```

The fallback is an explicit migration aid, not a second application service
path. All supported adapters must preserve one-shot semantics, provider policy,
The REPL and one-shot CLI are sibling adapters, not separate application service
paths. All supported adapters must preserve one-shot semantics, provider policy,
stable errors, and source-file safety guarantees.

## Command registration model

Built-in commands are declared once in the explicit module registry. Each
module has a `ModuleDescriptor` for identity and implementation location plus a
transport-neutral `CommandSpec` containing its actions and typed arguments. The
one-shot CLI parser, REPL router, help output, active-module `run` command, and
completion all consume that metadata.

Module implementations stay thin: they receive parsed, typed invocation data,
delegate to application services, and return serializable DTOs or stable coded
errors. They do not own terminal input, Rich rendering, storage, provider
selection, consent policy, or secret retrieval.

## Explicitly rejected shortcuts

- Calling application services directly from completion or input widgets.
Expand All @@ -130,11 +140,12 @@ stable errors, and source-file safety guarantees.
## Allowed dependencies

`prompt-toolkit` is a main-package dependency and supplies the asynchronous
prompt, completion primitives, history support, and terminal editing. The
project-specific completion adapter composes those public primitives with the
existing command specifications and privacy policy; it does not introduce a
second completion library or a new runtime framework dependency.
prompt, completion primitives, history support, and terminal editing. Rich is
used only by presentation adapters for terminal rendering. The project-specific
completion adapter composes prompt-toolkit public primitives with the existing
command specifications and privacy policy; it does not introduce a second
completion library or a new runtime framework dependency.

The one-shot CLI and REPL are sibling adapters over the same execution and
service contracts. Any implementation that makes services depend on `cmd2`,
`prompt_toolkit`, or Rich remains outside the target architecture.
service contracts. Any implementation that makes services depend on
`prompt_toolkit` or Rich remains outside the target architecture.
11 changes: 6 additions & 5 deletions docs/THREAT_MODEL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@

Sensitive assets are genealogy records, living-person status, notes, provider
credentials, SQLCipher keys, prompts/responses, consent grants, and RootsMagic
source files. Data crosses boundaries at console input, GEDCOM/RootsMagic parsing,
source files. Data crosses boundaries at prompt-toolkit/Rich REPL input,
one-shot CLI input, GEDCOM/RootsMagic parsing,
the OS keyring, encrypted database, configured provider endpoints, and exported
files. The local operator is trusted to choose data and consent; imported
genealogy content and every model response are untrusted.
Expand All @@ -21,11 +22,11 @@ untrusted GEDCOM / RootsMagic -> bounded parsers -> application services
| Risk | Applicability and controls | Verification |
|---|---|---|
| A01 Broken Access Control | Single-user local v1; paths are scoped, module registry is explicit, RootsMagic is immutable. Future API must add authentication/authorization. | Path traversal and disabled-module tests. |
| A02 Security Misconfiguration | Secure defaults, no network provider by default, no shell/redirection, restrictive permissions, bounded limits. | Clean-install and console tests. |
| A02 Security Misconfiguration | Secure defaults, no network provider by default, prompt-toolkit/Rich REPL as the only interactive console, no shell/redirection, restrictive permissions, bounded limits. | Clean-install and console tests. |
| A03 Software Supply Chain Failures | Locked dependencies, optional provider extras, Dependabot, audit, SBOM, pinned CI actions. | `uv lock`, `pip-audit`, CycloneDX. |
| A04 Cryptographic Failures | SQLCipher required; 256-bit random key in OS keyring; plaintext and wrong keys rejected; encrypted backups. | Header, wrong-key, integrity, backup tests. |
| A05 Injection | SQL AST validation, allowlisted schema, SQLite authorizer; no generated command/code execution; prompt content delimited as data. | SQL/prompt injection tests and Semgrep. |
| A06 Insecure Design | Separate adapters/services, explicit consent, data minimization, conservative deletions, threat review. | Architecture and invariant tests. |
| A05 Injection | SQL AST validation, allowlisted schema, SQLite authorizer; no generated command/code execution; prompt content delimited as data; REPL parsing is strict and transport-neutral. | SQL/prompt/console injection tests and Semgrep. |
| A06 Insecure Design | Separate adapters/services, explicit consent, explicit module registry and command specifications, data minimization, conservative deletions, threat review. | Architecture and invariant tests. |
| A07 Authentication Failures | Not applicable to the single-user local adapter; OS keyring supplies platform authentication. API remains out of scope. | Future-API release blocker. |
| A08 Software or Data Integrity Failures | Source hashes, SQLCipher integrity, validated structured output, immutable prompt revisions, atomic writes. | Hash, schema, round-trip, rollback tests. |
| A09 Security Logging and Alerting Failures | Stable error codes and privacy-minimal run metadata; payload logging off; secret redaction. Local v1 has no remote alerting. | Error/redaction tests; documented limitation. |
Expand All @@ -40,7 +41,7 @@ untrusted GEDCOM / RootsMagic -> bounded parsers -> application services
| LLM03 Supply Chain | Provider SDKs are optional and locked; dependency/SBOM/security scans gate release. |
| LLM04 Data and Model Poisoning | Retrieval is not implemented. Any future local index must fingerprint sources, preserve provenance, treat retrieved text as untrusted context, and detect stale/conflicting material before display or generation. |
| LLM05 Improper Output Handling | JSON Schema validation and length caps; output is never executable. |
| LLM06 Excessive Agency | No autonomous agents, tool calls, shell, write-capable SQL, or automatic destructive decisions. |
| LLM06 Excessive Agency | No autonomous agents, tool calls, shell, interactive-console escape hatch, write-capable SQL, or automatic destructive decisions. |
| LLM07 System Prompt Leakage | Prompts contain no credentials; templates and untrusted content are separated; disclosure is treated as possible. |
| LLM08 Vector and Embedding Weaknesses | Embeddings/vector stores remain unimplemented. A future feature requires SQLCipher-local storage by default, workspace and consent partitioning, restricted-data exclusion, versioned invalidation, bounded retrieval, and explicit cloud-retention consent. |
| LLM09 Misinformation | Deterministic evidence remains authoritative; LLM adjudication is optional and cannot delete conflicts. |
Expand Down
Loading