diff --git a/README.md b/README.md index 8aaf585..927a29c 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,8 @@ and testing tools. The quotes prevent the shell from interpreting the brackets. Rerun the command after changing project metadata or dependencies; editable mode only makes source-code changes immediately available. -Run `.venv/bin/ancestry` with no arguments for the interactive console. The +Run `.venv/bin/ancestry` with no arguments for the prompt-toolkit/Rich +interactive console. It is the only supported interactive console; the canonical command reference, examples, offline defaults, and privacy rules are in [the CLI guide](docs/CLI.md); see [the console guide](docs/CONSOLE.md) for interactive use. diff --git a/docs/CONSOLE.md b/docs/CONSOLE.md index eccccdf..c4af938 100644 --- a/docs/CONSOLE.md +++ b/docs/CONSOLE.md @@ -1,21 +1,15 @@ # Console guide -Running `ancestry` with no arguments starts the default asynchronous -`prompt_toolkit` REPL installed by the main package. The REPL is an input and -presentation adapter over the same command specifications and application -services used by one-shot execution. One-shot usage is unchanged: -`ancestry MODULE ACTION ...` continues to parse and dispatch the documented -CLI grammar. +Running `ancestry` with no arguments starts the supported interactive console. +The console is the asynchronous prompt-toolkit/Rich REPL installed by the main +package. It is the only interactive console surface; one-shot usage is +unchanged: `ancestry MODULE ACTION ...` continues to parse and dispatch the +documented CLI grammar. -The legacy `cmd2` console remains available temporarily with: - -```text -ancestry --legacy-console -``` - -This switch is a compatibility fallback during the migration. It does not -change one-shot behavior, and final removal of the `cmd2` implementation is -future work after parity and security validation. +The REPL is an input and presentation adapter over the same transport-neutral +command specifications and application services used by one-shot execution. It +does not provide a shell, Python evaluator, plugin loader, or separate business +logic path. ## Navigation and controls @@ -60,6 +54,20 @@ execution, script loading, aliases/macros, command substitution or other expansion, pipes, redirects, and related shell syntax. There is no command path that evaluates user input as Python or generated code. +## Command registration + +Built-in modules are registered in the explicit module registry as +`ModuleDescriptor` entries with matching transport-neutral `CommandSpec` +metadata. The same command metadata drives one-shot argparse wiring, REPL +routing, help text, completion, and validation. Interactive commands are +therefore not authored as terminal-specific command classes; they are exposed +through descriptors, action specifications, argument specifications, and thin +service dispatchers that return serializable DTOs or stable coded errors. + +A module can be disabled through configuration only when it is present in the +registry. Disabled modules are not available through direct root commands, +module context navigation, or completion. + ## Tab completion Completion is context-aware and read-only. At the root it offers commands and diff --git a/docs/MODULE_AUTHORING.md b/docs/MODULE_AUTHORING.md index 00c6fe4..05097f0 100644 --- a/docs/MODULE_AUTHORING.md +++ b/docs/MODULE_AUTHORING.md @@ -1,12 +1,33 @@ # Built-in module authoring -A module declares a `ModuleDescriptor` and a `cmd2.CommandSet` in the explicit -registry. Its console adapter delegates to a service; it must not open storage, -read secrets, call providers, or implement business rules directly. Add the same -argument parser action to the one-shot dispatcher and command set so behavior and -errors remain identical. The user-facing command inventory belongs in -[the CLI reference](CLI.md), rather than module-specific documentation. - -Modules are built-in only in v1. Add tests proving disabled modules are not -imported, commands are unavailable, service DTOs serialize, secrets stay out of -history, and offline defaults make no network calls. +A built-in module is registered through the explicit module registry with a +`ModuleDescriptor` and transport-neutral `CommandSpec`. The descriptor records +the module identity, summary, implementation path, and supported action names. +The command specification records action metadata and typed arguments used by +both one-shot CLI execution and the prompt-toolkit/Rich REPL. + +Module authors should add or update the command specification first, then wire +the action to a thin dispatcher that delegates to an application service. The +dispatch layer must not open storage directly, read secrets, call providers, or +implement business rules. Services return serializable DTOs, progress events, +or stable `AncestryError` instances so terminal, JSON, and future adapters can +present the same result contract. + +Interactive behavior is derived from the shared metadata: + +- one-shot `ancestry MODULE ACTION ...` parsing and validation; +- root-level direct module commands in the REPL; +- active-module `run ACTION ...` routing; +- help text, action listings, and option validation; and +- privacy-filtered completion. + +Do not author terminal-specific command classes for new modules. A module +should not depend on prompt-toolkit, Rich, or any console input framework. Rich +rendering belongs in presentation adapters only, and JSON output must remain a +serialization of the same service result. + +The user-facing command inventory belongs in [the CLI reference](CLI.md), +rather than module-specific documentation. Modules are built-in only in v1. Add +tests proving disabled modules are not imported, commands are unavailable, +service DTOs serialize, secrets stay out of history, and offline defaults make +no network calls. diff --git a/docs/PRIVACY_AND_CONSENT.md b/docs/PRIVACY_AND_CONSENT.md index a1fee6a..8efbf21 100644 --- a/docs/PRIVACY_AND_CONSENT.md +++ b/docs/PRIVACY_AND_CONSENT.md @@ -14,3 +14,25 @@ input/output is stored only with explicit retention consent in SQLCipher. The research workspace is curated supporting data, not the authoritative family tree. Store provenance and RootsMagic/GEDCOM identifiers so claims can be traced without copying an entire tree into the workspace. + + +## Interactive console privacy + +The only supported interactive console is the prompt-toolkit/Rich REPL. It uses +the same command specifications, provider policy, and consent checks as one-shot +CLI execution; there is no separate interactive path that can bypass consent or +provider selection. Session options are non-secret, secret-like option names are +rejected, and secret entry must go through no-echo `secrets` commands backed by +the OS-keyring service. + +Completion is privacy-filtered and read-only. It may use command metadata, +static enum values, enabled module names, startup snapshots of configured +profile and consent names, static secret-reference types, and bounded local file +listings for file-valued arguments. It must not query databases, keyrings, +providers, networks, people, trees, prompts, workspaces, prompt names, or secret +values. + +Interactive history is stored with owner-only permissions. Secret entry and +secret-like commands are excluded from history and defensively redacted from +persisted history. Do not paste credentials, private genealogy records, or +prompt/response payloads into ordinary commands. diff --git a/docs/REPL_ARCHITECTURE.md b/docs/REPL_ARCHITECTURE.md index e427c5f..4f13c3d 100644 --- a/docs/REPL_ARCHITECTURE.md +++ b/docs/REPL_ARCHITECTURE.md @@ -2,23 +2,23 @@ The interactive console is a local UI over the same application services used by the one-shot CLI. The consolidated Issues #46–#49 implementation makes the -asynchronous `prompt_toolkit` REPL, installed by the main package, the default -no-argument interface while preserving the one-shot interface and a temporary -legacy fallback. +asynchronous prompt-toolkit/Rich REPL, installed by the main package, the only +supported no-argument interactive interface while preserving the one-shot +interface. ## Decision record -**Status:** Implemented for the default shell and context-aware completion; -background jobs/cooperative cancellation and final `cmd2` removal remain -future migration work. +**Status:** Implemented for the prompt-toolkit/Rich REPL and context-aware +completion. Interactive console consolidation is complete; remaining future +work is limited to features such as structured background jobs and cooperative +cancellation. The application uses transport-neutral command specifications and invocation/result contracts between input adapters and application services. `ancestry MODULE ACTION ...` remains the one-shot compatibility authority. A -no-argument `ancestry` invocation starts the asynchronous `prompt_toolkit` -REPL, while `ancestry --legacy-console` starts the temporary `cmd2` fallback. -No API, WebUI, multi-user server, autonomous agent, Python execution, or LLM -tool-execution capability is part of this decision. +no-argument `ancestry` invocation starts the asynchronous prompt-toolkit/Rich +REPL. No API, WebUI, multi-user server, autonomous agent, Python execution, or +LLM tool-execution capability is part of this decision. ## Target layers @@ -83,10 +83,9 @@ dependency. history, and interactive history is stored with owner-only permissions. - Provider selection and consent stay explicit. `provider=none` remains network-free even when keys or provider SDKs are installed. -- Background jobs, progress management, and cooperative cancellation are not - yet part of the completed migration. Final removal of `cmd2` is also future - work; the `--legacy-console` switch remains until parity and security gates - are satisfied. +- Structured background jobs, progress management, and cooperative cancellation + are not part of the completed console consolidation. They require their own + lifecycle and shutdown design before implementation. ## Migration status @@ -94,24 +93,35 @@ dependency. |---|---|---| | Shared command specifications and typed invocation contracts | Implemented | One-shot and REPL use the same command metadata | | Session routing and root/active-module controls | Implemented | State is non-secret and UI-independent | -| Default asynchronous `prompt_toolkit` REPL | Implemented | Starts with no arguments and is installed by the main package | +| Asynchronous prompt-toolkit/Rich REPL | Implemented | Starts with no arguments and is installed by the main package | | Context-aware completion | Implemented | Static/snapshot-driven, privacy-filtered, CWD-bounded | | Secure history and no-echo secret entry | Implemented | Owner-only history; secrets excluded and redacted | | Background jobs and cooperative cancellation | Future work | Requires structured job lifecycle and atomic shutdown behavior | -| Final `cmd2` removal | Future work | Retain `ancestry --legacy-console` until parity/security validation | ## Compatibility paths ```text ancestry MODULE ACTION ... -> unchanged one-shot parser and dispatcher -ancestry -> asynchronous prompt_toolkit REPL -ancestry --legacy-console -> temporary cmd2 compatibility console +ancestry -> asynchronous prompt-toolkit/Rich REPL ``` -The fallback is an explicit migration aid, not a second application service -path. All supported adapters must preserve one-shot semantics, provider policy, +The REPL and one-shot CLI are sibling adapters, not separate application service +paths. All supported adapters must preserve one-shot semantics, provider policy, stable errors, and source-file safety guarantees. +## Command registration model + +Built-in commands are declared once in the explicit module registry. Each +module has a `ModuleDescriptor` for identity and implementation location plus a +transport-neutral `CommandSpec` containing its actions and typed arguments. The +one-shot CLI parser, REPL router, help output, active-module `run` command, and +completion all consume that metadata. + +Module implementations stay thin: they receive parsed, typed invocation data, +delegate to application services, and return serializable DTOs or stable coded +errors. They do not own terminal input, Rich rendering, storage, provider +selection, consent policy, or secret retrieval. + ## Explicitly rejected shortcuts - Calling application services directly from completion or input widgets. @@ -130,11 +140,12 @@ stable errors, and source-file safety guarantees. ## Allowed dependencies `prompt-toolkit` is a main-package dependency and supplies the asynchronous -prompt, completion primitives, history support, and terminal editing. The -project-specific completion adapter composes those public primitives with the -existing command specifications and privacy policy; it does not introduce a -second completion library or a new runtime framework dependency. +prompt, completion primitives, history support, and terminal editing. Rich is +used only by presentation adapters for terminal rendering. The project-specific +completion adapter composes prompt-toolkit public primitives with the existing +command specifications and privacy policy; it does not introduce a second +completion library or a new runtime framework dependency. The one-shot CLI and REPL are sibling adapters over the same execution and -service contracts. Any implementation that makes services depend on `cmd2`, -`prompt_toolkit`, or Rich remains outside the target architecture. +service contracts. Any implementation that makes services depend on +`prompt_toolkit` or Rich remains outside the target architecture. diff --git a/docs/THREAT_MODEL.md b/docs/THREAT_MODEL.md index 05b4f40..c4b5590 100644 --- a/docs/THREAT_MODEL.md +++ b/docs/THREAT_MODEL.md @@ -4,7 +4,8 @@ Sensitive assets are genealogy records, living-person status, notes, provider credentials, SQLCipher keys, prompts/responses, consent grants, and RootsMagic -source files. Data crosses boundaries at console input, GEDCOM/RootsMagic parsing, +source files. Data crosses boundaries at prompt-toolkit/Rich REPL input, +one-shot CLI input, GEDCOM/RootsMagic parsing, the OS keyring, encrypted database, configured provider endpoints, and exported files. The local operator is trusted to choose data and consent; imported genealogy content and every model response are untrusted. @@ -21,11 +22,11 @@ untrusted GEDCOM / RootsMagic -> bounded parsers -> application services | Risk | Applicability and controls | Verification | |---|---|---| | A01 Broken Access Control | Single-user local v1; paths are scoped, module registry is explicit, RootsMagic is immutable. Future API must add authentication/authorization. | Path traversal and disabled-module tests. | -| A02 Security Misconfiguration | Secure defaults, no network provider by default, no shell/redirection, restrictive permissions, bounded limits. | Clean-install and console tests. | +| A02 Security Misconfiguration | Secure defaults, no network provider by default, prompt-toolkit/Rich REPL as the only interactive console, no shell/redirection, restrictive permissions, bounded limits. | Clean-install and console tests. | | A03 Software Supply Chain Failures | Locked dependencies, optional provider extras, Dependabot, audit, SBOM, pinned CI actions. | `uv lock`, `pip-audit`, CycloneDX. | | A04 Cryptographic Failures | SQLCipher required; 256-bit random key in OS keyring; plaintext and wrong keys rejected; encrypted backups. | Header, wrong-key, integrity, backup tests. | -| A05 Injection | SQL AST validation, allowlisted schema, SQLite authorizer; no generated command/code execution; prompt content delimited as data. | SQL/prompt injection tests and Semgrep. | -| A06 Insecure Design | Separate adapters/services, explicit consent, data minimization, conservative deletions, threat review. | Architecture and invariant tests. | +| A05 Injection | SQL AST validation, allowlisted schema, SQLite authorizer; no generated command/code execution; prompt content delimited as data; REPL parsing is strict and transport-neutral. | SQL/prompt/console injection tests and Semgrep. | +| A06 Insecure Design | Separate adapters/services, explicit consent, explicit module registry and command specifications, data minimization, conservative deletions, threat review. | Architecture and invariant tests. | | A07 Authentication Failures | Not applicable to the single-user local adapter; OS keyring supplies platform authentication. API remains out of scope. | Future-API release blocker. | | A08 Software or Data Integrity Failures | Source hashes, SQLCipher integrity, validated structured output, immutable prompt revisions, atomic writes. | Hash, schema, round-trip, rollback tests. | | A09 Security Logging and Alerting Failures | Stable error codes and privacy-minimal run metadata; payload logging off; secret redaction. Local v1 has no remote alerting. | Error/redaction tests; documented limitation. | @@ -40,7 +41,7 @@ untrusted GEDCOM / RootsMagic -> bounded parsers -> application services | LLM03 Supply Chain | Provider SDKs are optional and locked; dependency/SBOM/security scans gate release. | | LLM04 Data and Model Poisoning | Retrieval is not implemented. Any future local index must fingerprint sources, preserve provenance, treat retrieved text as untrusted context, and detect stale/conflicting material before display or generation. | | LLM05 Improper Output Handling | JSON Schema validation and length caps; output is never executable. | -| LLM06 Excessive Agency | No autonomous agents, tool calls, shell, write-capable SQL, or automatic destructive decisions. | +| LLM06 Excessive Agency | No autonomous agents, tool calls, shell, interactive-console escape hatch, write-capable SQL, or automatic destructive decisions. | | LLM07 System Prompt Leakage | Prompts contain no credentials; templates and untrusted content are separated; disclosure is treated as possible. | | LLM08 Vector and Embedding Weaknesses | Embeddings/vector stores remain unimplemented. A future feature requires SQLCipher-local storage by default, workspace and consent partitioning, restricted-data exclusion, versioned invalidation, bounded retrieval, and explicit cloud-retention consent. | | LLM09 Misinformation | Deterministic evidence remains authoritative; LLM adjudication is optional and cannot delete conflicts. |