Skip to content

Commit 1f09150

Browse files
authored
feat(status): redesign status command; add validate; add export sqlite (#311) (#400)
* feat(status): redesign status command; add validate command; add export sqlite - status: four verbosity levels (compact/normal/verbose/extra-verbose, default normal) - compact: one-line CI summary - normal: verdict list grouped COMPLETE-first then INCOMPLETE with per-req reason - verbose: today's per-req table without %-breakdown blocks - extra-verbose: verdict list with per-req SVC/MVR/test drill-down - status: --incomplete flag to narrow to incomplete requirements only - status: --req-ids/--svc-ids filtering for --format json - status: --verbosity is console-only; warns+ignores when --format json - status: --group-by/--sort-by removed from status subparsers (not applicable) - export: add --format sqlite via SQLite backup API (requires -o <file>) - validate: new command surfacing SemanticValidator + coverage checks - reports requirements without SVCs and manual SVCs without MVRs - referential errors = fail; coverage gaps = warning (--strict to fail) - remove deprecated generate-json command - export_service: add resolve_filter_scope() public helper - database: add backup_to(path) method (clears authorizer for backup API) Closes #311 Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * fix(status): address code review findings - _filtered_status_dict: use `is not None` instead of truthiness check so --req-ids with a nonexistent ID correctly returns zero requirements (empty set is falsy and was treated as "no filter applied") - backup_to: restore authorizer in finally block so any post-backup queries on the same connection still run under the security policy - StatisticsService: add public `initial_urn` property; remove direct `_repo` access from status rendering functions Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * refactor(status): apply full-pr-review findings - Extract VerbosityLevel enum; use in command.py choices and status.py match - Rename incomplete → incomplete_only consistently in Python code (CLI --incomplete unchanged) - Extract _status_verdict(), _build_sections() to eliminate duplication between _status_normal and _status_extra_verbose - Add type annotations to _incomplete_reasons, _build_drill_down_block, _render_mvrs, _render_test_results (_build_drill_down_block imports UrnId, SVCData, MVRData) - getattr(export_args, "no_filters", False) for defensive access consistency - Clarify backup_to() docstring (always restores authorizer, even on error) - Help text on --req-ids / --svc-ids: note they follow the location subcommand - Tests: strict-no-warnings exits 0; referential errors exit 1; non-IN_CODE _incomplete_reasons; --verbosity warning logged for --format json; VerbosityLevel enum - Fix sqlite conn leak in test (close() in finally block) - Restore removed explanatory comments in _summarize_statistics Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * style: apply black formatting Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * fix(tests): rename ambiguous variable l → line (E741) Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * fix: commit before sqlite backup; update test_command for removed generate-json - database.py: commit open transaction before backup_to() so Connection.backup() does not block waiting for write lock (populator leaves transaction uncommitted) - test_command.py: replace generate-json tests (removed command) with validate routing test Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * fix(e2e): update generate-json e2e test to verify command was removed Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * docs(usage): update for status redesign, new validate command, export sqlite - status: replace stale table example with new verbosity-level reference (compact/normal/verbose/extra-verbose), document --incomplete flag, note that --verbosity is console-only - validate: add new command section (spec completeness, --strict, example output) - export: document --format sqlite option; remove generate-json section (command removed) - synopsis: add validate to command list Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * docs: update index and how_it_works for status/validate changes - index.adoc: expand feature list to mention spec validation and sqlite export - how_it_works.adoc: add validate to command component label; update sequence diagram end label from "status table" to "status verdict" Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> * refactor(status): remove dead _summarize_statistics and its tests The %-breakdown tables (In Code / Config / Platform / Framework / N/A + test & SVC totals) were removed from default status output in the #311 redesign. No CLI command calls _summarize_statistics; its only callers were the unit tests themselves. Remove function + private helpers (__numbers_as_percentage, __colorize_headers) + 9 test cases. Also drop now-unused Columns and TotalStats imports. Signed-off-by: Jimisola Laursen <jimisola@jimisola.com> --------- Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
1 parent 17543ba commit 1f09150

16 files changed

Lines changed: 1043 additions & 440 deletions

File tree

docs/modules/ROOT/pages/how_it_works.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ See xref:usage.adoc#mvr-supersession[MVR supersession] for the YAML format rules
5757
@startuml
5858
!include <C4/C4_Component>
5959
60-
Component(Command, "Command", "Handles user commands (status, report, export)")
60+
Component(Command, "Command", "Handles user commands (status, validate, report, export)")
6161
Component(CombinedRawDatasetsGenerator, "CombinedRawDatasetsGenerator", "Two-phase graph traversal and SQLite population")
6262
Component(DatabaseFilterProcessor, "DatabaseFilterProcessor", "Post-parse requirement/SVC filtering")
6363
Component(RequirementsRepository, "RequirementsRepository", "Data access layer over SQLite")
@@ -109,7 +109,7 @@ Rel(reqsTool, impl_reqs, "parse + metadata only")
109109
Rel(reqsTool, impl_svcs, "parse + FK-scoped insert")
110110
Rel(reqsTool, impl_tests, "parse + scoped insert")
111111
Rel(reqsTool, reqsTool, "post-parse: delete impl-child requirements")
112-
Rel(reqsTool, user, "status table (exit code = unmet requirements)")
112+
Rel(reqsTool, user, "status verdict (exit code = unmet requirements)")
113113
114114
@enduml
115115
....

docs/modules/ROOT/pages/index.adoc

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,10 @@ The reqstool command line client is the core tool for managing requirements trac
44

55
== Features
66

7-
* **Status checks** -- verify that all requirements are implemented and tested, with an exit code for CI/CD gates
7+
* **Status checks** -- verdict surface for developers and CI/CD: compact one-liner, per-requirement reason list, or full drill-down; exit code = unmet requirements
8+
* **Spec validation** -- check that every requirement has SVCs and every manual SVC has an MVR, without needing code or test results
89
* **AsciiDoc reports** -- generate detailed reports for auditors and stakeholders
9-
* **JSON export** -- raw data export for custom tooling
10+
* **Data export** -- full dataset as JSON or binary SQLite for custom tooling
1011

1112
== Quick links
1213

docs/modules/ROOT/pages/usage.adoc

Lines changed: 95 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
[source,bash]
44
----
5-
reqstool [-h] {command: report,export,status} {location: local,git,maven,pypi} ...
5+
reqstool [-h] {command: report,export,status,validate} {location: local,git,maven,pypi} ...
66
----
77

88
Use `-h/--help` for more information about each command and location.
@@ -63,63 +63,100 @@ imports:
6363
[[status]]
6464
== Command: status
6565

66-
Shows the status of all requirements -- whether they are implemented, have passing tests, and have manual verification results.
66+
Verdict surface for developers and CI/CD: shows which requirements are met, why the unmet ones are failing, and exits with the number of unmet requirements (`exit 0` = all met).
6767

6868
*Usage:*
6969
[source,bash]
7070
----
71-
reqstool status local -p path_to_dir
71+
reqstool status [--verbosity compact|normal|verbose|extra-verbose] [--incomplete] [--format console|json] \
72+
local -p path_to_dir [--req-ids ID...] [--svc-ids ID...]
73+
----
74+
75+
Options:
76+
77+
* `--verbosity` -- console detail level (default: `normal`); ignored for `--format json`
78+
* `--incomplete` -- show only incomplete requirements (console only)
79+
* `--format` -- `console` (default) or `json`
80+
* `--req-ids` / `--svc-ids` -- scope JSON output to specific IDs (must follow the location subcommand)
81+
* `--check-all-reqs-met` -- exit 1 unless every requirement is met
82+
83+
=== Verbosity levels
84+
85+
[cols="1,4"]
86+
|===
87+
| Level | Output
88+
89+
| `compact`
90+
| Single line -- ideal for CI logs.
91+
`ms-001: 6 requirements · 1 complete · 5 incomplete · FAIL`
92+
93+
| `normal` *(default)*
94+
| Verdict list grouped COMPLETE first, then INCOMPLETE with one reason per line.
95+
Actionable failures appear nearest the prompt.
96+
97+
| `verbose`
98+
| Per-requirement table (URN / ID / Implementation / Automated Tests / Manual Tests).
99+
100+
| `extra-verbose`
101+
| Verdict list with full drill-down per incomplete requirement: SVCs, individual test results (✓/✗), MVR pass/fail, and implementation annotations.
102+
|===
103+
104+
*Example -- `normal` output:*
105+
[source]
106+
----
107+
Requirements status · ms-001
108+
109+
COMPLETE (1)
110+
REQ_ext001_100 ext-001
111+
112+
INCOMPLETE (5)
113+
REQ_010 ms-001 automated test failed (2/3 passed)
114+
REQ_020 ms-001 manual verification failed
115+
REQ_sys001_505 sys-001 not implemented · automated test missing · manual result missing
116+
REQ_ext002_300 ext-002 automated test missing
117+
REQ_ext002_400 ext-002 manual result missing
118+
119+
1/6 complete · 5 incomplete · FAIL
120+
----
121+
122+
*Example -- show only what's still missing:*
123+
[source,bash]
124+
----
125+
reqstool status --incomplete --verbosity extra-verbose local -p path_to_dir
126+
----
127+
128+
[[validate]]
129+
== Command: validate
130+
131+
Checks spec completeness using only authored data (`requirements.yml`, `svcs.yml`, `mvrs.yml`) -- no code or test results required.
132+
Useful while authoring requirements, before any implementation exists.
133+
134+
Checks performed:
135+
136+
* Every requirement has at least one SVC defined.
137+
* Every manual SVC has at least one MVR defined.
138+
* All SVC/MVR/annotation cross-references point to existing IDs (referential integrity via `SemanticValidator`).
139+
140+
Referential errors are always fatal (exit 1). Coverage gaps are warnings by default; use `--strict` to promote them to errors (CI gate for spec completeness).
141+
142+
*Usage:*
143+
[source,bash]
144+
----
145+
reqstool validate [--strict] local -p path_to_dir
72146
----
73147

74148
*Example output:*
75149
[source]
76150
----
77-
╒═══════════════════════════════════════════════════════════════════════════╕
78-
│ REQUIREMENTS: 6 │
79-
╘═══════════════════════════════════════════════════════════════════════════╛
80-
╒═════════╤════════════════╤══════════════════╤═══════════════════╤════════════════╕
81-
│ URN │ ID │ Implementation │ Automated Tests │ Manual Tests │
82-
╞═════════╪════════════════╪══════════════════╪═══════════════════╪════════════════╡
83-
│ ms-001 │ REQ_010 │ Implemented │ T3 P2 F1 M1 │ T1 P1 │
84-
├─────────┼────────────────┼──────────────────┼───────────────────┼────────────────┤
85-
│ ms-001 │ REQ_020 │ Implemented │ T1 P1 M1 │ T1 F1 │
86-
├─────────┼────────────────┼──────────────────┼───────────────────┼────────────────┤
87-
│ sys-001 │ REQ_sys001_505 │ Missing │ T1 P1 M2 │ T0 M1 │
88-
├─────────┼────────────────┼──────────────────┼───────────────────┼────────────────┤
89-
│ ext-001 │ REQ_ext001_100 │ Implemented │ N/A │ T1 P1 │
90-
├─────────┼────────────────┼──────────────────┼───────────────────┼────────────────┤
91-
│ ext-002 │ REQ_ext002_300 │ Implemented │ T0 M1 │ N/A │
92-
├─────────┼────────────────┼──────────────────┼───────────────────┼────────────────┤
93-
│ ext-002 │ REQ_ext002_400 │ Implemented │ T1 P1 M1 │ T0 M1 │
94-
╘═════════╧════════════════╧══════════════════╧═══════════════════╧════════════════╛
95-
╒═══════════╤════════════╤════════════╤═════════════╤═════════════╕
96-
│ T = Total │ P = Passed │ F = Failed │ S = Skipped │ M = Missing │
97-
╘═══════════╧════════════╧════════════╧═════════════╧═════════════╛
98-
----
99-
100-
The requirements table shows for each requirement:
101-
102-
* *URN* -- the source system or microservice
103-
* *ID* -- the requirement identifier
104-
* *Implementation* -- how the requirement is fulfilled, determined by the `implementation` field in `requirements.yml`:
105-
** a number (green/red) -- `in-code` (default): count of `@Requirements` annotations found; red means none found
106-
** `configuration` -- `configuration`: fulfilled by application config (YAML properties, env vars); no annotation required
107-
** `platform` -- `platform`: fulfilled by infrastructure or platform layer (Kubernetes, load balancing, network policies); no annotation required
108-
** `framework` -- `framework`: fulfilled by framework auto-configuration (Spring Boot starters, OTel agents); no annotation required
109-
** `N/A` -- not applicable, intentionally deferred, or out of scope; no annotation required
110-
* *Automated Tests* -- test results from JUnit XML: T=total SVCs, P=passed, F=failed, S=skipped, M=missing (no test found for SVC)
111-
* *Manual Tests* -- manual verification results: T=total, P=passed, F=failed, M=missing.
112-
When multiple MVRs exist for the same SVC, only the one with the latest `date` counts (supersession); older ones are retained as audit history in the report but excluded from the verdict.
113-
114-
The summary panels below the table break down requirements by implementation type.
115-
The `In Code` panel shows total, verified, annotated-not-verified, and missing-annotation counts (all percentages relative to in-code requirements).
116-
The `N/A`, `Configuration`, `Platform`, and `Framework` panels each show three columns:
117-
118-
* *Total (% of all)* -- count of this type, as a percentage of _all_ requirements; shows the type's share of the full requirement set
119-
* *Verified* -- count verified, as a percentage of _this type_
120-
* *Not Verified* -- count not yet verified, as a percentage of _this type_
121-
122-
The exit code equals the number of unmet requirements, making it suitable for CI/CD gates (`exit 0` = all requirements met).
151+
Validating reqstool setup · ms-001
152+
153+
⚠ ms-001:SVC_026 manual-test — no MVR defined
154+
⚠ sys-001:SVC_sys001_600 manual-test — no MVR defined
155+
156+
2 warnings
157+
(use --strict to treat warnings as errors)
158+
----
159+
123160

124161
[[mvr-supersession]]
125162
=== MVR supersession
@@ -153,6 +190,7 @@ results:
153190
* `date` is *required* on every MVR that references a given SVC as soon as more than one MVR does. A missing `date` in that situation is a semantic validation error.
154191
* Two MVRs for the same SVC with the *exact same UTC moment* are a validation error — add a time component to disambiguate (e.g. `T09:00:00Z` vs `T14:30:00Z`).
155192

193+
156194
[[report]]
157195
== Command: report
158196

@@ -183,38 +221,27 @@ This command still works but delegates to the `report` command with `--format as
183221
[[export]]
184222
== Command: export
185223

186-
Exports requirement data in the specified format, with optional filters for specific requirements or SVCs.
224+
Full data dump for interchange, archival, and tooling.
225+
Two serialization formats of the same dataset:
226+
227+
* `json` (default) -- structured JSON conforming to `export_output.schema.json`
228+
* `sqlite` -- binary SQLite database written via the SQLite backup API; requires `-o <file>`
187229

188230
*Usage:*
189231
[source,bash]
190232
----
191233
reqstool export local -p path_to_requirements_dir -o path_to_output_file.json
234+
reqstool export --format sqlite local -p path_to_requirements_dir -o path_to_output.db
192235
reqstool export local -p path_to_requirements_dir --req-ids REQ_001 REQ_002
193236
reqstool export local -p path_to_requirements_dir --svc-ids SVC_001
194237
----
195238

196239
Options:
197240

198-
* `--format` -- output format (default: `json`)
199-
* `--req-ids` -- filter by specific requirement IDs
200-
* `--svc-ids` -- filter by specific SVC IDs
241+
* `--format` -- `json` (default) or `sqlite`
242+
* `--req-ids` / `--svc-ids` -- filter JSON output to specific IDs (must follow the location subcommand)
201243
* `--no-filters` -- disable requirement/SVC filters defined in YAML
202244

203-
[[generate-json]]
204-
== Command: generate-json (deprecated)
205-
206-
WARNING: `generate-json` is deprecated. Use `export --format json` instead.
207-
208-
Exports all requirement data as JSON.
209-
210-
*Usage:*
211-
[source,bash]
212-
----
213-
reqstool generate-json local -p path_to_requirements_dir -o path_to_output_file.json
214-
----
215-
216-
NOTE: There is currently no JSON Schema for this file and it is a raw dump of the data structures using https://github.com/jsonpickle/jsonpickle[jsonpickle]. The format might change at any time.
217-
218245
== Using the Docker image
219246

220247
You can also run reqstool from a container, using the same commands as above. Mount the paths to the input data and output directory:

0 commit comments

Comments
 (0)