feat: add curated knowledge stores with source-mapped extraction and file-backed persistence by jflowers · Pull Request #51 · unbound-force/dewey

jflowers · 2026-04-22T00:52:38Z

Summary

The largest feature in Dewey's history — transforms Dewey from a passive index into an active knowledge curator. Per org discussion #114.

Eight Capabilities

File-backed learnings (feat: persist learnings as markdown files for backup and portability #50): store_learning dual-writes to SQLite + .uf/dewey/learnings/{tag}-{seq}.md. Auto re-ingests orphaned files on startup.
Knowledge store config: .uf/dewey/knowledge-stores.yaml maps sources to named curated stores
Curation pipeline: New curate/ package — LLM extracts decisions/facts/patterns from indexed sources with source traceability
Quality analysis: Missing rationale, implied assumptions, incongruent info detection with confidence scoring (high/medium/low/flagged)
Background curation: Per-store goroutine polls at configurable intervals, shares indexing mutex via TryLock
Curated trust tier: authored > curated > validated > draft — filterable in semantic_search_filtered
Lint integration: Confidence distribution, quality flag counts, stale store detection
Auto-indexing: Curated files persisted as searchable pages with source_id=knowledge-{store-name}

Deliverables

New curate/ package (config + pipeline + quality analysis)
1 new MCP tool: curate (tool fix: dewey init scaffolds slash commands on re-init, not just first init (#48) #49)
1 new CLI command: dewey curate with --store, --force, --no-embeddings
1 new slash command: /dewey-curate
71 new tests across 7 packages
32 files changed, +7,508 lines

Spec Artifacts

Full Speckit spec in specs/015-curated-knowledge-stores/ — spec (8 user stories, 28 FRs), plan, research (7 items), quickstart (5 steps), 3 contracts, tasks (43 tasks, all complete).

…file-backed persistence - New curate/ package with curation pipeline: LLM extracts decisions, facts, patterns from indexed sources with quality analysis (missing_rationale, implied_assumption, incongruent) and confidence scoring (high/medium/low/flagged) - File-backed learnings: store_learning dual-writes to SQLite + .uf/dewey/learnings/{tag}-{seq}.md with auto re-ingestion on startup when graph.db is missing - Knowledge store config via .uf/dewey/knowledge-stores.yaml mapping sources to named curated stores - Background curation goroutine polls per-store at configurable intervals, sharing indexing mutex via TryLock - New curated trust tier (authored > curated > validated > draft) with semantic_search_filtered support - dewey lint extended with knowledge quality metrics: confidence distribution, quality flags, stale store detection - Auto-indexing: curated files persisted as searchable pages with source_id=knowledge-{store-name} - dewey curate CLI command + curate MCP tool (tool #49) + /dewey-curate slash command - 71 new tests across 7 packages

github-actions · 2026-04-22T00:54:33Z

✅⚠️MegaLinter analysis: Success with warnings

Descriptor	Linter	Files	Fixed	Errors	Warnings	Elapsed time
✅ GO	golangci-lint	yes	yes	no	no	47.24s
⚠️ MARKDOWN	markdownlint	2	0	156	0	1.9s
✅ REPOSITORY	gitleaks	yes		no	no	2.37s
✅ REPOSITORY	git_diff	yes		no	no	0.27s
✅ YAML	yamllint	1		0	0	0.81s

Detailed Issues

⚠️ MARKDOWN / markdownlint - 156 errors

AGENTS.md:5:401 error MD013/line-length Line length [Expected: 400; Actual: 656]
AGENTS.md:79:401 error MD013/line-length Line length [Expected: 400; Actual: 461]
AGENTS.md:118:15 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
AGENTS.md:118:35 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
AGENTS.md:118:130 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
AGENTS.md:119:16 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
AGENTS.md:119:43 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
AGENTS.md:119:163 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
AGENTS.md:144:11 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:144:21 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:144:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:144:11 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:167:13 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:167:34 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:167:56 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:167:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:167:13 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:167:34 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:266:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:266:16 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:266:30 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:266:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:266:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:266:16 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:275:11 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:275:25 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:275:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:275:11 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:295:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:295:22 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:295:40 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:295:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:295:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:295:22 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:353:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:353:17 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:353:31 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:353:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:353:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:353:17 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:419:15 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:419:27 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:419:41 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:419:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:419:15 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:419:27 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:491:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:491:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:491:33 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:491:42 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
AGENTS.md:491:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:491:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:491:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
AGENTS.md:491:33 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:34:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:34:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:34:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:34:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:34:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:34:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:45:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:45:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:45:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:45:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:45:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:45:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:54:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:54:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:54:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:54:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:54:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:54:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:65:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:65:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:65:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:65:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:65:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:65:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:80:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:80:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:80:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:80:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:80:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:80:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:91:19 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:91:26 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:91:78 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:92:20 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:92:27 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:92:81 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:97:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:97:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:97:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:97:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:97:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:97:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:106:22 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:106:31 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:106:62 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:107:20 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:107:29 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:107:84 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:112:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:112:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:112:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:112:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:112:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:112:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:121:13 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:121:20 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:121:83 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:127:10 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:127:17 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:127:117 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:133:13 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:133:20 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:133:76 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:138:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:138:18 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:138:32 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:138:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:138:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:138:18 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:147:20 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:147:27 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:147:88 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:153:12 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:153:19 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:153:122 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:208:401 error MD013/line-length Line length [Expected: 400; Actual: 434]
README.md:301:12 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:301:22 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:301:36 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:301:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:301:12 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:301:22 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:313:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:313:16 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:313:30 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:313:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:313:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:313:16 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:543 error MD024/no-duplicate-heading Multiple headings with the same content [Context: "Configuration"]
README.md:549:27 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:549:53 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:549:73 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:550:30 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:550:57 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:550:79 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:561:8 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:561:17 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:561:31 error MD060/table-column-style Table column style [Table pipe is missing space to the left for style "compact"]
README.md:561:1 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:561:8 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:561:17 error MD060/table-column-style Table column style [Table pipe is missing space to the right for style "compact"]
README.md:598 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]

See detailed reports in MegaLinter artifacts
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

fix: lint errcheck, lowercase error strings, bump gaze-crapload 43→45

6d1615d

jflowers merged commit 09fa99f into main Apr 22, 2026
3 checks passed

jflowers deleted the 015-curated-knowledge-stores branch April 22, 2026 01:05

jflowers mentioned this pull request Apr 23, 2026

feat: persist learnings as markdown files for backup and portability #50

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add curated knowledge stores with source-mapped extraction and file-backed persistence#51

feat: add curated knowledge stores with source-mapped extraction and file-backed persistence#51
jflowers merged 2 commits intomainfrom
015-curated-knowledge-stores

jflowers commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jflowers commented Apr 22, 2026

Summary

Eight Capabilities

Deliverables

Spec Artifacts

Uh oh!

github-actions Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅⚠️MegaLinter analysis: Success with warnings

Detailed Issues

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Apr 22, 2026 •

edited

Loading