Add comprehensive documentation suite with architecture diagrams and developer reference guide#21
Open
dmatrix wants to merge 4 commits intodatabricks-solutions:mainfrom
Conversation
…developer guides This commit introduces a complete documentation system for Casper's Kitchens: Documentation Structure: - docs/README.md - Documentation hub with navigation and overview - docs/technical-reference.md - Detailed schemas, APIs, and configurations - docs/developer-onboarding.md - Step-by-step guide for new developers - docs/dataflow-diagram.md - Visual architecture documentation - docs/mermaid-image-generation.md - Guide for diagram generation Architecture Visualization: - docs/dataflow-diagram.mermaid - Source Mermaid diagram definition - docs/images/dataflow-diagram.png - Standard resolution diagram - docs/images/dataflow-diagram-hd.png - High-resolution diagram (2400x1800) - docs/images/dataflow-diagram.svg - Scalable vector format - docs/generate-diagrams.sh - Automation script for diagram generation Main README Enhancements: - Added technology badges (Python, PySpark, Databricks, etc.) - Added prominent link to comprehensive documentation - Improved visual structure with badges and separators Technical Reference Content: - Complete table schemas for Bronze/Silver/Gold layers - Event type definitions and lifecycle tracking - API endpoint specifications with request/response examples - Configuration parameters and performance optimization details - Streaming watermark and partitioning strategies Developer Onboarding Content: - Architecture overview and key concepts walkthrough - Essential files explanation with code examples - Common development tasks and patterns - SQL queries for monitoring and debugging - Troubleshooting guide for common issues Dataflow Diagram Features: - Complete visual representation of the data architecture - Shows event sources through to applications layer - Includes medallion architecture (Bronze → Silver → Gold) - Visualizes streaming intelligence and ML components - Documents Lakebase integration and application layer - Available in multiple formats for different use cases Benefits: - Reduces onboarding time for new developers - Provides quick reference for technical specifications - Visual documentation for presentations and design discussions - Comprehensive SQL examples for operations and monitoring - Clear architectural patterns and data flow understanding
Author
|
@nkarpovdb added a data flow diagram and added documentation. Prompted and guided the cursor at appropriate .py and .pynp files to create some docs. See if that makes sense to add it. |
Signed-off-by: Jules Damji <dmatrix@comcast.net>
Signed-off-by: Jules Damji <dmatrix@comcast.net>
Signed-off-by: Jules Damji <dmatrix@comcast.net>
nkarpov
added a commit
that referenced
this pull request
Mar 2, 2026
99081a0 Merge pull request #21 from databricksdevrel/fix/user-pinned-pi-and-type-selection 363ca9d fix(terminal-types): support user-token bootstrap and explicit type selection git-subtree-dir: apps/caspersai-terminal git-subtree-split: 99081a0c1cf4e44411de4e6a7028489dce96cbd3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit introduces a complete documentation system for Casper's Kitchens:
Documentation Structure:
Architecture Visualization:
Main README Enhancements:
Technical Reference Content:
Developer Onboarding Content:
Dataflow Diagram Features:
Benefits: