Shattered - #6
Merged
Merged
Conversation
Group 1 Feature Parity Implementation: 1. Regex Search (Search Shard) - New RegexSearchEngine with PostgreSQL native regex (~, ~*) for scalability - Pattern presets for common patterns (email, phone, SSN, credit card, etc.) - Pattern validation with performance estimation (fast/moderate/slow/dangerous) - UI panel with detected patterns integration from Patterns shard - Fixed sentence chunking to preserve emails, URLs, and abbreviations 2. Document Deduplication (Documents Shard) - New DeduplicationService with exact hash and fuzzy similarity matching - SimHash-based near-duplicate detection with configurable thresholds - API endpoints for finding duplicates, grouping, and merging - UI panel in DocumentViewer for duplicate management 3. Graph Pathfinding (Graph Shard) - New PathfindingEngine with BFS, DFS, Dijkstra, and A* algorithms - Shortest path and all paths discovery - Path analysis with relationship type filtering - API endpoints for path queries UI Improvements: - RegexSearchPanel with dark theme CSS styling - DeduplicationPanel with match highlighting - Detected patterns section showing patterns from Patterns shard Bug Fix: - Fixed sentence chunking regex that was splitting on periods in email addresses (e.g., "agent.smith@cia.gov" was becoming "agent smith@cia gov") Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix frontend MergeRequest to match backend API schema: - source_ids -> duplicate_ids - target_id -> primary_id - strategy -> cleanup_action - Add MERGED status to DocumentStatus enums in both frame and documents shard - Update DeduplicationPanel dropdown options to match backend cleanup_action values Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Anomalies Shard - Hidden Content Detection: - Add steganography detection with entropy analysis (Shannon entropy) - Implement LSB (Least Significant Bit) pattern analysis with chi-square tests - Add file type mismatch detection via magic bytes comparison - Create HiddenContentTab UI component with scan controls and results display - Add entropy region visualization with bar charts - Display stego indicators with confidence scores Provenance Shard - Metadata Forensics: - Add EXIF metadata extraction for images - Implement PDF metadata extraction (author, creator, dates, version) - Add Office document metadata extraction (docx/xlsx/pptx) - Create timeline reconstruction from document metadata - Add document comparison feature with similarity scoring - Create ForensicsTab UI component with comprehensive metadata display - Add file hash computation (MD5, SHA256, SHA512) - Display integrity status indicators UI Components: - Tab navigation for Anomalies (list view / hidden content detection) - Tab navigation for Provenance (artifacts / chains / lineage / forensics) - Stats grids, scan controls, and detailed results displays - Responsive CSS with proper theming Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Credibility Shard - Deception Analysis Improvements: - Increase default credibility_weight from 0.3 to 0.7 for meaningful impact - Add severity override system: critical deception (>=80%) caps credibility at 15, high (>=60%) caps at 30, moderate (>=40%) caps at 50 - Auto-link deception assessments to existing credibility assessments - Fix DeceptionPanel to pass linked_assessment_id when creating assessments - Fix DeceptionPanel to reset state when switching between documents Embed Shard - Statistics Fix: - Replace in-memory cache stats with actual database metrics - Show total embeddings, pending chunks, coverage rate, and document counts - Stats now reflect real embedding coverage instead of always showing zeros - Update UI labels: "Embeddings", "Pending", "Coverage", "Docs / Chunks" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bringing main up to date with current Shattered development