Added statistics to deeplake tables. #3110
Conversation
There was a problem hiding this comment.
Pull request overview
This PR integrates PostgreSQL's statistics collector with deeplake tables, enabling proper autovacuum support. The changes allow deeplake tables to report DML operation statistics (INSERT, UPDATE, DELETE) to PostgreSQL's pgstat system, which autovacuum uses to determine when to run ANALYZE.
Changes:
- Added statistics tracking calls (
pgstat_count_heap_*) to DML operations in the table access method - Implemented VACUUM support with explicit rejection of VACUUM FULL for columnar storage
- Enhanced ANALYZE support to properly handle columnar storage characteristics
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 12 comments.
| File | Description |
|---|---|
| postgres/tests/py_tests/test_statistics_integration.py | Comprehensive test suite validating statistics collection for INSERT/UPDATE/DELETE operations, ANALYZE behavior, and autovacuum integration |
| cpp/deeplake_pg/table_am.cpp | Core implementation adding pgstat integration, VACUUM support, and improved ANALYZE scanning for columnar storage |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
| except: | ||
| pass |
There was a problem hiding this comment.
Bare except clauses should specify the exception type. Use except Exception: to catch all standard exceptions while allowing system exits and keyboard interrupts to propagate.
|
🚀 🚀 Pull Request
Impact
Description
Things to be aware of
Things to worry about
Additional Context