feat: optimize PR metadata cache with smart date range queries #146
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add intelligent cache optimization to reduce redundant GitHub API calls by tracking and using cached PR date ranges for smarter fetch strategies. ## Changes **New SQLiteCache Method:** - Added `get_pr_metadata_date_range()` to query min/max dates from cached PRs - Returns date range and count, filtering out expired entries - Enables quick assessment of cache coverage per repository **CacheManager Enhancement:** - Added `get_cached_date_range()` wrapper method - Integrates seamlessly with existing cache infrastructure **Optimized Fetch Logic (get_prs.py):** - Retrieve cached date range before making API calls - Implement smart gap detection: - Backward fetch: Only if requested start < cached minimum - Forward fetch: Only if needed and requested end > cached maximum - Skip redundant fetches when cache already covers requested range - Print debug messages showing backward/forward fetch operations **Comprehensive Testing:** - Created 5 new tests in `tests/cache/test_pr_metadata_date_range.py` - Test coverage includes: - Empty cache handling - Single and multiple PR date ranges - Repository isolation - Expired entry filtering - All 65 tests passing (60 existing + 5 new) ## Benefits - **Reduced API calls**: Only fetches missing data gaps instead of full refetches - **Better cache utilization**: Leverages pr_metadata table for quick queries - **Improved performance**: Especially beneficial for repos with extensive PR history - **Debug visibility**: Clear logging when gap fetches occur ## Performance Impact Integration tests show efficient cache utilization with minimal redundant API calls on subsequent runs for the same date ranges. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fix PermissionError on Windows when cleaning up temporary test directories by explicitly closing the SQLite database connection before fixture teardown. ## Problem On Windows, the test teardown was failing with: ``` PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: api_cache.db ``` This occurred because SQLite keeps file locks on Windows, preventing the temporary directory cleanup from deleting the database file. ## Solution Modified the cache_manager fixture to: - Yield the SQLiteCache instance instead of returning it - Explicitly call cache.close() during teardown - Properly release the database connection before temp directory cleanup ## Testing - ✅ All 65 tests passing on Linux - ✅ Verified fixture properly closes connection - ✅ No change to test behavior or assertions - ✅ Fix addresses Windows-specific file locking issue 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Replace pr_index_cache with direct queries on pr_metadata_cache to simplify caching architecture and reduce code complexity. Changes: - Add get_pr_summaries() and get_pr_metadata_stats() to sqlite_cache.py - Update cache_manager.py to use pr_metadata queries instead of pr_index - Simplify get_prs.py by removing pr_index management logic - Remove pr_index table creation and all related methods - Fix timezone import in cache_manager.py All tests pass and integration test verified with expressjs organization. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
remove get_pr_summary_list and call the cache.get_pr_summaries directly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implements intelligent cache optimization to reduce redundant GitHub API calls by tracking and using cached PR date ranges for smarter fetch strategies.
Key Features
1. New Cache Query Method
get_pr_metadata_date_range()to SQLiteCache2. Smart Gap Detection
3. Optimized Fetch Logic
The
get_pull_requests_between_dates()function now:Changes
Modified Files:
reviewtally/cache/sqlite_cache.py: Addedget_pr_metadata_date_range()methodreviewtally/cache/cache_manager.py: Addedget_cached_date_range()wrapperreviewtally/queries/get_prs.py: Optimized to use cached date rangesNew Tests:
tests/cache/test_pr_metadata_date_range.py(5 new tests)Benefits
Test Plan
Performance Impact
Integration tests show efficient cache utilization with minimal redundant API calls on subsequent runs for the same date ranges. The system now intelligently fills only the gaps in cached data.
🤖 Generated with Claude Code