-
-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Search: optimize queryset when searching for subprojects with organizations #12635
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR optimizes database queries when searching projects with organizations by implementing strategic caching and prefetching. The key optimization is that since all subprojects share the same organization as their parent project, the organization can be cached and reused across all related projects.
Key Changes
- Converted
Project.organizationfrom@propertyto@cached_propertyto cache organization lookups - Added
prefetch_organization()queryset method for efficient bulk loading of organizations - Implemented manual organization caching in search executor for subprojects via
_organizationsattribute - Replaced all
project.organizations.first()calls withproject.organizationthroughout the codebase - Added resolver caching for organization-level feature checks (custom domains)
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| readthedocs/projects/models.py | Converted organization property to cached_property with RTD_ALLOW_ORGANIZATIONS check |
| readthedocs/projects/querysets.py | Added prefetch_organization() method for bulk loading with select_related support |
| readthedocs/search/api/v3/executor.py | Manually cache organization for subprojects by setting _organizations attribute; added prefetch_organization to project queries |
| readthedocs/search/api/v2/serializers.py | Reuse single Resolver instance across projects to leverage organization caching |
| readthedocs/core/resolver.py | Added organization-level caching for custom domain checks with @cache decorator |
| readthedocs/telemetry/models.py | Updated to use project.organization property |
| readthedocs/core/permissions.py | Updated to use project.organization property |
| readthedocs/builds/querysets.py | Updated to use project.organization property |
| readthedocs/organizations/tasks.py | Updated to use project.organization property |
| readthedocs/audit/models.py | Updated to use project.organization property |
| readthedocs/api/v3/tests/test_subprojects.py | Removed erroneous organization.projects.add() line |
| readthedocs/search/api/v3/tests/test_api.py | Added comprehensive tests validating query counts with organizations |
| readthedocs/rtd_tests/tests/test_project_querysets.py | Added @override_settings decorator and cache clearing for organization tests |
| readthedocs/builds/tests/test_build_queryset.py | Added @override_settings decorator for organization tests |
| readthedocs/invitations/tests/test_views.py | Added @override_settings decorator for organization tests |
| readthedocs/audit/tests/test_tasks.py | Added @override_settings decorator for organization tests |
| readthedocs/audit/tests/test_models.py | Added @override_settings decorator for organization tests |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
On .com having organizations introduces some extra queries. Since all subprojects share the same organization as the parent project, we can cache that organization, so it's shared across all subprojects.
This also allows caching the .organization property, and skips querying organizations if we are on .org.
Results: