Skip to content

fix: eliminate N+1 queries in ContestList scoreboard visibility check#550

Open
whlongg wants to merge 2 commits intoVNOI-Admin:masterfrom
whlongg:fix/n1-query-contest-scoreboard-list
Open

fix: eliminate N+1 queries in ContestList scoreboard visibility check#550
whlongg wants to merge 2 commits intoVNOI-Admin:masterfrom
whlongg:fix/n1-query-contest-scoreboard-list

Conversation

@whlongg
Copy link
Contributor

@whlongg whlongg commented Feb 26, 2026

Description

Type of change: bug fix

What

Eliminates N+1 queries in ContestList when checking scoreboard visibility for hidden-scoreboard contests (SCOREBOARD_HIDDEN / SCOREBOARD_AFTER_CONTEST).

  • author_ids / editor_ids: check _prefetched_objects_cache (via getattr) before falling back to through-table queries, so prefetched authors/curators data is reused.
  • can_see_full_scoreboard: use Python-side any() over prefetched data instead of .filter(...).exists() per contest.
  • ContestList._get_queryset: add view_contest_scoreboard to prefetch_related.
  • ContestList.get_context_data: also prefetch contest__view_contest_scoreboard for active participations.

Why

For each contest with a hidden scoreboard, Django was firing 3 extra queries per contest:

  1. Contest.authors.through.objects.filter(contest=self) — bypasses prefetch cache
  2. Contest.curators.through.objects.filter(contest=self) — same
  3. view_contest_scoreboard.filter(id=user.profile.id).exists() — not prefetched at all

All fallback paths are preserved for contexts without prefetch (admin, APIs, single-object views) using getattr(self, '_prefetched_objects_cache', {}).

Fixes #549

How Has This Been Tested?

Automated testjudge/tests/test_contest_list_perf.py:

  • Creates 10 contests with SCOREBOARD_HIDDEN, authenticated user.
  • Measures SQL query count via connection.queries (DEBUG=True).
  • Asserts count ≤ 35. Before fix: 46 queries (FAIL). After fix: 27 queries (PASS).

Direct method benchmarkcan_see_own_scoreboard() across multiple scenarios (5 reps each):

+----------------------------------------+--------------+-----------+-------------+----------+-------------+---------+
| Scenario                               | Before(time) | Before(q) | After(time) | After(q) | Delta       | Speedup |
+----------------------------------------+--------------+-----------+-------------+----------+-------------+---------+
| Visible scoreboard            (N=500)  |      0.9ms   |      0    |      0.9ms  |      0   |     -0.0ms  | 1.02x   |
| Visible scoreboard            (N=1000) |      1.8ms   |      0    |      1.9ms  |      0   | +     0.1ms | 0.95x   |
| Visible scoreboard            (N=2000) |      3.8ms   |      0    |     14.1ms  |      0   | +    10.3ms | 0.27x   |
+----------------------------------------+--------------+-----------+-------------+----------+-------------+---------+
| Hidden, no authors/scoreboard (N=500)  |   1123.4ms   |   2001    |    627.8ms  |    500   |   -495.7ms  | 1.79x   |
| Hidden, no authors/scoreboard (N=1000) |   2359.3ms   |   4000    |    859.7ms  |   1000   |  -1499.5ms  | 2.74x   |
+----------------------------------------+--------------+-----------+-------------+----------+-------------+---------+
| Hidden, with authors          (N=500)  |    995.1ms   |   2000    |    420.3ms  |    500   |   -574.8ms  | 2.37x   |
| Hidden, with authors          (N=1000) |   1976.6ms   |   4000    |    847.2ms  |   1000   |  -1129.4ms  | 2.33x   |
+----------------------------------------+--------------+-----------+-------------+----------+-------------+---------+
| Hidden, with view_scoreboard  (N=500)  |    984.4ms   |   2000    |    417.4ms  |    500   |   -567.0ms  | 2.36x   |
| Hidden, with view_scoreboard  (N=1000) |   1978.3ms   |   4000    |    839.2ms  |   1000   |  -1139.1ms  | 2.36x   |
+----------------------------------------+--------------+-----------+-------------+----------+-------------+---------+

Before fix: 4 queries/contest (2× through-table + exists + has_completed_contest).
After fix: 1 query/contest (only has_completed_contest remains, out of scope).

# Automated regression test
python manage.py test judge.tests.test_contest_list_perf
# [10 hidden-scoreboard contests] query count: 27 (budget: 35)
# Ran 1 test in 0.475s — OK
  • judge.tests.test_contest_list_perf — query count bounded (27 ≤ 35, before fix was 46)

Checklist

  • I have explained the purpose of this PR.
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the README/documentation
  • Any dependent changes have been merged and published in downstream modules
  • Informed of breaking changes, testing and migrations (if applicable).

By submitting this pull request, I confirm that my contribution is made under the terms of the AGPL-3.0 License.

- author_ids / editor_ids: use _prefetched_objects_cache when authors/curators
  are already prefetched, avoiding duplicate through-table queries per contest.
- can_see_full_scoreboard: check prefetch cache before calling
  view_contest_scoreboard.filter(...).exists() to avoid per-contest DB hit.
- ContestList._get_queryset: add view_contest_scoreboard to prefetch_related.
- ContestList.get_context_data: also prefetch contest__view_contest_scoreboard
  for active participations.
- Add judge/tests/test_contest_list_perf.py: regression guard asserting query
  count stays <= QUERY_BUDGET for 10 hidden-scoreboard contests.
@magnified103 magnified103 self-requested a review February 26, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Performance] Fix N+1 queries in ContestList scoreboard visibility check

2 participants