Add quota UI: inline gates, usage awareness, and notifications

[harry-rhesis]

Purpose

Backend quota enforcement landed in #2505, but the UI barely reflected it: a banner at 80% and a preflight gate in one drawer, both phrased as though the reader personally owned a limit their organization owns. This makes quota legible everywhere it matters and fixes the framing.

One rule drives all of it: quota is organization state. Personal chrome never carries it, the org is always the grammatical subject ("Your organization is at its projects limit"), and "you" appears only for what the reader can actually do next.

What Changed

Backend

• The 402 body carries kind and period_end, so the client can classify a blocked resource and name its reset date without a second round trip. period_end is null for stock resources, which never reset.
• New usage_notifications service notifies the org owner when usage crosses 80% of a limit or its ceiling. It compares the before/after range rather than classifying the new value, so an org sitting past a threshold is not renotified on every subsequent call. Flow resources hook into increment_usage; stock resources hook into their three creation routes.
• QUOTA_RESOURCE_LABELS mirrors the frontend label map, so the backend stops calling test_executions "Test Executions" where the rest of the product says "Test Runs".

Frontend

• utils/quota.ts is the single source for classification and copy: classifyZone (healthy / approaching / pastIncluded / blocked), quotaCopy (the copy catalog as code), parseQuotaError, flaggedResources. Every surface reads from it instead of re-deriving thresholds or hand-writing strings.
• The usage page moves red from limit to ceiling and shows the overage allowance as a hatched second segment, so a soft-tier org can see the band it still has.
• A count badge on the brand row and an "Org usage" block in the org menu give ambient awareness. The badge answers "how many"; the banner and bars answer "how bad". Each org-menu usage row now fills a bar to how much of its limit is used, and the user-menu avatar badges its own unread-notification count.
• A notifications drawer, the drill-down the GET /notifications/ docstring already called for. Quota alerts land in a new usage section that badges no nav item. Mark-all-read now clears the list (rather than just stamping read_at) when filtered to "Unread only" -- otherwise rows the user just marked read stayed on screen contradicting their own filter -- and same-day rows now have visible spacing between them instead of sitting flush against each other.
• Inline gates on projects, endpoints, seats and test generation via two hooks, useQuotaGate and useQuotaErrorHandler. useQuotaGate takes an amount, because inviting five people consumes five seats and a per-1 gate waves through a submit the backend refuses partway. The shared <QuotaNotice> these gates render now matches the design record's Q4 mockup: a zone-coloured icon and left border, not just a bold sentence, plus separate "Org usage" (unconditional -- usage:read is granted to every member) and "Upgrade plan" (gated on canUpgrade) links rendered by the component instead of folded into the recourse string.
• Every stock-resource mutation (endpoints, projects, users/seats) now goes through a dedicated hook, useCreateEndpoint, useDeleteEndpoint, useCreateProject, useDeleteProject, useCreateUser, useDeleteUser, that invalidates the cached /usage response on success. Five call sites (both endpoint-creation forms, the duplicate-endpoint action, and onboarding's user and project creation) had bypassed the hook and called the API client or a server action directly, so a user who did what a quota notice told them to do could still see themselves as blocked for up to five minutes. An ESLint rule now blocks calling those methods outside src/hooks/, so a future call site can't reintroduce the same bug silently.
• The AI test-generation preview ("Generate Test Set"'s sample/config stream) had no quota awareness anywhere: the frontend only gated the final submit, the backend enforced nothing on the streaming endpoint, and the NDJSON reader had no timeout. An org already over its test_generation limit could open the stream, and if the underlying model call stalled (as it does when model_tokens is also exhausted), the UI hung on skeleton loaders forever with no notification. Fixed at three points: the frontend now preflight-checks test_generation before every preview-generation entry point, not just the final submit; the backend route now carries the same require_quota(TEST_GENERATION) dependency /test_sets/generate already has, and the one un-guarded LLM-resolution call in the pipeline now catches a quota exhaustion (including the existing model_tokens pre-call gate) and emits a proper error event instead of letting it escape a generator whose response had already committed; and the NDJSON stream reader gets the same 150s idle-timeout ExplorerClient already has, so any stall surfaces as a clear error rather than an infinite hang.
• The same "unguarded LLM resolution before the first yield" defect existed independently in Explorer's suggestion_pipeline_stream (live, wired to SuggestionsDialog) and evaluate_suggestions_stream (no frontend caller yet). Both now catch the failure and emit a new top-level {"type": "error", "message": ...} event -- a pipeline-level failure, distinct from the per-item error field output/evaluation events already carry for one row failing without stopping the rest. The quota-aware message-building this needs is a shared quota.enforcement.stream_error_message, not a third copy of the same isinstance(e, QuotaExceededError) check.

Review Fixes

A review pass over this branch's own commits turned up several gaps, all fixed:

• Endpoint creation showed "created successfully" on a 402. createEndpoint (a server action) never throws on a business failure -- it returns {success: false, error} -- and both EndpointForm and SwaggerEndpointForm ignored that, always showing the success toast and navigating away. An org at its endpoints limit saw a false success with nothing actually created.
• The test-generation preview's 402 showed raw JSON. generateTestPipelineStream threw a plain Error with the unparsed response body, so require_quota (added earlier in this PR) produced a toast reading the literal {"error":"quota_exceeded",...} payload. Now throws the same status/data-bearing shape BaseApiClient itself uses, and all five preview-generation catch blocks (two, handleRegenerateSample/handleLoadMoreSamples, had neither a preflight check nor real error handling) try asQuotaError() first, same as the final submit already did.
• The same "unguarded setup before the first yield" defect also lived in _fetch_db_context, one line after the LLM-resolution call this PR had already wrapped -- a deleted/inaccessible project reintroduced the exact hang this whole fix exists to prevent. Folded into the same try.
• Three more generation routes had no quota gate at all. /generate/tests, /generate/multiturn-tests and /generate/test_config back the same preview flow as the now-gated streaming route but had no require_quota dependency of their own.
• useCreateProject/useDeleteProject invalidated usage but not the project list cache, unlike their sibling useDeleteEndpoint -- a project picker elsewhere in the app kept a deleted project listed until staleTime elapsed.
• The sidebar avatar's unread badge was clipped by an overflow: hidden its sibling (the also-badged org-icon button) never had.
• Smaller: a no-op overflowX: 'visible' removed from FilterDrawerShell (the CSS Overflow spec promotes a lone visible axis to auto, so it never did anything); the GREYSCALE/EE-boundary ESLint selectors, copied into three rule blocks, hoisted into shared constants (180→269 lines, no behavior change); two duplicate 8-line "why this try/except exists" comments in the two streaming pipelines consolidated into one docstring both point at.

Additional Context

• Builds on feat: enforce usage quotas across execute, generate, and hosted-model paths #2505. Rebased onto main after Self-hosted deployments get no usage quotas #2524, which made USAGE_QUOTAS_ENABLED default to off for self-hosted. With quotas off every limit is null, so the badge, banner, gates and notifications all correctly disappear; the last commit fixes a header that was still rendering over zero rows in that case.
• A stale comment sent the original design down the wrong path. capabilities.ts claimed usage:read was "restricted to org admins". feat: enforce usage quotas across execute, generate, and hosted-model paths #2505 had already removed that restriction, deliberately, on the reasoning that quota enforcement blocks members so hiding the reason from them just turns a 402 into a support ticket. Several surfaces were built admin-only on that false premise and have been corrected: everyone sees the numbers, and Organization.UPDATE is the gate for "can act on billing", which is what the upgrade CTA now uses. The comment is fixed too.
• Three defects worth calling out, all found reviewing this work rather than in production. An unguarded block after increment_usage's commit meant a raised exception made Celery retry an accrual that had already landed, inflating the counter and blocking the org at a fraction of its real limit. auto_stamp was filling project_id on org-wide notifications from the acting admin's active project, so RLS then hid them in every other project. And findWorstResource sorted by ratio alone, which let a soft-tier resource deep in its grace band outrank a hard-blocked one and hide a real block behind a warning.
• Deferred: per-user attribution ("who used the last slot"). usage_attribution.py exists on the backend and the usage page is its eventual home.

Testing

Automated:

• Frontend: 215 suites, 2036 tests. New coverage for utils/quota.ts (43, including every zone boundary, every copy-catalog row, and the usage-bar fill-percent helper), the quota hooks (12), <QuotaNotice> (4), the notifications drawer (18, including mark-all-read-while-filtered and same-day row spacing), the sidebar's usage bar and unread badge, ServicesClient.generateTestPipelineStream's NDJSON parsing, stall timeout and 402 handling (3), and EndpointForm's create-success/create-failure paths (2).
• Backend: the quota, usage and notification suites pass (87 tests across test_quota_gates.py, test_quota_enforcement.py, test_test_generation_pipeline.py, test_services.py, test_suggestions.py), including a regression test for the project_id stamping that fails without its fix, and new coverage for the test-generation and Explorer suggestion pipelines' 402 gates and LLM-resolution error paths (including a QuotaExceededError surfacing the organization-subject copy, not the exception's technical string, in each). Caveat: the full backend suite has not been re-run since the rebase onto Self-hosted deployments get no usage quotas #2524 — Docker on this machine wedged under disk pressure. Since Self-hosted deployments get no usage quotas #2524 touches the same quota module, please let CI confirm, or re-run the suites named above locally.
• npx tsc --noEmit clean; npm run lint reports 0 errors (61 pre-existing warnings, none from this PR's own files); ruff check/format clean on every touched Python file.

Manual, on a community-edition org:

1. Set a small limit and consume most of it. The brand-row badge appears with a count, the org menu shows "Org usage" with per-resource rows, and the banner names the organization rather than you.
2. Cross the ceiling. Submit buttons disable with an explanation in the drawer footer, never on the FAB that opens it. The owner gets a notification, visible from any project.
3. Delete the blocking project, then via the Swagger import form and the manual form, create an endpoint. Both the project-deletion and endpoint-creation counts update immediately rather than after the cache expires.
4. Open the notifications drawer's filter panel. The "Unread only" switch renders flush against the left edge, not clipped.
5. As a non-admin member, confirm the numbers are visible but no upgrade link is offered.
6. With USAGE_QUOTAS_ENABLED off, confirm the badge, banner, gates and org-menu block are all absent.
7. With test_generation at its limit, open "Generate Test Set" and try to continue past the input screen, regenerate samples, or use "Further refine". Each shows the blocking toast immediately rather than opening the stream.
8. Trigger a 402 on a gated drawer as a member and as an admin. Both show the icon, the bordered notice, and an "Org usage" link; only the admin also sees "Upgrade plan".

[peqy]

[Improvement] NotificationsDrawer “Unread only” doesn’t stay true after marking rows read

If unreadOnly is on, both handleRowClick and handleMarkAllRead only set read_at locally, so read rows remain visible even though the filter says “Unread only”.

Fix: when unreadOnly is true, remove the row(s) from items (e.g. filter them out) or trigger a refetch (fetchPage(0, true) + reset paging state) after marking read, so the UI matches the filter semantics.

Overall the quota single-source module + widened 402 body look solid.

Found 1 issue (0 critical, 1 improvement).

[peqy]

[Improvement] NotificationsDrawer “Unread only” doesn’t match behavior after “Mark all read”: rows stay visible (just tinted as read) even though the filter implies the list should become empty.

[Question] Backend 402 quota body message still uses “You've …”; given the MR’s org-subject copy rule, should that become “Your organization …” as a safety net for any surface that shows raw API error messages?

[peqy]

[Improvement] When unreadOnly is enabled, handleMarkAllRead marks read_at locally but keeps the rows in items, so the UI still shows notifications even though the filter says “Unread only”.

Fix: in the unreadOnly case, clear the list and reset paging (e.g. setItems([]), setPagesLoaded(0), setHasMore(false)), or refetch page 0 and reset paging state.

[peqy]

Rechecked on the latest commits: if (unreadOnly) setHasMore(false) helps pagination, but the list still renders rows that were just marked read. If unreadOnly is true, I think handleMarkAllRead should also clear/filter items (and ideally reset pagesLoaded) so the UI immediately flips to the “You’re all caught up” empty state.

[harry-rhesis]

Fixed in 658dfbc: handleMarkAllRead now clears items and resets pagesLoaded/hasMore when unreadOnly is true, instead of leaving now-read rows rendered. Added a regression test (empties the list immediately when marking all read while filtered to unread).

[peqy]

Thanks — that fix matches what I was aiming for (clear the list + reset paging under unreadOnly), and the regression test is a nice guard. 👍

[peqy]

[Question] quota_exceeded_response_body() still returns a message with “You've reached your …”. Since this MR’s copy rule is “quota is organization state”, should this be “Your organization …” as well (in case any client surfaces data.message directly)?

[peqy]

Follow-up: the frontend can surface data.message from 402s (base-client falls back to errorData.message, and parseQuotaError also carries it), so it’s worth aligning this string with the “quota is organization state” copy too (e.g. “Your organization is at its … limit …”).

[harry-rhesis]

Fixed in 23b4525: reworded to "Your organization is at its {resource} limit...", matching quotaCopy()'s phrasing on the frontend. The one test asserting on this string checks for "test runs" in body["message"], which still holds.

[peqy]

Thanks — that addresses my concern. Aligning the 402 message with the org-subject copy (and keeping the existing test assertion) sounds good.

[peqy]

Rebased on the latest commits — the two earlier points still look outstanding:

[Improvement] NotificationsDrawer: when “Unread only” is enabled, handleMarkAllRead marks read_at locally but keeps rows in items, so the list still shows notifications despite the filter.

Fix: in the unreadOnly case, clear/filter items (and reset paging state) or refetch page 0 after marking read.

[Improvement] Backend 402 body: quota_exceeded_response_body() still sets message to “You've reached your …”. The frontend can surface data.message, so this should probably follow the MR’s “quota is organization state” rule.

Fix: change to “Your organization …” (and keep the period suffix logic).

[peqy]

Rechecked latest commits — earlier feedback still applies:

[Improvement] NotificationsDrawer: with “Unread only” enabled, handleMarkAllRead marks read_at locally but keeps rows in items, so the list still shows notifications despite the filter.

Fix: in the unreadOnly case, clear/filter items (and reset paging state) or refetch page 0.

[Improvement] Backend 402 body: quota_exceeded_response_body() still returns message as “You've reached your …”. The frontend can surface data.message, so this should follow the “quota is organization state” copy rule.

Fix: change to “Your organization …” (keeping the period suffix logic).

Found 2 issues (0 critical, 2 improvements).

[peqy]

Rechecked the latest commits: both previously-raised items look addressed.

• NotificationsDrawer: under “Unread only”, handleMarkAllRead now clears items / resets paging so the list immediately goes empty.
• Backend 402 body: quota_exceeded_response_body() now uses org-subject copy and includes kind/period_end.

Looks good. Ship it.

[peqy]

[Question] Template auto-init + quota gating may still fail open while usage is loading (see inline comment).

Otherwise, the latest follow-ups (unread-only mark-all-read + org-subject 402 message + preview gating) look solid.

Found 1 issue (0 critical, 0 improvements, 1 question).

[peqy]

[Question] Potential race with usage loading

useQuotaGate() returns message: undefined while usage is still loading (fails open). In the selectedTemplateId auto-init path, that means a quota-blocked org could still start generateTestPipelineStream() before usage arrives, and then never hit this guard.

Fix (if this is possible in practice): consider waiting for usage to be loaded before auto-starting the template stream (or add an explicit “usage loading” signal to the gate and treat that as “don’t auto-run yet”).

[peqy]

Rechecked after the latest commits: /services/generate/test_pipeline is now quota-gated server-side (require_quota(TEST_GENERATION)), and ServicesClient.generateTestPipelineStream() now throws an error with status + parsed data, so asQuotaError(...)?.message surfaces the org-subject quota copy cleanly.

Given that, the “usage still loading → fails open” window in useQuotaGate() shouldn’t cause a silent bypass anymore (worst case you get a handled 402 + toast), so I’m comfortable with this now.

[peqy]

Rechecked the latest commits (preview-pipeline quota gate + stream timeout + QuotaNotice update).

[Improvement] Quota 402s from /services/generate/test_pipeline won’t surface the quota copy cleanly

Backend now correctly blocks the preview pipeline with require_quota(TEST_GENERATION), but ServicesClient.generateTestPipelineStream() still throws a plain Error with a stringified body on non-2xx. That means the frontend can’t parse quota errors via parseQuotaError/useQuotaErrorHandler, so users may see raw JSON / generic “Failed to load template” rather than the org-subject quota message.

Fix: on !response.ok, parse the JSON body and throw an error that preserves status + data in the same shape as BaseApiClient.fetch (and set message from data.message), then use asQuotaError(e)?.message ?? … in the preview/template path too.

Everything else in these commits (quota gate on preview, NDJSON idle timeout + tests, quota-aware error events in the backend stream) looks solid.

[peqy]

Rechecked the latest commits (explorer suggestion-stream failure surfacing + preview-pipeline quota gate + NDJSON stall timeout) — overall still looks solid.

[Improvement] ServicesClient.generateTestPipelineStream() still throws a plain Error on !response.ok (stringified body). That means 402 quota responses from /services/generate/test_pipeline can’t be parsed by parseQuotaError / useQuotaErrorHandler, so the UI may show raw JSON / generic errors instead of the org-subject quota copy.

Fix: when !response.ok, parse the JSON body (when present) and throw an Error that preserves status + data in the same shape as BaseApiClient.fetch (and set message from data.message). Then callers can do asQuotaError(e)?.message ?? getApiErrorMessage(e, ...) and quota UX stays consistent.

[peqy]

Rechecked after the latest commits: ServicesClient.generateTestPipelineStream() now preserves status + parsed data on !response.ok, so 402 quota responses from /services/generate/test_pipeline can be parsed by parseQuotaError / useQuotaErrorHandler and show the org-subject quota copy.

Given that (and the server-side require_quota(TEST_GENERATION) gate now in place), the remaining quota/streaming surfaces look solid. Ship it.

[peqy]

Rechecked the latest update: the new commit is just a CI retrigger (no code diff). The quota-gated streaming + error surfacing fixes still look solid. Ship it.