Skip to content

Extend infinite scroll to portal list surfaces (Closes #972)#974

Merged
cjimti merged 1 commit into
mainfrom
fix/972-extend-portal-pagination
Jul 16, 2026
Merged

Extend infinite scroll to portal list surfaces (Closes #972)#974
cjimti merged 1 commit into
mainfrom
fix/972-extend-portal-pagination

Conversation

@cjimti

@cjimti cjimti commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

Closes #972. The Assets library loads all items via infinite scroll, but the other portal list surfaces fetched only their first page, so a user or deployment with more items than one page silently could not reach the rest. This PR extends the same pattern to every affected surface that has a stable, offset-capable backend, so each list can display its full contents.

Surfaces now paginated

Each of these accumulates pages as you scroll, with a "Load more" button as the fallback and indicator, matching the Assets library:

  • Collections (My Collections and Shared)
  • Feedback: Recent (activity feed), Worklist (practitioner and SME), and General (standalone channel)
  • Resources (My Resources, Global, and per-persona/admin tabs)
  • Knowledge Pages
  • Settings > Users
  • Settings > Changelog

Shared infrastructure

Paging is implemented once and reused, rather than reimplemented per page:

  • ui/src/api/portal/hooks/infinite.ts holds useOffsetInfiniteQuery plus the pure helpers nextOffset, flattenPages, toPaginated, and paginatedFetch. The Assets hooks now consume this shared module too (the paging helpers moved out of assets.ts, which re-exports them for backward compatibility).
  • ui/src/components/InfiniteFooter.tsx is the shared Load-more control: it owns the scroll sentinel and wires useInfiniteScroll, and renders nothing once every page is loaded.
  • Rows are merged in fetch order and de-duplicated by a stable key, so offset paging over a created_at/updated_at DESC list cannot double-count a row when an insert shifts the window. Paging stops on the latest page's total or an empty trailing page, so "Load more" cannot spin forever.
  • Endpoints that return a non-PaginatedResponse envelope (Resources {resources,total}, Users {users,total}, Knowledge Pages {pages,total}, Changelog {entries,total}) adapt their response to the shared shape at the fetch boundary via toPaginated, so all surfaces share one merge, dedupe, and stop-condition path.

Backend

  • Resources: handleList now honors a client-supplied limit, clamped to a new resource.MaxListLimit. Previously it ignored the client limit and always used the default page size.
  • Config changelog: configstore.Store.Changelog gained offset and a full-history total (Postgres OFFSET plus a COUNT(*), the file store, and the admin handler), returned as a {entries,total} envelope and clamped to maxChangelogLimit. Older entries beyond the most-recent window are now reachable.
  • Swagger annotations and generated internal/apidocs are updated for the new limit parameter and the changelog response shape.

DataHub Catalog and Context Docs (not wired)

Catalog and Context Docs browse are intentionally left single-page. They page against an external DataHub via a query:"*" search that sends no sort criteria, so results come back in relevance/segment order that is not stable across requests: numeric offset paging over it can drop or duplicate rows between fetches, and catalog browse returns no total. Infinite scroll here would be a correctness regression worse than the current cap, so it stays gated on the upstream mcp-datahub client gaining a stable sort or a scroll/search-after cursor. The constraint is documented in ui/src/api/portal/datahub.ts. Search remains the escape hatch.

Tests and verification

  • New shared hooks and the InfiniteFooter control have unit tests; the feedback component tests are updated to the infinite hooks.
  • dev/seed.sql seeds more than one page of resources, collections, threads, knowledge pages, users, and changelog entries so every surface exercises pagination in the dev environment. The seed is validated against a real Postgres by make migrate-check.
  • MSW mocks for the changelog and resources endpoints window by limit/offset so the e2e and screenshot pipelines exercise paging.
  • Full make verify passes locally (unit tests with race, coverage and patch coverage, golangci-lint, gosec, govulncheck, semgrep, CodeQL, frontend tests and e2e, swagger-check, migrate-check).

Notes

  • The config changelog request is clamped to a maximum page size, matching the clamp every other paginated store applies, so a client cannot request an unbounded window.
  • The "Showing X of Y" caption on paginated lists is shown only while more pages can be loaded, so a de-duplicated count never leaves a permanent off-by-one with no control to resolve it.
  • Knowledge Pages request 100 per page, the store's largest honored search limit; the previous fixed limit: 200 request was silently collapsed to 20 by the store, so the tag facet is now more complete on first load, not less.

Follow-up (out of scope)

Real pagination for the currently unbounded "fetch-all" endpoints (My Prompts, Admin Prompts, Prompt Review Queue, Tools, and the Settings lists) remains a lower-priority follow-up, and DataHub-backed browse is blocked on the upstream paging guarantee described above.

Portal list surfaces beyond Assets silently capped at their first page.
This adds offset-based infinite scroll (Load-more fallback) to Collections,
Feedback (Recent/Worklist/General), Resources, Knowledge Pages, Settings >
Users, and Settings > Changelog, so each can display its full contents.

Shared infrastructure so no surface reimplements paging:
- useOffsetInfiniteQuery + nextOffset/flattenPages/toPaginated/paginatedFetch
  in ui/src/api/portal/hooks/infinite.ts (Assets now consumes it too)
- InfiniteFooter control (scroll sentinel + useInfiniteScroll + Load more)
Endpoints returning a non-PaginatedResponse envelope adapt at the fetch
boundary so all surfaces share one merge/dedupe path.

Backend:
- resources: handleList honors a client limit, clamped to MaxListLimit
- config changelog: Changelog gains offset+total (postgres OFFSET/COUNT,
  file store, admin envelope {entries,total}) clamped to maxChangelogLimit,
  so older entries are reachable
- swagger regenerated

DataHub Catalog/Context Docs browse is intentionally left single-page: the
upstream "*" browse sends no sort criteria, so offset paging can drop or
duplicate rows and catalog browse returns no total. The constraint is
documented in datahub.ts pending a stable upstream sort/cursor.

dev/seed.sql seeds >page-size volume per surface (validated via migrate-check).
MSW mocks and unit tests updated; new shared hooks/component covered by tests.
@cjimti
cjimti merged commit 75cee5d into main Jul 16, 2026
10 checks passed
@cjimti
cjimti deleted the fix/972-extend-portal-pagination branch July 16, 2026 04:23
@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.10345% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.06%. Comparing base (01d36b1) to head (97af12c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pkg/configstore/postgres/store.go 83.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #974      +/-   ##
==========================================
+ Coverage   89.03%   89.06%   +0.02%     
==========================================
  Files         424      424              
  Lines       48581    48598      +17     
==========================================
+ Hits        43256    43283      +27     
+ Misses       3549     3543       -6     
+ Partials     1776     1772       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Portal list pages silently cap at the first page — extend pagination beyond Assets

1 participant