Skip to content

refactor(drizzle): projected select in createDrizzleHandler (deletes both hand-written row remaps) - #91

Merged
mxkaske merged 2 commits into
mainfrom
refactor/drizzle-projected-select
Aug 14, 2026
Merged

refactor(drizzle): projected select in createDrizzleHandler (deletes both hand-written row remaps)#91
mxkaske merged 2 commits into
mainfrom
refactor/drizzle-projected-select

Conversation

@mxkaske

@mxkaske mxkaske commented Aug 11, 2026

Copy link
Copy Markdown
Member

Implements RFC #88. Stacked on #90 (RFC #87), which is stacked on #89 (RFC #86). This PR targets feat/filter-semantics.

Problem

createDrizzleHandler fetched rows with db.select(). That returns rows keyed by Drizzle property names (timingDns), but the wire contract and the whole schema layer are keyed by schema keys ("timing.dns"). So every caller translated, by hand, on the way out.

The remap was written twice, verbatim — and the two copies had already drifted: the MCP route emitted date.toISOString() where REST emitted a Date, and omitted headers and percentile. Its own TODO already named the fix:

// TODO: extract response row mapping — columnMapping already defines the camelCase↔dot-notation relationship, could be reused here

Fix

The handler builds its select from the mapping it already has, so rows come back keyed by schema keys and both remaps are deleted. Columns that belong in the payload but are never filtered or sorted go in a new select option.

allConditions: SQL[]scope: DrizzleQueryScope:

{ db, table, columns, where, whereWithoutSliders, range, bucketMs }

That leak is what made ~110 lines of untested db: any SQL necessary in the first place — the three things such a caller needs (resolved range, bucket interval, composed WHERE) were all computed inside the handler and thrown away. getChartData keeps its aggregate SQL and stops re-deriving anything.

Net −78 lines across the two routes.

Latent bug fixed

getCursorValue read row[cursorCol.name] — the SQL column name, not the JS key. They coincide for date. For any cursor column whose DB name differs (timing_dns), the lookup was undefinedgetCursorValue returned null → boundaryValue was null → the entire tie-snapping block was skipped → nextCursor came back null, silently ending pagination after one page.

No test used a cursor column whose SQL name differed from its JS key. The regression test now builds one deliberately, and was mutation-verified: reverting that single line fails 7 tests.

Unmapped keys stop vanishing

buildWhereConditions and buildOrderBy both silently skipped a key missing from columnMapping, and ColumnMapping is Record<string, Column> so a typo wasn't a type error either — the filter just stopped filtering, with no error, no warning, and no failing test. The handler now throws at construction, naming the missing keys with the snippet to paste.

Also fixed, found while testing

  • interval.ts shipped in the block's code but was missing from registry.json. npx shadcn add data-table-drizzle would have installed files importing a module it never wrote. registry.test.ts only resolved alias imports; it now resolves relative ones too — the check that would have caught it. Verified to fail when the entry is removed.
  • The deleted remap was papering over a NULL. It normalized message with ?? undefined; projecting the column directly returns SQL NULL, and the sheet condition tested !== undefined — so rows with no message would have started rendering an empty field. Condition and Zod schema now both accept null.

Tests

All ungated on PGLite (available since #90).

  • Projection identity — every key in columnMapping is present on a fetched row with the right value. This is the test that makes deleting the remaps safe.
  • column-mapping.ts is finally imported by a test. The suite had been substituting its own divergent copy that added a message entry production doesn't have — a mapping no test exercises is exactly where an unmapped key hides.
  • REST and MCP agree, modulo the two deliberate differences (Date vs ISO string; percentile is REST-only). Both real route modules are invoked end-to-end. This pins drift that had already happened, and was mutation-verified by making MCP drop headers again.
  • The interval ladder gets its first test — every rung boundary, the fall-through, negatives, zero, non-finite.

2245 tests pass (1825 registry, 420 web). Typecheck, lint, prettier, registry:build clean.

Note

The RFC suggested adding the replacement as a new entry point and deprecating the old one. I changed createDrizzleHandler in place: result.data's key shape is breaking, but registry blocks are copy-in so no installed project breaks silently, and the in-repo routes and docs land in the same change. Two parallel entry points would have meant maintaining the star-select path indefinitely.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
data-table-filters Ready Ready Preview Aug 14, 2026 7:05am

Request Review

@mxkaske

mxkaske commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

@claude review

@claude

claude Bot commented Aug 12, 2026

Copy link
Copy Markdown

Claude finished @mxkaske's task in 3m 15s —— View job


Review

  • Gather context (PR description, changed files list)
  • Read diff against origin/feat/filter-semantics
  • Review handler.ts (projected select, scope, cursor fix, unmapped-key throw)
  • Review route changes (route.ts, mcp/route.ts)
  • Review tests (handler-projection, interval, payload-parity)
  • Review registry/docs changes
  • Post review feedback

Solid refactor — the diff matches the description closely and the regression tests are convincingly designed (the handler-projection.test.ts table deliberately uses SQL names that diverge from schema keys, which is exactly what's needed to catch this bug class). A few notes, nothing blocking:

1. discoverRange assumes cursorColumn is the "time" column (handler.ts:63-83, used at handler.ts:286).
scope.range is computed as MIN/MAX(cursorCol) when no explicit date filter is set. That's correct here since cursorColumn: "date" in both routes, but nothing in the types or docs constrains cursorColumn to be a timestamp column — getCursorValue explicitly supports numeric cursors too. A future table with e.g. cursorColumn: "id" would get a nonsense scope.range/bucketMs (an id coerced via new Date(id)) with no error, silently. Given DrizzleQueryScope.range's docstring says "resolved from an explicit date filter, else MIN/MAX of the set" without naming which column, it might be worth either restricting discoverRange to only run when dateKeys.length > 0 (deriving the range from the mapped date-filter column rather than the cursor column), or documenting the "cursor column must be your date column for scope.range to be meaningful" constraint explicitly. Low severity since it doesn't affect the current app.

2. discoverRange's falsy-zero check (handler.ts:78): if (!row?.from || !row?.to) return null; treats a legitimate MIN/MAX value of 0 the same as "no rows." Immaterial for Date columns (a Date object is always truthy) but would misfire if discoverRange were ever called against a numeric zero-based column — related to point 1.

3. Minor doc typo in interval.ts:4: "Thirteen rungs, from one second to twelve-and-a-half minutes" — MAX_INTERVAL_MS is 46_080_000 ms = 768 minutes = 12.5 hours, not minutes. Ships into consumers' trees verbatim via the registry, so worth a one-word fix.

4. Handler construction is duplicated verbatim between route.ts and mcp/route.ts — same filters, columnMapping, cursorColumn, and select. Not a bug (this predates the PR and is arguably fine for a demo app), but it's exactly the kind of two-copies-drift risk this PR's own description calls out for the old row-remapping code. Could be worth extracting a shared createLogsHandler() factory in a follow-up.

Everything else checks out:

  • The projected select correctly excludes message/headers/uuid from columnMapping (they're not filtered/sorted) and both routes pass identical select objects, so REST/MCP parity holds.
  • getCursorValue now reads by schema key (row[cursorColumn]) everywhere, including the tied-group refetch at handler.ts:386 — the fix is applied consistently, not just in the common path.
  • The unmapped-key throw at construction only checks filters.specs against columnMapping (not the reverse), which is correct — label-style mapped-but-not-filtered keys are legitimately allowed, and the test suite pins that.
  • message: z.string().nullish() and the != null sheet condition correctly replace the ?? undefined normalization that the deleted remap used to do.
  • registry.test.ts's new relative-import resolution check is a good general fix, not just a one-off patch for interval.ts — it'll catch the same class of manifest gap for any future file.
  • Registry manifests (registry.json, both public/r/*.json) all correctly include interval.ts and the new DrizzleQueryScope/evaluateIntervalMs exports.

Not verified: I could not run pnpm turbo typecheck/lint/test in this sandbox — pnpm commands require interactive approval that isn't available here. Given the PR description states 2245 tests pass and typecheck/lint/prettier/registry:build are clean, and CI should confirm this independently, I'd suggest checking gh pr checks for this PR before merging if that hasn't run yet.

@mxkaske
mxkaske force-pushed the refactor/drizzle-projected-select branch from 282b246 to 0b1597e Compare August 13, 2026 12:21
mxkaske added a commit that referenced this pull request Aug 13, 2026
…n dropped

`handler.test.ts` read `row.uuid`, but `uuid` is not in `columnMapping` — it is
never filtered or sorted, so under this branch's "the projection IS the
mapping" rule it stops coming back unless a caller names it in `select`, which
is exactly what both production routes do. Every row therefore had
`uuid: undefined`, and `pages forward without overlap` compared `undefined` to
`undefined` and failed. Red in CI since the projection landed; only reproducible
there, since the suite is `skipIf(!hasDatabase)`.

The test helper now passes `select: { uuid }` like the routes do, and a new
case pins the contract that caused this — rows carry the mapped keys and
nothing else.

Also, both doc-only:

- `cursorColumn` now says that `scope.range`/`bucketMs` are read off it, so a
  non-time cursor paginates correctly but hands aggregate callers a range built
  from `new Date(id)`.
- The interval ladder's docstring said its thirteen rungs top out at
  "twelve-and-a-half minutes". They top out at 384 minutes; 768 is the
  fall-through above them. Ships into consumer trees verbatim.

Refs #91

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mxkaske
mxkaske force-pushed the refactor/drizzle-projected-select branch from 0b1597e to 6dd0263 Compare August 13, 2026 13:07
mxkaske added a commit that referenced this pull request Aug 14, 2026
…n dropped

`handler.test.ts` read `row.uuid`, but `uuid` is not in `columnMapping` — it is
never filtered or sorted, so under this branch's "the projection IS the
mapping" rule it stops coming back unless a caller names it in `select`, which
is exactly what both production routes do. Every row therefore had
`uuid: undefined`, and `pages forward without overlap` compared `undefined` to
`undefined` and failed. Red in CI since the projection landed; only reproducible
there, since the suite is `skipIf(!hasDatabase)`.

The test helper now passes `select: { uuid }` like the routes do, and a new
case pins the contract that caused this — rows carry the mapped keys and
nothing else.

Also, both doc-only:

- `cursorColumn` now says that `scope.range`/`bucketMs` are read off it, so a
  non-time cursor paginates correctly but hands aggregate callers a range built
  from `new Date(id)`.
- The interval ladder's docstring said its thirteen rungs top out at
  "twelve-and-a-half minutes". They top out at 384 minutes; 768 is the
  fall-through above them. Ships into consumer trees verbatim.

Refs #91

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mxkaske
mxkaske force-pushed the refactor/drizzle-projected-select branch from 6dd0263 to 8c28095 Compare August 14, 2026 07:02
Base automatically changed from feat/filter-semantics to main August 14, 2026 07:05
mxkaske and others added 2 commits August 14, 2026 09:05
…llers a typed scope

`createDrizzleHandler` fetched rows with a star select, so they came back keyed
by Drizzle property names (`timingDns`) while the wire contract and the entire
schema layer are keyed by schema keys (`"timing.dns"`). Every caller translated
by hand on the way out.

That remap was written twice, verbatim, and the two copies had already drifted:
the MCP route emitted `date.toISOString()` where REST emitted a `Date`, and
omitted `headers` and `percentile`. Its TODO already named the fix — the
camelCase↔dot-notation bijection was right there in `columnMapping`.

The handler now builds its select from that mapping, so rows arrive keyed by
schema keys and both remaps are deleted. Columns that belong in the payload but
are never filtered or sorted go in a new `select` option.

`allConditions: SQL[]` is replaced by `scope: DrizzleQueryScope`. That leak was
what made 110 lines of untested `db: any` SQL necessary: the three things such
a caller needs — the resolved range, the bucket interval, and the composed
WHERE — were all computed inside the handler and thrown away. `getChartData`
keeps its aggregate SQL and stops re-deriving anything.

Fixes a latent pagination bug. `getCursorValue` read `row[cursorCol.name]` —
the SQL column name, not the JS key. They coincide for `date`. For any cursor
column whose DB name differs (`timing_dns`), the lookup was `undefined`, so
`getCursorValue` returned null, `boundaryValue` was null, the whole tie-snapping
block was skipped, and `nextCursor` came back null — silently ending pagination
after one page. A projected select makes this reachable to fix, and the
regression test uses a table whose cursor column is deliberately renamed.

Unmapped keys stop vanishing. `buildWhereConditions` and `buildOrderBy` both
skipped a key missing from `columnMapping`, and `ColumnMapping` is
`Record<string, Column>` so a typo was not a type error either — the filter just
stopped filtering, with no error and no failing test. The handler now throws at
construction, listing the missing keys with the snippet to paste.

`evaluateIntervalMs` moves out of the demo route into the drizzle block, where
its 13 rungs can be tested directly instead of through a chart query.

Also fixed, found while testing:
- `interval.ts` shipped in the block's code but was missing from the manifest,
  so `shadcn add data-table-drizzle` installed files importing a module it never
  wrote. `registry.test.ts` only checked alias imports; it now also resolves
  relative ones, which is the check that would have caught it.
- The deleted remap normalized `message` with `?? undefined`. Projecting the
  column directly returns SQL NULL, and the sheet condition tested
  `!== undefined` — so rows with no message would have started rendering an
  empty field. The condition and the Zod schema now both accept null.

Tests: the projection identity test is what makes deleting the remaps safe.
`column-mapping.ts` is finally imported by a test — the suite had been
substituting its own divergent copy that added a `message` entry production did
not have. REST and MCP payloads are asserted to agree modulo the two deliberate
differences. All of it runs ungated on PGLite.

Refs #88

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n dropped

`handler.test.ts` read `row.uuid`, but `uuid` is not in `columnMapping` — it is
never filtered or sorted, so under this branch's "the projection IS the
mapping" rule it stops coming back unless a caller names it in `select`, which
is exactly what both production routes do. Every row therefore had
`uuid: undefined`, and `pages forward without overlap` compared `undefined` to
`undefined` and failed. Red in CI since the projection landed; only reproducible
there, since the suite is `skipIf(!hasDatabase)`.

The test helper now passes `select: { uuid }` like the routes do, and a new
case pins the contract that caused this — rows carry the mapped keys and
nothing else.

Also, both doc-only:

- `cursorColumn` now says that `scope.range`/`bucketMs` are read off it, so a
  non-time cursor paginates correctly but hands aggregate callers a range built
  from `new Date(id)`.
- The interval ladder's docstring said its thirteen rungs top out at
  "twelve-and-a-half minutes". They top out at 384 minutes; 768 is the
  fall-through above them. Ships into consumer trees verbatim.

Refs #91

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mxkaske
mxkaske force-pushed the refactor/drizzle-projected-select branch from 8c28095 to a05c9bc Compare August 14, 2026 07:05
@mxkaske
mxkaske merged commit 5017387 into main Aug 14, 2026
20 checks passed
@mxkaske
mxkaske deleted the refactor/drizzle-projected-select branch August 14, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant