Skip to content

Add inline sqlfu support for Durable Objects#139

Merged
mmkal merged 36 commits into
mainfrom
bedtime/2026-05-27-durable-object-inline-sqlfu
Jun 11, 2026
Merged

Add inline sqlfu support for Durable Objects#139
mmkal merged 36 commits into
mainfrom
bedtime/2026-05-27-durable-object-inline-sqlfu

Conversation

@mmkal

@mmkal mmkal commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds inline sqlfu modules for self-contained Durable Objects. A Durable Object can keep its schema, migrations, and queries in one Worker TypeScript module as a static defineConfig(...) class property, then point the CLI at that module:

npx sqlfu --config src/durable-objects/posts.ts draft
npx sqlfu --config src/durable-objects/posts.ts generate
npx sqlfu --config src/durable-objects/posts.ts generate --watch

The inline path is source-backed, not config-import-backed: the CLI scans TypeScript source for inline defineConfig({...}) object literals on top-level consts or static properties of top-level named classes. Inline handling is enabled for draft and generate; other commands continue to require a file-backed sqlfu config.

The generated inline query shape is now compact. Users can start with plain direct sql query values, and generate rewrites them into sql.many<{...}>, sql.one<{...}>, sql.nullableOne<{...}>, or sql.run<{...}> tags. The mode tags are also available on the main sql export generally; sql.run is the preferred alias for the metadata/run-result mode, with sql.metadata still available.

Example

import {DurableObject} from 'cloudflare:workers';
import {createDurableObjectClient, defineConfig, sql} from 'sqlfu';

export class PostsObject extends DurableObject {
  static db = defineConfig({
    definitions: sql`
      create table posts (
        slug text primary key not null,
        published_at text,
        body text
      );
    `,
    migrations: [],
    queries: {
      listPosts: sql`
        select slug, published_at, body
        from posts
        order by slug
        limit :limit
      `,
      createPost: sql`
        insert into posts(slug, body)
        values (:slug, :body)
      `,
    },
  });

  db: ReturnType<typeof PostsObject.db<ReturnType<typeof createDurableObjectClient>>>;

  constructor(ctx: DurableObjectState, env: {}) {
    super(ctx, env);
    this.db = PostsObject.db(createDurableObjectClient(ctx.storage));
    this.db.migrate();
  }

  posts() {
    return this.db.listPosts({limit: 10});
  }
}

After draft and generate, sqlfu writes the migration entries and generated query metadata back into the same source module:

import {DurableObject} from 'cloudflare:workers';
import {createDurableObjectClient, defineConfig, sql} from 'sqlfu';

export class PostsObject extends DurableObject {
  static db = defineConfig({
    definitions: sql`
      create table posts (
        slug text primary key not null,
        published_at text,
        body text
      );
    `,
    migrations: [
      {
        name: '2026-05-28T00.00.00.000Z_create_posts',
        content: sql`
          create table posts (
            slug text primary key not null,
            published_at text,
            body text
          );
        `,
      },
    ],
    queries: {
      listPosts: sql.many<{ parameters: { limit: number }; result: { slug: string; published_at: string | null; body: string | null } }>`
        select slug, published_at, body
        from posts
        order by slug
        limit :limit
      `,
      createPost: sql.run<{ parameters: { slug: string; body: string | null } }>`
        insert into posts(slug, body)
        values (:slug, :body)
      `,
    },
  });

  db: ReturnType<typeof PostsObject.db<ReturnType<typeof createDurableObjectClient>>>;

  constructor(ctx: DurableObjectState, env: {}) {
    super(ctx, env);
    this.db = PostsObject.db(createDurableObjectClient(ctx.storage));
    this.db.migrate();
  }

  posts() {
    return this.db.listPosts({limit: 10});
  }
}

Before / After

Before this PR, Durable Objects had to use the normal file-backed shape: a separate sqlfu.config.ts, definitions.sql, migrations/, and sql/ directory per object. Inline modules were not supported, and generate --watch rejected inline defineConfig(...) modules.

Notes for Reviewers

  • Static class properties are the recommended Durable Object shape because the config stays attached to the object that owns the storage. Module-level const configs remain supported, including multiple configs in one module.
  • The public inline API is the root defineConfig(...) overload plus sql from sqlfu; there is no separate inlineSqlfu helper.
  • Inline source edits infer indentation, quote style, and trailing-comma preference to avoid formatter churn when writing migrations and generated query tags.
  • The source reader does not import the configured module, so Worker globals and user module side effects do not run in the Node CLI.
  • Inline query result types intentionally match raw Durable Object SQLite rows: column names stay as returned by SQLite, and nullable result columns are typed as T | null.
  • Inline generation rejects query shapes that need generated-wrapper runtime expansion, such as in (:ids) and object-field parameters, because the inline runtime binds the SQL template directly today.
  • Inline definitions, migrations, and config queries must have no template interpolations. The public sql.many / sql.one / sql.run tags still allow interpolations in non-inline contexts.
  • The source writer still handles the object-form inline query shape so existing branch output can be regenerated safely, but the docs and primary fixtures now prefer compact direct query tags.

Verification

pnpm lint --fix
pnpm --filter sqlfu typecheck
pnpm build
pnpm sync:root-readme:check
pnpm --filter sqlfu exec vitest run test/pkg.test.ts
pnpm --filter sqlfu exec vitest run test/generate-watch.test.ts
pnpm --filter sqlfu exec vitest run test/inline-source.test.ts test/config-inline.test.ts test/generate-watch.test.ts test/pkg.test.ts test/import-surface.test.ts test/adapters/durable-object.test.ts

Self-review hardening (2026-06-11)

A red/green verification pass over the whole branch (failing tests pushed first in 02a3b40, fixes after):

  • Scanner robustness: the inline source scanner ran on every config load and threw on valid TypeScript — regex literals (name.replace(/['"]/g, '_') anywhere in the file broke every CLI command), spread properties, and arrow function types inside sql.many<...> type args. It now lexes regexes, skips object elements it can't model while probing, and treats => as an arrow.
  • SQL comments: the runtime sql tag collapses whitespace, so a -- comment used to swallow the rest of the statement — an inline migration could be recorded as applied while its DDL never ran. Comments are now stripped (string-aware) before collapse.
  • Hoisted definitions: definitions: someHoistedTag silently classified the module as file-backed, making the CLI dynamic-import the Durable Object module and crash on cloudflare:workers. It now fails with an error naming the fix.
  • Drafting: appendInlineMigration no longer writes the separating comma inside a trailing // comment; static SQL is template-escape-decoded so it matches what the runtime executes.
  • Generate: one failing query no longer stales every other query's types — valid types are written, then a summary error names the broken queries.
  • sqlfu/api: dynamic imports are literal again (bundling sqlfu/api with esbuild now works — there's a packed-package test for it), Confirm is re-exported from core, and config()/serve()/kill() regained precise return types.
  • Dead code: the {query, $type, mode} object query form (unused since compact tags) is deleted along with ~130 lines of writer machinery; inline query binding validates once at bind time and the DO adapter caches the named→positional SQL rewrite per query.
  • Watch: file-backed and inline watch share one loop, and the inline watcher skips its own type-annotation writes (previously every save cost a redundant second typegen cycle).
  • Published package types: a publish-shaped build used to ship .d.ts files referencing deleted vendor modules, so consumers with skipLibCheck: false couldn't compile against the package. bundle-vendor now keeps the declaration graph coherent.
  • CI: the sqlfu job only ran test/migrations + test/schemadiff, which is how stale fixtures and the issues above shipped silently. It now builds (publish-shaped), typechecks, and runs the full suite.
Package size — packed 280.3 kB (+24.4 kB, +9.5%)

Package size

main this PR Δ
packed 255.9 kB 280.3 kB +9.5%
unpacked 1.03 MB 1.12 MB +8.8%
files 190 226 +36

dist/vendor/*.js bundles

main this PR Δ
vendor/sha256.js 4.3 kB 4.3 kB 0
vendor/sql-formatter/*.js 94.5 kB 94.5 kB 0
vendor/sqlfu-sqlite-parser/*.js 17.2 kB 17.2 kB 0
vendor/standard-schema/*.js 2.8 kB 2.8 kB 0
vendor/typesql/*.js 133.6 kB 134.8 kB +0.9%

Measured with npm pack --dry-run --json on sqlfu (0.0.3-7 on main vs 0.0.3-7 on this PR).


Note

Medium Risk
Large surface area (source parser, CLI routing, runtime binding, vendored bundles) with extensive tests; inline configs are intentionally limited to generate/draft, reducing blast radius for migrate/serve paths.

Overview
Adds inline defineConfig for self-contained Durable Object modules: schema, migrations, and queries live in a Worker .ts file (e.g. static db = defineConfig({...})), and --config can point at that module instead of sqlfu.config.ts.

CLI & API: draft, generate, and generate --watch detect inline configs via static source parsing (no dynamic import of the Worker module). They append migration entries and rewrite queries to compact tags (sql.one<{...}>, sql.many<{...}>, sql.run<{...}>). Migrate, serve, UI, and most other commands reject inline projects with a clear error. sqlfu/api lazy-loads Node code; format is async.

Runtime: Overloaded root defineConfig returns an inline factory with migrate() and typed query methods. The sql tag gains mode helpers and strips SQL comments before whitespace collapse. Durable Object prepare() caches named-parameter rewriting.

Tooling: Shared watchAndRegenerate; vendor TypeSQL split for browser-safe analysis; bundle-vendor fixes published .d.ts; CI runs full sqlfu build, typecheck, and vitest (plus Bun for adapter tests).

Reviewed by Cursor Bugbot for commit 2da59d5. Bugbot is set up for automated code reviews on this repo. Configure here.

@pkg-pr-new

pkg-pr-new Bot commented May 27, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/sqlfu@139

commit: 2da59d5

Comment thread packages/sqlfu/src/api/inline.ts Outdated
Comment thread packages/sqlfu/src/api/exports.ts
Comment thread packages/sqlfu/package.json Outdated
Comment thread packages/sqlfu/src/node/inline-source.ts
mmkal added 6 commits May 28, 2026 12:03
Move inline SQL configs onto the root defineConfig API, including static class properties for Durable Object-owned schemas.

Generate and draft inline query metadata and migrations from defineConfig sources, with markdown fixture coverage for class and module-level configs.

Add runtime tests for fully augmented inline configs and update Durable Object docs/import guidance.
Comment thread packages/sqlfu/src/node/inline-commands.ts Outdated
Comment thread packages/sqlfu/src/config-inline.ts
Comment thread packages/sqlfu/src/node/inline-source.ts Outdated
Comment thread packages/sqlfu/src/config-inline.ts
Comment thread packages/sqlfu/src/config-inline.ts
ensureMigrationTableGen had its own fallback DDL string (no check
constraint) that diverged from sqliteDialect's defaultMigrationTableDdl.
Callers that don't pass a dialect — inline defineConfig().migrate() and
generated migration bundles — created a subtly different table.

Move the canonical DDL (with the `name not like '%.sql'` check) into
migrations/preset-queries.ts and have dialect.ts import it, so every
sqlite path creates an identical table without the migration runtime
needing to import dialect.ts. Dialects with different column flavors
(@sqlfu/pg) still override via defaultMigrationTableDdl.

Addresses bugbot finding on PR #139.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deferred from a bugbot finding on PR #139 — the type hole exists in both
the inline runtime and the generated wrappers, so it should be fixed for
both together rather than patched in the inline PR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/sqlfu/src/config-inline.ts Outdated
Red tests for the verified bugs from the PR self-review, fixes to follow
in the next commit(s):

1. The inline source scanner runs on every config load and hard-throws
   on valid TypeScript: regex literals (mis-lexed as string openers),
   spread properties in file-backed defineConfig calls, and arrow
   function types inside sql tag type arguments (the => counts as an
   angle-bracket closer).
2. Inline migrations/queries execute whitespace-collapsed SQL, so a
   `--` line comment swallows the rest of the statement; the migration
   is recorded as applied while its DDL never ran.
3. Hoisted (non-literal) definitions make the scanner silently classify
   an inline module as file-backed, so the CLI dynamic-imports the
   Durable Object module and crashes with an unrelated error.
4. appendInlineMigration puts the separating comma inside a trailing
   line comment, corrupting the module.
5. readSqlTemplate returns raw source text without decoding template
   escapes, so static analysis sees different SQL than runtime executes.
6. generateInlineConfigTypes aborts the whole module on the first
   failing query instead of writing the valid queries' types.
7. sqlfu/api routes dynamic imports through load(specifier), which
   bundlers cannot follow (bundle breaks at call time), and config()/
   serve() regressed to Promise<unknown>.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/sqlfu/src/typegen/index.ts
Comment thread packages/sqlfu/src/node/inline-source.ts
Comment thread packages/sqlfu/src/node/inline-source.ts Outdated
mmkal and others added 3 commits June 11, 2026 11:56
Goes green on the failing tests from the previous commit:

1. Scanner robustness (inline-source.ts): lex regex literals with the
   standard prev-token heuristic (bailing to division on no same-line
   close, so misdetection degrades gracefully); skip object elements the
   scanner can't model (spread/shorthand/methods) instead of throwing —
   confirmed-inline contexts still reject them with a clear message; and
   treat the > of => as part of an arrow, not an angle closer.
2. Strip sql comments in the runtime sql tag before whitespace collapse
   (sql.ts), respecting strings and quoted identifiers, so a -- comment
   no longer swallows the rest of a migration or query.
3. Hoisted/computed `definitions` in an sql-tag-bearing defineConfig
   call now throws a descriptive error instead of silently classifying
   the module as file-backed (which made the CLI dynamic-import Durable
   Object modules and crash on cloudflare:workers).
4. appendInlineMigration inserts the separating comma after the previous
   element's last code character instead of appending it inside a
   trailing line comment.
5. readSqlTemplate cooks template escapes so static analysis sees the
   same SQL the runtime executes.
6. generateInlineConfigTypes collects per-query analysis failures and
   still writes the valid queries' types before throwing a summary
   error naming the failing queries; one mid-edit typo in watch mode no
   longer stales every other query's types.
7. sqlfu/api uses literal dynamic imports (bundler-followable, same
   laziness), re-exports Confirm from core instead of duplicating it,
   restores precise config()/serve()/kill() return types, and uses the
   process global instead of dynamically importing node:process. The
   bundling test externalizes bun:sqlite and vite, the two genuinely
   environment-optional imports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Commit 1e21311 removed the single-line rendering for short inline
migration bodies, but the inline-source fixtures and the durable object
redeploy test still expected the old compact form. CI never caught this
because the sqlfu unit-tests job only runs test/migrations and
test/schemadiff. Fixture expectations regenerated with --update; the
redeploy test's regexes now accept the multiline rendering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…O hot path

The {query: sql`...`, $type, mode} object form has been dead since
"Prefer compact inline query tags" — the generator only emits compact
tags and the runtime mode detection never looked inside it (the open
bugbot finding). Deleting it removes the runtime branch, the type-level
$type plumbing, and ~130 lines of position-sensitive writer machinery
in inline-source.ts along with its fixtures.

With the query shape down to plain tagged templates, inline query
binding now validates mode/interpolations once at bind time and
captures the sync/async dispatch in the closure — per-call work inside
a durable object's event loop is just prepare + execute. Validation
errors still surface on first call rather than at bind, so an
ungenerated module can still construct its durable object.

The durable object adapter also caches the named→positional SQL rewrite
per distinct query text (prepareSqlParamsBinding in sql-params.ts,
which bindSqlParamsToPositional now delegates to): the rewritten SQL is
a pure function of the query text, so re-execing the same statement no
longer rescans every character of the SQL per call.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/sqlfu/src/node/inline-source.ts
mmkal and others added 3 commits June 11, 2026 15:30
watchGenerateInlineConfigModule was a near-verbatim copy of the
file-backed watcher's coalescing loop, minus its self-write guard — the
inline generator writes type annotations into the very file it watches,
so every save triggered a redundant second full typegen cycle (temp db
materialization + reanalysis) that converged only because the second
write was a no-op.

Both modes now run through watchAndRegenerate in node/watcher.ts
(initial run, debounce, coalescing, ready/abort/close, error logging),
and the inline mode passes a shouldRegenerate veto that compares the
module content against what it last generated, so the watcher's own
write no longer costs a generate cycle. Regression test asserts a user
edit costs exactly one cycle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lfu suite in CI

A publish-shaped build (pnpm build with bundle-vendor) left
dist/vendor/sql-formatter/allDialects.d.ts referencing 18 deleted
dialect modules, and dialect.d.ts/FormatOptions.d.ts referencing the
deleted formatter/lexer/parser output — any consumer compiling with
skipLibCheck: false failed to typecheck the published package.
bundle-vendor now keeps the declaration files those kept declarations
import (deleting only the runtime .js, ~36 small .d.ts files; tarball
stays at 281K) and rewrites allDialects.d.ts to match the runtime
exposed-dialects rewrite (sqlite + postgresql only).

The CI sqlfu job previously ran only test/migrations and
test/schemadiff — which is how broken fixtures, a failing durable
object test, and the published-package type breakage all shipped
silently. It now does a full publish-shaped build, typechecks, and runs
the whole suite, so test/pkg.test.ts exercises the real publish
artifact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first full-suite CI run surfaced the tests' dev-machine assumptions:
bun adapter tests spawn real bun subprocesses (setup-bun), the
resolve-sqlfu-ui tests resolve @sqlfu/ui's #serialized-assets import
from its build output (build the ui package), and the alchemy-dev and
better-auth integration tests exceeded timeouts tuned for warm local
runs. Test timeouts now scale on CI only — local stays at vitest's
aggressive 5s default so hangs keep surfacing fast in the dev loop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/sqlfu/src/node/config.ts
Alchemy hard-refuses its default local state store when CI is set. The
miniflare-d1-path fixture runs alchemy dev against a throwaway temp dir
with fake credentials — exactly the case ALCHEMY_CI_STATE_STORE_CHECK
exists for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/sqlfu/src/node/inline-source.ts
stripSqlComments was string/identifier-aware but didn't know postgres
dollar quoting, which @sqlfu/pg users hit through the shared sql tag: a
-- or /* */ sequence inside $$...$$ data got stripped, corrupting the
string value. Scan $tag$...$tag$ ranges like sql-params.ts already
does; a $ that doesn't open a dollar string (e.g. a $1 placeholder)
stays a plain character.

Prompted by a bugbot thread on PR #139 whose stated claim (inline draft
baselines diverging from runtime over comments) doesn't hold — both
representations are parsed by sqlite and nothing compares them
textually — but whose verification surfaced this adjacent gap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/sqlfu/src/node/cli-router.ts
Pointing serve (also the default command) at an inline defineConfig
module started the server and logged "sqlfu ready", but the UI resolver
throws for inline projects on every request — a success message
fronting a dead studio. Both serve paths (CLI handler and sqlfu/api)
now reject inline projects up front with the same error the other
non-inline commands use, via assertServableProject next to
LoadedSqlfuProject in config.ts. Uninitialized directories still serve
fine — the UI's init flow depends on that.

Addresses a bugbot finding on PR #139.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 48cd01f. Configure here.

Comment thread packages/sqlfu/src/index.ts
The overloads already reject them at compile time, but an untyped
caller mixing a sql`...` template with a string path got the plain
file-backed object back and a confusing "db is not a function" far from
the config. The runtime dispatch now throws a shape error naming both
valid forms. Configs with one field simply missing still route to the
file-backed path so its missing-field validation can name the field.

Addresses a bugbot finding on PR #139.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mmkal mmkal merged commit ed9ad0f into main Jun 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant