sqlite: add tagged template

[0hmX]

Closes #57570

This pr introduces the support for tagged templates and an LRU to cache the templates. We introduced a new object called SqlTagStore that holds the ref to Lru. This acts as the main object that allows us to use tagged templates.

Look at the test file test-sqlite-template-tag.js to see how the api is designed.

---

[OLD]
The current implementation is experimental and intended to gather early feedback on the proposed direction.

This initial approach introduces two new functions:

• processSqlTemplate(): [the actual string interpolation logic.]
• createSqlTag(): [a higher order function that stores the native db and returns another processSqlTemplate .]

Note: The API design is not final and is expected to evolve based on feedback.

Questions for Maintainers

1. Is this direction a suitable foundation for solving the issue? Are there any alternative approaches I should consider?

As this is my first feature development for the project, any feedback on the code, approach, or contribution process would be greatly appreciated.

Here is my current plane for the usage

const {createSqlTag, DatabaseSync} = require("node:sqlite")

const db = new DatabaseSync(":memeory:")

const template = createSqlTag(db)

template`sql statement` // return the prepared statement

[END OLD]

[0hmX]

looking forward to your answers cc @nodejs/sqlite

[RafaelGSS]

cc: @geeksilva97 @miguelmarcondesf

[jasnell]

Since the API requires passing in the database instance, I'd prefer something like...

const db = new DatabaseSync(":memory:");
const template = db.createSqlTag();

As opposed to a standalone top-level function.

[geeksilva97]

Hello @0hmX . I wonder how this will fit the current API interface. I like @jasnell's suggestion. Do you think we could make it like the following?

const db = new DatabaseSync(":memory:");
const template = db.createSqlTag();

template.run`...`;
template.get`...`
template.all`...`

EDIT: I'm not sure if my question makes sense. I asked about it in the issue to get more information.

[geeksilva97]

From a technical point of view, @0hmX . This PR needs tests and documentation.

The cache is important in the implementation since statements are meant to be reused. With template tags, we miss this. In such a situation, the cache is important, as in Matteo's implementation.

I also wonder if we could have this implementation on C++ side, as all the implementations of node:sqlite module.

[0hmX]

@geeksilva97 and @jasnell, thank you for the feedback.

One of the key features that tag templates could bring is SQL syntax highlighting. However, this will only work if we have a pattern like this:

const willSyntaxHighlight = sql`SELECT * FROM users WHERE age = ${age}` // will get syntax highlighted with extensions
const willNotSyntaxHighlight = `SELECT * FROM users WHERE age = ${age}`

This is the pattern I see in all the extensions that allow you to highlight a string template inside ts or js for SQL in VS Code.

We need to have a template function called sql in front. Therefore, we should export a top-level function called SQL that returns an object with the raw SQL string and parameters. I have implemented this in C++ and added it! This means we will need to add support for this custom object in database.exec (we will convert the custom object into a raw string) and database.prepare.

const db = new DatabaseSync(":memory:");
const template = db.createSqlTag();

I think the name should be changed to something like db.createCache(). as this is creating a cache, storing the SQL string, and checking if a prepared statement is already made for it and calling the appropriate method on it and returning the outputs.

the final Api should look something like this

const { sql, DatabaseSync } = require("node:sql")

const db = new DatabaseSync(":memory:")

// adding sql template function in front for syntax highlight, else it's the same as using a multiline string
db.exec(sql`CREATE TABLE contacts (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    phone TEXT NOT NULL UNIQUE,
    email TEXT NOT NULL UNIQUE
)`)

// same as using a multiline string
db.exec(sql`INSERT INTO contacts (name, phone, email) VALUES (${"Alice"}, ${"111-222-3333"}, ${"[email protected]"})`)
db.exec(sql`INSERT INTO contacts (name, phone, email) VALUES (${"Bob"}, ${"444-555-6666"}, ${"[email protected]"})`)
db.exec(sql`INSERT INTO contacts (name, phone, email) VALUES (${"Charlie"}, ${"777-888-9999"}, ${"[email protected]"})`)

const cache = db.createCache()

const emailDomain = "%@example.com"
cache.all(sql`SELECT * FROM contacts WHERE email LIKE ${emailDomain}`)

const contactId = 2
cache.get(sql`SELECT * FROM contacts WHERE id = ${contactId}`)

const namePrefix = "C%"
cache.iterate(sql`SELECT * FROM contacts WHERE name LIKE ${namePrefix} ORDER BY name`)

[bakkot]

One of the key features that tag templates could bring is SQL syntax highlighting. However, this will only work if we have a pattern like this: [...] We need to have a template function called sql in front.

VSCode supports more complex patterns than just literally an identifier, I'm almost certain. It can be made to handle db.prepare or anything.sql just as well as a bare sql. See this repo for an example.

// same as using a multiline string

It should very much not be the same as using a multiline string. The whole point of tagged templates - literally the only reason they are in the language - is that unlike strings they allow using a representation which is immune to problems like sql injection. From your implementation it looks like you're correctly avoiding sql injections, but this is something which should be emphasized in the docs and tested extensively. Users need to be aware that omitting the tag is not safe; it's not just a convenience for getting syntax highlighting.

On that note, you should also be doing parsing of the string up front: users should get an error as soon as they write sql`invalid`; rather than needing to actually pass it somewhere to get an error. Note that the first argument to the template tag is always the same value on subsequent calls so that it can be used as a key in a cache instead of needing to check every time the function is called. Probably the easiest thing is to prepare the statement right away, and store the prepared statement in a cache.

[codecov]

Codecov Report

Attention: Patch coverage is 77.23785% with 89 lines in your changes missing coverage. Please review.

Project coverage is 90.04%. Comparing base (5f7dbf4) to head (686a09c).
Report is 193 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_sqlite.cc	75.22%	27 Missing and 56 partials ⚠️
src/lru_cache-inl.h	88.88%	2 Missing and 2 partials ⚠️
src/node_sqlite.h	90.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #58748      +/-   ##
==========================================
- Coverage   90.16%   90.04%   -0.13%     
==========================================
  Files         636      646      +10     
  Lines      188060   189432    +1372     
  Branches    36899    37142     +243     
==========================================
+ Hits       169568   170565     +997     
- Misses      11235    11523     +288     
- Partials     7257     7344      +87     

Files with missing lines	Coverage Δ
src/node_sqlite.h	80.39% <90.00%> (+12.39%)	⬆️
src/lru_cache-inl.h	88.88% <88.88%> (ø)
src/node_sqlite.cc	79.27% <75.22%> (-1.52%)	⬇️

... and 144 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[0hmX]

The PR is as stable as it gets now with all the tests passing. I need reviews on the PR. Look at the test file test-sqlite-template-tag.js to see how the api is designed.

cc: @jasnell @geeksilva97 @bakkot also @mcollina (as creator of the main issues)

[mcollina]

lgtm

[mcollina]

lgtm

[geeksilva97]

Thanks, just wanted to ask, can we make this capital naming conversion some lint rule? We already have lint rules for C++! Then I will make an issue and a Pr for it! just wanted to know if there is any special reason this was not already done!

I think so. Could you open an issue for discussion? I'm pretty new to Node C++ codebase too. Getting more experienced folks discussing will be good.

[geeksilva97]

A humble ping to @nodejs/cpp-reviewers to get more experienced folks' opinions.

[github-actions]

Failed to start CI

   ⚠  Commits were pushed since the last approving review:
   ⚠  - sqlite: add tagged template support
   ⚠  - Update doc/api/sqlite.md
   ⚠  - Update doc/api/sqlite.md
   ⚠  - Update src/node_sqlite.cc
   ⚠  - Update src/node_sqlite.cc
   ⚠  - Update src/lru_cache-inl.h
   ⚠  - Update src/lru_cache-inl.h
   ⚠  - Update src/lru_cache-inl.h
   ⚠  - Update src/lru_cache-inl.h
   ⚠  - Update src/lru_cache-inl.h
   ⚠  - Update src/lru_cache-inl.h
   ⚠  - fix lint
   ⚠  - rename to SQLTagStore
   ⚠  - sqlite: refactor
   ⚠  - Update src/node_sqlite.cc
   ⚠  - Update src/node_sqlite.cc
   ⚠  - Update src/node_sqlite.cc
   ⚠  - Update src/node_sqlite.cc
   ⚠  - Update src/node_sqlite.cc
   ⚠  - sqlite: getting back to old days
   ⚠  - sqlite: remove excess use of v8::
   ⚠  - sqlite: support for MemoryInfo
   ✘  Refusing to run CI on potentially unsafe PR

https://github.com/nodejs/node/actions/runs/16306591791

[0hmX]

Should I manually squash these commits?