feat: add source parameter support to query execution endpoints#670
Open
quirkyllama wants to merge 11 commits intomainfrom
Open
feat: add source parameter support to query execution endpoints#670quirkyllama wants to merge 11 commits intomainfrom
quirkyllama wants to merge 11 commits intomainfrom
Conversation
Allow callers to pass source parameters (key-value string maps) to query execution and notebook cell execution endpoints. Parameters are validated against declared Malloy source parameter types, converted to the correct Malloy literal syntax, and injected into query strings. - New `source_params.ts` utility with type-aware parsing, validation, and Malloy clause building - REST endpoints accept `sourceParameters` in request body/query params - MCP `malloy_executeQuery` tool accepts `sourceParameters` field - Notebook cells with compilation errors (e.g. missing required params) are deferred to execution time and recompiled with injected params - OpenAPI spec and agent docs updated - Comprehensive unit tests (59 new tests across 2 files) Made-with: Cursor
…rd<string, unknown> Allow callers to pass native numbers, booleans, etc. directly instead of requiring everything to be string-encoded. The paramValueToMalloyLiteral function now handles both native types and string representations. Made-with: Cursor
Made-with: Cursor
Extra parameters not declared on the source are now skipped rather than rejected, so callers can pass a superset of params without error. Made-with: Cursor
I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: 74f0ff5 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: 7171a62 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: 6586e76 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: 615380d Signed-off-by: Josh Sacks <josh.sacks@gmail.com>
When a notebook cell fails compilation (e.g. missing required source parameters), the model-level getModel() also fails because the materializer includes the broken cell text. Previously this caused the entire notebook to be stored with a compilationError, making getNotebook() throw. Now Model.create() catches notebook model-level compilation failures gracefully: it collects sourceInfos from successfully compiled cells and returns a usable Model with per-cell error info intact. This lets the publisher load packages containing parameterized notebooks without crashing. Adds integration test with real DuckDB + Malloy experimental parameters that verifies the full load → getNotebook → executeCell flow. Made-with: Cursor
mhwilder
reviewed
Apr 13, 2026
| "packageName": "my_package", | ||
| "modelPath": "model.malloy", | ||
| "sourceName": "flights", | ||
| "queryName": "by_carrier", |
Collaborator
There was a problem hiding this comment.
I thought you were getting rid of this version of execute query.
Collaborator
Author
There was a problem hiding this comment.
I want to, but didn't think this PR was the right place. Happy to do this in another change though
Instead of deferring compilation errors for notebook cells that reference parameterized sources without providing values, the loader now injects type-appropriate stub values (e.g. 0 for number, "" for string) during compilation. This validates notebook structure at load time — real errors like typos are caught immediately, while parameter values are injected at execution time. Key changes: - Add stubValueForParam, buildStubParamClause, injectParamClauseIntoText utilities in source_params.ts - Rewrite getNotebookModelMaterializer to process cells sequentially, discovering parameterized sources incrementally and injecting stubs into subsequent cells - Cells compiled with stubs are marked requiresSourceParameters=true; executeNotebookCell enforces real params at execution time - Collect notebook sourceInfos from cell newSources (model-level modelDef.imports doesn't carry forward across extendModel stages) - Fix regex bug in injectParamClauseIntoText (global regex lastIndex advancing after test() caused replace() to miss) Made-with: Cursor
- Replace import.meta.dir with __dirname (CommonJS module mode) - Add optional chaining for c.text?.includes (text may be undefined) - Fix prettier formatting Made-with: Cursor
mhwilder
reviewed
Apr 13, 2026
The raw query execution path (query without sourceName/queryName) was validating sourceParameters but never injecting them into the query text. Now uses injectParamClauseIntoText to scan for `run: source ->` patterns and inject params, same as the notebook path. Also adds ##! experimental.parameters flag to the named-query path when params are present. Made-with: Cursor
Tests use a parameterized source over inline DuckDB data (4 rows with values 10, 50, 100, 200) filtered by min_val. Verifies both named query and ad-hoc query paths return correct row counts and aggregates for different parameter values, and that missing required params throw. Made-with: Cursor
I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: a4818b9 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: 55c7464 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: a4ccf42 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: ce347c7 I, Josh Sacks <josh.sacks@gmail.com>, hereby add my Signed-off-by to this commit: 636fdc5 Signed-off-by: Josh Sacks <josh.sacks@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sourceParameterssupport to all query execution endpoints (REST, MCP, notebook cells), enabling callers to pass typed key-value parameters to Malloy parameterized sourcessource_params.tsutility handles type-aware parsing (string, number, boolean, date, timestamp, filter expression), required-param validation, and Malloy literal clause buildingapi-doc.yaml) and agent documentation (docs/ai-agents.md) to describe the newsourceParametersfield and theimplicit_naming conventionModel.getQueryResultswith parameters, and notebook cell resilience/recompilationTest plan
source_params.spec.tstests pass (type conversion, validation, clause building)model.spec.tstests pass (including 16 new tests for getQueryResults + executeNotebookCell with sourceParameters)sourceParametersMade with Cursor