Skip to content

Commit 44677b1

Browse files
timsaucerclaude
andcommitted
docs: broaden SQL lambda dialect coverage
Other dialects (ClickHouse, Snowflake, Databricks) also enable lambda parsing via sqlparser-rs. Document the full set and recommend the ``lambda x: x`` keyword form, since DuckDB will drop the ``x -> x`` arrow form in v2.1. Parametrize the SQL test over the four dialects using the keyword syntax. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent bfc79e9 commit 44677b1

2 files changed

Lines changed: 15 additions & 17 deletions

File tree

docs/source/user-guide/common-operations/expressions.rst

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -186,10 +186,13 @@ If you need explicit control over parameter names, build the lambda with
186186

187187
Lambda expressions cannot yet be serialized: calling
188188
:py:meth:`~datafusion.expr.Expr.to_bytes` or pickling an expression that
189-
contains a lambda raises ``Lambda not implemented``. SQL lambda syntax
190-
(``x -> x * 2``) is only parsed by dialects that support lambdas; set
191-
``datafusion.sql_parser.dialect`` to ``DuckDB`` to use it. The Python
192-
expression builder shown above works regardless of dialect.
189+
contains a lambda raises ``Lambda not implemented``. SQL lambda syntax is
190+
only parsed by dialects that support lambdas; set
191+
``datafusion.sql_parser.dialect`` to one of ``DuckDB``, ``ClickHouse``,
192+
``Snowflake``, or ``Databricks``. Both arrow syntax (``x -> x * 2``) and
193+
keyword syntax (``lambda x: x * 2``) parse. DuckDB will drop the arrow
194+
form in v2.1, so prefer ``lambda x: x * 2`` for forward compatibility.
195+
The Python expression builder shown above works regardless of dialect.
193196

194197

195198
Testing membership in a list

python/tests/test_lambda.py

Lines changed: 8 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -125,19 +125,14 @@ def test_to_lambda_rejects_invalid_arg(arg, exc_type, match):
125125
f.array_transform(col("a"), arg)
126126

127127

128-
def test_sql_lambda_requires_duckdb_dialect():
129-
# Lambda arrow syntax (``x -> ...``) is only parsed by dialects that
130-
# support lambda functions. The default Generic dialect treats ``->`` as
131-
# the JSON arrow operator, so ``x`` is read as a column reference.
132-
ctx = SessionContext()
133-
with pytest.raises(Exception, match="No field named x"):
134-
ctx.sql("select array_transform([1, 2, 3], x -> x * 2) as d").collect()
135-
136-
duckdb_ctx = SessionContext(
137-
SessionConfig().set("datafusion.sql_parser.dialect", "DuckDB")
138-
)
139-
result = duckdb_ctx.sql(
140-
"select array_transform([1, 2, 3], x -> x * 2) as d"
128+
@pytest.mark.parametrize("dialect", ["DuckDB", "ClickHouse", "Snowflake", "Databricks"])
129+
def test_sql_lambda_keyword_syntax(dialect):
130+
# ``lambda x: x * 2`` is the forward-compatible syntax. DuckDB will drop
131+
# the arrow form (``x -> ...``) in v2.1; the keyword form is supported by
132+
# every dialect in sqlparser-rs that enables lambda functions.
133+
ctx = SessionContext(SessionConfig().set("datafusion.sql_parser.dialect", dialect))
134+
result = ctx.sql(
135+
"select array_transform([1, 2, 3], lambda x: x * 2) as d"
141136
).collect_column("d")
142137
assert result.to_pylist() == [[2, 4, 6]]
143138

0 commit comments

Comments
 (0)