Skip to content

fix: fail loudly on an unknown workflow expression filter#3074

Open
doquanghuy wants to merge 1 commit into
github:mainfrom
doquanghuy:fix/3073-unknown-filter-fail-loud
Open

fix: fail loudly on an unknown workflow expression filter#3074
doquanghuy wants to merge 1 commit into
github:mainfrom
doquanghuy:fix/3073-unknown-filter-fail-loud

Conversation

@doquanghuy

Copy link
Copy Markdown
Contributor

Description

Fixes #3073.

The expression evaluator's filter dispatch fell through to return value for any unregistered filter, so a typo'd or unsupported filter — e.g. {{ items | length }} — rendered the value unchanged with no error, and the run completed. A silent wrong result, with nothing pointing at the cause.

This makes the unknown-filter path fail loudly: an unrecognized filter raises a clear ValueError naming the offending filter and the valid ones, reusing the same strict-ValueError style already used for from_json (which raises on its mis-wired forms precisely to avoid "silently falling through to the unknown-filter path and returning the unparsed value"). It closes that path in general, consistent with the fail-loud direction of #2957 (fan-out items) and #2961 (from_json).

  • The five registered filters (default / join / map / contains / from_json) are unchanged — no behavior change for any valid usage.
  • Both forms of an unknown filter are caught: the no-arg form (| length) and the name(arg) form (| upper('x')), which previously also fell through silently.

Backward-compat note (calling it out honestly): this turns a previously silent author error into a loud one. Any workflow that today relies on an unknown filter being silently dropped would now error. That passthrough was never intentional (the from_json strictness comment already names it as a path to avoid), and a scan of the repo's own workflow/template YAMLs found no {{ … | <unknown-filter> }} usage, and the full suite stays green — but flagging it so you can weigh the direction. Happy to rework as a warning-instead-of-raise, or gate it, if you'd prefer.

Testing

  • Tested locally with uv run specify --help
  • Ran existing tests with uv sync && uv run pytest — full suite 3912 passed, 59 skipped
  • Tested with a sample project (if applicable) — ran a workflow whose shell step used {{ [1,2,3] | length }}: pre-fix it printed count=[1, 2, 3] and completed; post-fix the run fails loudly with unknown filter 'length'.

Three new tests in TestExpressions (tests/test_workflows.py):

  • test_filter_unknown_name_raises| length raises ValueError.
  • test_filter_unknown_name_with_args_raises| upper('x') raises (the name(arg) form).
  • test_registered_filters_unaffected — regression: all five registered filters still work.

The two unknown-filter tests are red against main (DID NOT RAISE), green with the fix.

AI Disclosure

  • I did use AI assistance (describe below)

This change was authored with Claude Code: the AI drafted the fix and the tests and ran the suite. I discovered the behavior through end-to-end workflow-expression testing, verified the root cause and the red-against-main / green-with-fix tests myself, and reviewed the diff. The fix deliberately reuses the existing from_json error style rather than inventing a new one.

The expression evaluator's filter dispatch fell through to `return value`
for any unregistered filter, so a typo'd or unsupported filter such as
`{{ items | length }}` rendered the value unchanged with no error and the
run completed — a silent wrong result.

Raise a clear ValueError instead, naming the offending filter and the valid
ones, mirroring the strict handling already used for `from_json`. The five
registered filters (default/join/map/contains/from_json) are unchanged; the
`name(arg)` form of an unknown filter is now caught too.
@doquanghuy

Copy link
Copy Markdown
Contributor Author

@mnriem this is the fix for #3073 — fail-loud on unknown expression filters, reusing the from_json ValueError style; the five registered filters are unchanged and the full suite stays green. Happy to adjust the direction (warn instead of raise, or gate it) if you'd prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: workflow expression evaluator silently ignores unknown filters

1 participant