feat: predicate pushdown for WHERE conjunctions on MATCH nodes - #2
Open
villelaitila wants to merge 1 commit into
Open
feat: predicate pushdown for WHERE conjunctions on MATCH nodes#2villelaitila wants to merge 1 commit into
villelaitila wants to merge 1 commit into
Conversation
When a MATCH pattern is constrained by a WHERE clause of the form:
MATCH (a)-[*1..30]->(b)
WHERE a.path = '/foo' AND b.path = '/bar'
RETURN ...
the executor used to enumerate every structural candidate across the
whole graph via DFS and then evaluate the WHERE expression per row in
a pandas DataFrame, discarding almost all of them. On large graphs
with variable-length paths this is extremely slow.
This commit recognises top-level AND conjunctions of the shape
`<variable>.<property> = <literal>` (in either operand order) in the
WHERE AST and folds them into the pattern node's property dict before
the DFS runs. The DFSMatcher already checks per-node property
constraints via node_matches — the pushdown reuses that machinery.
Supported literal types: string, integer, float, and boolean.
Null literals are deliberately not pushed down (Cypher uses
three-valued null-equality semantics that require the post-DFS
evaluator). Unsupported WHERE shapes (OR, NOT, non-equality,
cross-variable comparisons, non-literal RHS) fall through to the
existing post-DFS WHERE evaluation unchanged. Partial extraction
is supported: in a mixed AND, pushable conjuncts are folded while
others remain for the post-DFS filter.
Implementation:
- spycy/predicate_pushdown.py: new module with collect_pushdown_predicates()
and apply_pushdown() — AST walker + property dict merger.
- spycy/spycy.py: 8-line hook in _process_match after pattern property
evaluation, before the DFS call.
- spycy/dfsmatcher.py: node_matches gates on `pnode.id_ in
node_ids_to_props` instead of checking `pnode.properties`, so
pushdown entries are picked up without mutating the pattern AST.
- test/test_predicate_pushdown.py: 27 tests covering happy-path
(string/int/float/bool literals, reversed operands, multi-AND),
fallthrough (OR, NOT, >, <, cross-var, non-literal RHS), partial
AND (one pushable + one non-pushable conjunct), OPTIONAL MATCH,
null equality, and edge cases (inline props + WHERE combined,
unbound variables, escaped strings, missing properties).
No regressions in the openCypher TCK suite.
aneeshdurg
reviewed
Apr 13, 2026
aneeshdurg
left a comment
aneeshdurg
left a comment
Owner
There was a problem hiding this comment.
Thanks for contributing! The actual predicate pushdown implementation and matcher changes seem reasonable.
| from spycy.gen.CypherParser import CypherParser | ||
|
|
||
|
|
||
| PushdownTriple = Tuple[str, str, Any] |
Owner
There was a problem hiding this comment.
Can you make this a dataclass instead?
Comment on lines
+257
to
+265
| .replace('\\\\', '\x00') | ||
| .replace("\\'", "'") | ||
| .replace('\\"', '"') | ||
| .replace('\\n', '\n') | ||
| .replace('\\r', '\r') | ||
| .replace('\\t', '\t') | ||
| .replace('\\b', '\b') | ||
| .replace('\\f', '\f') | ||
| .replace('\x00', '\\')) |
Owner
There was a problem hiding this comment.
I'm not sure I understand why this is needed?
If it really is needed, shouldn't it also be in the expression_evaluator?
| return (var_node.getText(), key.getText()) | ||
|
|
||
|
|
||
| def _try_extract_literal(non_arith_expr) -> Optional[Any]: |
Owner
There was a problem hiding this comment.
This should really be a call to the expression evaluator instead. There's already support for evaluating a literal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WHERE var.prop = literalconjunctions, the executor used to enumerate every structural candidate via DFS and filter post-hoc — extremely slow on large graphs with variable-length pathsDFSMatcher.node_matchesprunes wrong candidates immediatelya.x = 1 AND a.y = b.y), pushable conjuncts are folded, others stay for post-DFSnode_matchesnow gates onpnode.id_ in node_ids_to_propsinstead ofpnode.propertiestruthiness, eliminating the need for sentinel objectsTest plan