Repository navigation
fix(parser): read && and || chains in a loop so a long chain cannot overflow the stack - #174
Merged
Merged
Conversation
…verflow the stack Parse.ChainOperator recursed once for each logical operator, and the ParameterReplacer visitor recursed once for each level of the chain. A flat chain of about 212 clauses overflowed a 256 KB stack and stopped the process. Both layers now read the chain in a loop. Precedence, left associativity, and the expression text do not change. Also correct three CS8603 warnings in the test projects.
…long flat chain QueryKit now parses a flat && or || chain in a loop. EF Core still reads the tree with recursion and overflowed at about 729 clauses on a 1 MB stack, 361 on 512 KB, and 177 on 256 KB. MaxInputLength stays the only limit on a flat chain, and a host with a small stack needs a lower value.
pdevito3
force-pushed
the
fix/iterative-logical-chain
branch
from
October 2, 2026 20:15
0c46ce1 to
2168dea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Precedence, left associativity, and the expression text do not change.
a && b && c || dstill gives((a AndAlso b) AndAlso c) OrElse d. An operator with no operand after it still ends the chain before that operator, soAge > 1 && Age > 2 &&throws the sameParsingExceptionat the same column. The aliases (and,or, and custom operators) go through the same code.v2 behavior (and main)
QueryKit used two recursive layers for a flat
&&or||chain. On a 256 KB thread, a chain of 212 clauses (about 2300 characters) overflowed the stack. A stack overflow stops the process, and acatchblock cannot stop it.New behavior
Flat
Age > 1 && Age > 2 && ...chains, measured in a child process for each case. The value is the largest chain that does not overflow. The||results are the same.FilterParser.ParseFilterParseFilterwith aHasQueryNamemappingCompile(), andApplyQueryKitFilteronIEnumerableApplyQueryKitFilter(...).ToQueryString()With only the parser change, the mapping row stopped at 3956, 8052, and 16244 clauses. The
ParameterReplacerchange removes that limit.QueryKit no longer has a recursive layer for a flat chain. The limits that stay come from outside QueryKit:
LambdaExpression.Compile(LambdaCompiler.EmitBranchOr) recurses once for each level.ExpressionTreeFuncletizer.VisitBinaryrecurses once for each level, and EF Core gets there before QueryKit returns.A balanced tree can remove these limits too, but it changes the expression text. This PR keeps the left-nested tree.
Parse time also gets shorter. On a 64 MB stack, 20000 clauses took 8.5 s before and 3.5 s after. Parse time still grows faster than the clause count.
MaxInputLength
Do not raise the default of 5000 from #135 because of this PR. EF Core overflows at 3967 characters on a 512 KB stack and at 8015 characters on a 1 MB stack, before and after this PR. On a 256 KB stack it overflows at 1943 characters, which is less than the current default. The parser is now not the limit, so a higher default only helps apps that use
IEnumerableor that run on a large stack. This PR does not change the default.The arithmetic operators (
+,-,*,/,%) also useParse.ChainOperator. A long arithmetic chain can still overflow in the parser. A later PR can give them the same loop.Evidence
The active test run was aborted. Reason: Test host process crashed : Stack overflow.With only the parser change, the mapping test crashed the host the same way.qk run(verify-querykit harness, memory and Postgres targets, default configuration on v2):&&and `QueryKitInputLengthExceededExceptionon both targetsOn main without a length limit, the same harness stopped with
Stack overflow.(exit 134) at 5000 clauses, in EF CoreExpressionTreeFuncletizer.VisitBinary.Tests
ParseLimitsTests:long_flat_chain_parses_on_a_small_stack(&&and||): 20000 clauses on a 256 KB thread.long_flat_chain_with_aliases_and_a_query_name_parses_on_a_small_stack: 20000 clauses with theandalias andHasQueryName("age")on a 256 KB thread. It compiles the first and the last clause with the lambda parameter, which shows that the visitor replaced the parameter in the whole chain.flat_chain_keeps_and_before_or_and_groups_from_the_left: pins the expression text of a mixed chain.flat_chain_that_ends_with_an_operator_throws_a_parsing_exception: pins the error column.Three
CS8603warnings in the test projects are also corrected, so the build has 0 warnings.Merge Danger
Door: two-way
The public API, the grammar, and the expression text do not change. A revert puts the recursion back.
Blast Radius: parser
Every filter with
&&or||goes throughChainLeft. All existing filter tests pass on both targets.