Repository navigation
fix(parity): resolve every WHERE operand through one path (P46) - #80
Merged
Merged
Conversation
Re-probing P46 from a second angle showed the cause is wider than column-vs-column: every right-hand operand of a WHERE predicate (the comparison RHS, IN-list items, BETWEEN bounds) went through a literal-only reader that answered NULL for anything else. Six cases, all DIFFER against the current binary, including the NOT (expr < col) shape where P48 leaks into WHERE and returns too many rows. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW
The WHERE evaluator read right-hand operands with a literal-only reader that ended `_ => Ok(Null)`: a column, CASE or expression on the right of a comparison, in an IN list or as a BETWEEN bound read as NULL, so the predicate was UNKNOWN on every row and returned zero rows. A literal on the LEFT was a hard error (`WHERE 1=0`), and an arithmetic operand handed the whole comparison to the ArithmeticEvaluator, which answers false for NULL - so NOT admitted the NULL rows. All operands now go through `evaluate_operand_value` (columns and literals direct, anything else via the ArithmeticEvaluator) and every comparison stays in `compare_trilean`, keeping the NULL -> UNKNOWN rule. `extract_value` and `ExprValue` are gone. Two traps handled on the way: - one ArithmeticEvaluator is now shared per evaluation; building one per row cost 9.8s on 100k rows and made an inline window quadratic (2k rows: 9.5s) - a raw window-function operand is refused, as evaluate_value_as_predicate already did: evaluating it ranks the rows WHERE is still filtering, so QUALIFY + WHERE silently returned the wrong rows (P15 stays GAP, guarded by qualify_inline_with_where) Parity 157 -> 168 AGREE over 197 cases. Three correlated-subquery cases move because outer references now bind to the inner table (P49), accepted and re-bucketed with comments. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW
P46 was wider than filed - every non-literal right-hand WHERE operand read as NULL - and the entry now records the re-probe, the fix, and the two traps it walked into. P49 logs the correlated-subquery trade accepted with it. P48 no longer needs to land with P46 and is projection-only. P15 gains a warning not to fix it inside the WHERE evaluator. R3 gains the catch-all row. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes P46, which turned out wider than it was filed, plus the
WHERE 1=0rejection seen in the countries demo.What was wrong
RecursiveWhereEvaluatorread right-hand operands with a literal-only reader that ended_ => Ok(Null). Any column,CASEor expression on the right of a comparison, in anINlist or as aBETWEENbound read as NULL, so the predicate was UNKNOWN on every row.id < partner_id,team = teamNOT (id < partner_id)score < CASE WHEN … ENDpartner_id BETWEEN id AND 10partner_id IN (id + 2, 5)NOT (id + 0 < partner_id)3 < id,WHERE 1=0The last two rows are two more holes in the same function. An arithmetic operand sent the whole comparison to the
ArithmeticEvaluator, which answersfalsefor NULL, so P48 leaked in underNOT. And the left operand had to be a column.JOIN … ONandHAVINGwere probed and are clean.The fix
Every operand now resolves through
evaluate_operand_value. Columns and literals are read directly; anything else goes to theArithmeticEvaluator. Every comparison stays incompare_trilean, so R10's NULL → UNKNOWN rule holds whatever shape the operands take.extract_valueandExprValueare deleted. As a result P48 no longer needs to land with this, and is projection-only.Two traps, handled before committing:
ArithmeticEvaluator::newbuilds every registry, and the old code built one per row. An arithmetic operand over 100k rows took 9.8 s; an inline window over 2k rows took 9.5 s (quadratic). One shared evaluator per WHERE evaluation brings those to 2.9 s and 60 ms.qualify_row_numberwent AGREE. But the window was computed over the rows WHERE is still filtering, so QUALIFY plus WHERE returned 1 row where DuckDB returns 3. A raw window operand is now refused, the ruleevaluate_value_as_predicatealready applied.qualify_inline_with_where(GAP) guards it.Accepted trade: P49
With right-hand columns resolved, correlated outer references (
x.region = s.region) bind to the inner table's column through the resolver's existing unqualified fallback. Two correlated scalar-subquery cases go from GAP to DIFFER, andin_subquery_correlatedagrees by coincidence. Filed as P49 with the options considered. The proper fix comes with P3.Corpus and tests
where_literal_on_left,where_constant_trueandqualify_inline_with_where.--checkpasses.operand_resolution_tests: six per-rowTrileanassertions, including NOT-over-NULL and the refused window operand.expander_rewritersquery 7, P26 shape) fails identically on main.Docs
P46 closed with the full diagnosis. P49 filed. P48 decoupled. P15 now warns against fixing it inside the WHERE evaluator. R3 gains the catch-all row. The fix-order table puts P47 next.
This work also showed WHERE and the value evaluator each implement every boolean operator, with different NULL semantics. That becomes R13 in a follow-up.
🤖 Generated with Claude Code
https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW