Skip to content

refactor(r13): WHERE comparisons delegate to the value evaluator; closes P54 (R13 slice 4, part 2) - #90

Merged
TimelordUK merged 4 commits into
mainfrom
refactor/r13-slice4b-comparisons
Sep 18, 2026
Merged

TimelordUK merged 4 commits into
mainfrom
refactor/r13-slice4b-comparisons

Conversation

@TimelordUK

Copy link
Copy Markdown
Owner

R13 slice 4, second part: WHERE's comparison operators (=, <>, !=, <, <=, >, >=) now go through the value evaluator and are converted back to a WHERE truth value once, in evaluate_delegated. Comparisons with a legacy method call on the left (Length(), IndexOf(), Trim*()) keep WHERE's path until slice 5.

Closes P54: a large integer literal matched the wrong row in WHERE

WHERE read every number literal through f64, which can't hold every integer above 2^53. So WHERE id = 9007199254740993 returned the row with id 9007199254740992, a different record, not an empty result. On that same row SELECT id = 9007199254740993 said false, so the query contradicted itself. The value evaluator reads i64 first and was already right.

Commits

  1. Pins. A new DuckDB-generated matrix (EXPECTED_OPERANDS) for the cases where the two evaluators turn a literal into different values. It records four wrong WHERE answers (three for P54, plus '5.0' = 5.0). Also adds the corpus case where_big_int_literal_equals (DIFFER) with an IN control, and data/big_ids.csv. Files P53: '5.0' = 5 compares as text in both evaluators, where DuckDB casts the string to a number. It's low priority and has no corpus case; the entry explains why.
  2. DATETIME() in local time. WHERE took today's date from local time, the value evaluator from UTC, so they disagreed between 00:00 and 01:00 under BST. WHERE was right (DuckDB's current_date uses the session time zone, local by default), so the value evaluator changed before the switch-over. This is the one step without a failing test first: the two only disagree during that hour.
  3. Switch over. Exactly the four pinned answers changed; the P53 pair and every other matrix entry held. Parity moved by exactly one case: 175 → 176 AGREE / 206. evaluate_binary_op now receives the whole expression node, so nothing rebuilds and clones it on each row.
  4. Docs. R13 entry, P53/P54 entries, and updated fix-order table and headline counts.

Things this turned up

  • Part 1 (refactor(r13): WHERE BETWEEN / IN delegate to the value evaluator (R13 slice 4, part 1) #87) wasn't quite a no-op. Routing IN and BETWEEN through the value evaluator also switched them to its literal reader, which fixed P54 for those two operators. The parity report couldn't show it because no case held an integer that large.
  • Harness blind spot. The harness compares numbers as floats, so both ids render as 9007199254740992.0 and only the name column showed the wrong row. A case that selects only a large integer can't tell two ids apart. This is noted in the corpus case and in P54.

Verification

  • Parity --check holds (206 cases). Reports compared against a saved baseline at each commit: the only change was the P54 case.
  • Evaluator matrix green, all three sets.
  • ./run_all_tests.sh green: Rust, 572 Python tests, all FORMAL examples. The one SMOKE failure (expander_rewriters query 7) also fails on main.
  • Over trades_100000.csv (~2.4 s of which is loading the CSV): WHERE price + 0 > quantity 3.0 s → 2.5 s; the other WHERE shapes are unchanged.

Next (slice 4 part 3)

LIKE: WHERE uses a regex cache, and the two evaluators disagree on a non-string operand (5.0 LIKE '5%' is FALSE in WHERE, TRUE in the value evaluator, an error in DuckDB), so that needs a decision first. Then IS NULL. AND/OR/NOT/CASE wait on slice 5.

🤖 Generated with Claude Code

TimelordUK and others added 4 commits September 18, 2026 18:48
…lice 4)

WHERE and the value evaluator each turn a literal into a value with their own
reader. A third matrix, generated by DuckDB, runs the comparison shapes where
the two readers can disagree through both evaluators:

- P54 (filed): WHERE reads a number literal through f64, so an integer beyond
  2^53 is rounded onto its neighbour. `WHERE id = 9007199254740993` returns
  the row with id ...992. Three WHERE entries recorded. Corpus case
  where_big_int_literal_equals pins it end to end (DIFFER).
- `s = 5.0` against the string '5.0': WHERE reads the literal as Integer 5
  and compares as text. One WHERE entry.
- P53 (filed, low priority): `'5.0' = 5` compares as text in both evaluators;
  DuckDB casts. Recorded for both; no corpus case (explained in the entry).

IN and BETWEEN already answer correctly: delegating them in part 1 fixed P54
for those operators without anyone noticing. Control case
where_big_int_literal_in_list AGREEs.

check_matrix now takes its table. Parity 175 AGREE / 206, contract holds.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…o (R13 slice 4)

WHERE's operand reader took today's date from `Local`, the value evaluator
from `Utc`. They differ for the hours where the local and UTC dates differ -
00:00-01:00 under BST - so `SELECT d >= DATETIME()` and `WHERE d >=
DATETIME()` compared against different days. DuckDB's `current_date` uses the
session time zone, local by default, so WHERE had it right.

Lands before the comparison arms delegate: otherwise delegating would have
moved WHERE onto the UTC date.

Not pinned first, unlike the rest of the slice: the divergence exists only
while the two dates differ, so a test could not fail on demand without
injecting a clock. Parity report byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…13 slice 4)

`=`, `<>`, `!=`, `<`, `<=`, `>`, `>=` in WHERE now evaluate through
`ArithmeticEvaluator` and cross back in `evaluate_delegated`. The raw window
check stays on WHERE's side. Two shapes keep WHERE's path for now: a legacy
method call on the left (`Length()`, `IndexOf()`, `Trim*()`), whose registry
twins are slice 5's, and the operators with their own arms (LIKE, IS NULL).

`evaluate_binary_op` now receives the node as well as its parts, so neither
the delegation nor the existing P37 value-as-predicate branch rebuilds (and
clones) the expression per row.

Closes P54: the four operand-matrix entries recorded by the pin commit went
FIXED and nothing else moved; `where_big_int_literal_equals` DIFFER -> AGREE,
parity 175 -> 176 AGREE over 206. Over trades_100000.csv,
`WHERE price + 0 > quantity` 3.0 s -> 2.5 s (load time); other WHERE shapes
unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@TimelordUK
TimelordUK merged commit 61f1d58 into main Sep 18, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant