Skip to content

refactor(r13): WHERE IS NULL and LIKE delegate; closes P55 (R13 slice 4, part 3) - #91

Merged
TimelordUK merged 8 commits into
mainfrom
refactor/r13-slice4c-like-is-null
Sep 19, 2026
Merged

TimelordUK merged 8 commits into
mainfrom
refactor/r13-slice4c-like-is-null

Conversation

@TimelordUK

Copy link
Copy Markdown
Owner

R13 slice 4, third part. WHERE's IS NULL, IS NOT NULL and LIKE now go through the value evaluator, so every WHERE operator now delegates. What's left of slice 4 (AND / OR / NOT / CASE) waits on slice 5, method calls.

Closes P55: a LIKE pattern in WHERE was read as a regex

WHERE built its matcher by replacing % and _ and escaping nothing else, so any regex syntax in a pattern took effect:

  • LIKE 'a.c' matched abc too (so LIKE '%.csv' matches xcsv)
  • LIKE '[a]bc' matched abc but not the literal [a]bc
  • LIKE 'a(b%' failed the whole query: regex parse error … unclosed group
  • % stopped at a newline

The bug was in both the cached path (EvaluationContext) and the uncached fallback.

Commits

  1. IS NULL / IS NOT NULL delegate. A true no-op: the parser always puts a Null literal on the right. Also deletes the IS, IS NOT and == operator arms, which the parser never produces.
  2. Pin LIKE. A fourth DuckDB-generated matrix set (EXPECTED_LIKE), which records six wrong WHERE answers. Three corpus cases over new data/like_patterns.csv pin it end to end (DIFFER, DIFFER, GAP).
  3. Fix the exponential LIKE matcher. The value evaluator's matcher was right on all ten patterns but recursive: one 80-character value against '%a%a%a%a%a%a%a%b' ran for over a minute (SELECT has had this all along). Replaced with the iterative wildcard algorithm, O(text × pattern), before WHERE switched to it. Adds 28 unit cases plus a 2,000-character case with a 1 s budget; the CLI case now takes 57 ms.
  4. LIKE over a number: D2. DuckDB rejects it, so there's no reference answer. Decision: match the number's displayed text, as the value evaluator already did (5.0 displays as 5). Recorded in SQL_PARITY.md as a deliberate difference from DuckDB. Pinned as a fifth matrix set whose expected answers are this decision; WHERE's old "always FALSE" is recorded against it.
  5. Delegate LIKE. All of LIKE (NULL handling, the text of a number, case mode, matching) is now one function, sql_like, called by the value evaluator and by WHERE's one remaining legacy arm (a method call on the left). Exactly the ten pinned WHERE answers changed, and parity moved by exactly the three P55 cases: 176 → 179 AGREE / 209.
  6. Remove the plumbing. Deletes EvaluationContext, WHERE's 'ctx lifetime and context constructors, with_config (no callers) and the always-zero cache stats. 147 lines out, parity byte-identical.
  7. Docs: R13 entry, P55 and D2 entries, headline counts.

Verification

  • Parity: before and after reports compared at each commit. Nothing moved except the pinned cases; contract holds at 209.
  • Evaluator matrix green, all six sets.
  • ./run_all_tests.sh green: 830 lib + 501 integration Rust tests, 572 Python tests, all FORMAL examples. The one SMOKE failure (expander_rewriters query 7) also fails on main.
  • Over trades_100000.csv, WHERE IS NULL and both LIKE shapes stay at CSV load time without the regex cache.

Next: slice 5, method calls

WHERE's own Contains, StartsWith and EndsWith always ignore case and answer FALSE for NULL, and it has its own left-side Length, IndexOf and Trim*. These need pinning against the registry functions the value evaluator maps them to, then moving over.

🤖 Generated with Claude Code

TimelordUK and others added 8 commits September 19, 2026 10:15
…aluator (R13 slice 4)

Both NULL tests join the comparisons in `delegates_to_value_evaluator` (renamed
from `is_comparison_operator`). A genuine no-op: the parser always puts a
`Null` literal on the right, and a NULL test only asks whether the left
operand is NULL, which no difference between the two operand readers can
change. The legacy method-call-on-the-left exception still applies.

Deleted WHERE's bare `IS` / `IS NOT` arms, and dropped `==`, `IS` and
`IS NOT` from `is_predicate_operator`: the parser never emits them, and an
operator nothing produces should fail loudly rather than be half-handled.

Parity report byte-identical, matrix green (all three sets); WHERE
comments IS NULL over trades_100000.csv unchanged at load time.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
A fourth matrix, generated by DuckDB, aimed at what separates the two LIKE
matchers: pattern characters that are regex syntax, and text across a
newline. The value evaluator gets all ten right. WHERE gets six wrong:

P55 (filed): WHERE compiles a LIKE pattern to a regex by replacing `%` and
`_` and escaping nothing else. `LIKE 'a.c'` matches `abc`, `LIKE '[a]bc'`
matches `abc` but not `[a]bc`, `LIKE 'a(b%'` fails the whole query with a
regex parse error, and `%` does not cross a newline. Same bug in the cached
path every query takes (EvaluationContext) and the uncached fallback.

Three corpus cases pin it end to end over new data/like_patterns.csv:
where_like_dot_is_literal, where_like_bracket_is_literal (DIFFER),
where_like_paren_is_literal (GAP). Newline shapes stay matrix-only.

Parity 176 AGREE / 209, contract holds.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The value evaluator's LIKE matched recursively, trying every split for every
`%`. One value against `'%a%a%a%a%a%a%a%b'`: 60 characters took 7 s, 80
characters over a minute. It already serves SELECT, HAVING and CASE, and
WHERE is about to delegate LIKE to it - where today's regex, whatever its
other faults (P55), never backtracks like this.

Replaced with the iterative wildcard algorithm: on a failed literal after a
`%`, only the most recent `%` absorbs one more character. Sufficient because
a later `%` can take whatever an earlier one would have, so the worst case is
O(text x pattern). Semantics unchanged: `%` and `_` the only wildcards, `%`
spans newlines, `_` is one char.

28 unit cases plus a 2,000-character pathological one with a 1 s budget.
Via the CLI the 80-character case went from >60 s to 57 ms (process start).
Matrix green (all four sets), parity report byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
… (R13 slice 4)

DuckDB rejects LIKE on a number, so there is no reference answer. Decision
(recorded as D2 in docs/SQL_PARITY.md): a number is matched as the text
sql-cli displays for it - what the value evaluator already does in SELECT,
HAVING and CASE. `5.0` displays as `5`, so it matches '5%' but not '5.0'.

A fifth matrix set, over the operand table, states that decision as the
expected answer. WHERE answers FALSE for every non-text operand today: four
entries recorded, to flip when LIKE delegates.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…tors (R13 slice 4)

LIKE joins the comparisons and NULL tests in `delegates_to_value_evaluator`.
The whole of LIKE - NULL handling, the text a non-text operand is matched as
(D2), case mode and matching - is now one function, `sql_like`, which the
value evaluator's arm and WHERE's remaining legacy arm (a method call on the
left) both call. WHERE's regex construction, cached and uncached, is gone.

Closes P55: the six LIKE-matrix entries and four number-LIKE entries went
FIXED and nothing else moved; the three corpus cases moved to AGREE, parity
176 -> 179 AGREE over 209. WHERE LIKE over trades_100000.csv unchanged at
load time without the regex cache.

Leaves `RecursiveWhereEvaluator::context` unread (it only carried the regex
cache); the next commit removes that plumbing.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…e 4)

With LIKE delegated, `EvaluationContext` carried a regex cache nothing read
and a case-insensitive flag already passed in directly. Removed:

- `data::evaluation_context` (the module, 69 lines);
- `RecursiveWhereEvaluator`'s `context` field and `'ctx` lifetime, and the
  `with_context` / `with_both_contexts` constructors that took one;
- `with_config`, which had no callers and an ignored date-notation argument
  (the same dead plumbing slice 3 removed from the value evaluator);
- the query engine's cache-statistics logging, now always zero.

The query engine builds the WHERE evaluator with `with_exec_context` or
`with_case_insensitive`. No-op: parity report byte-identical, 830 lib + 501
integration tests pass. 147 lines out, 13 in.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The P55 corpus cases read data/like_patterns.csv, which `.gitignore`'s
benchmark-output rule `like_*.csv` matched - so `git add -A` skipped it and
the file never reached the branch. Locally the cases passed against the
untracked copy; in CI both engines failed to find the file, and all three
cases landed in BOTH_ERR instead of AGREE (run 35435295922).

Renamed to data/pattern_matching.csv, which no ignore rule matches, rather
than force-adding a file the rules would keep skipping. Checked the other six
corpus data files are all tracked.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@TimelordUK
TimelordUK merged commit 6a3c5f8 into main Sep 19, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant