diff --git a/docs/SQL_PARITY.md b/docs/SQL_PARITY.md index 1b46e35d..f646ae7f 100644 --- a/docs/SQL_PARITY.md +++ b/docs/SQL_PARITY.md @@ -65,9 +65,9 @@ session apiece to fix properly, so **discovery is paused and the effort moves to picking them off**. Widen the corpus again when the open list is short, or opportunistically when a fix needs a case that doesn't exist yet. -Corpus coverage today: tiers 01–10, **181 cases** (156 AGREE / 10 DIFFER / -12 GAP / 1 OURS_ONLY / 2 BOTH_ERR as of 2026-09-06, after [P41](#p41) added four -MODE cases). The largest single movement so far remains the 2026-09-05 +Corpus coverage today: tiers 01–10, **188 cases** (157 AGREE / 16 DIFFER / +12 GAP / 1 OURS_ONLY / 2 BOTH_ERR as of 2026-09-12, after [P46](#p46)–[P48](#p48) +added seven — six DIFFER and one AGREE anchor). The largest single movement so far remains the 2026-09-05 NULL-ordering slice, which closed [P13](#p13) stage 2 and [P17](#p17) together — eleven cases in one change. **Tier 10 (aggregate & NULL edges) is still deliberately partial** — it holds the P14, P18–P20 and P41 cases and their @@ -91,7 +91,8 @@ Suggested fix order, by silent blast radius: | ~~9c~~ | ~~[P17](#p17) + [P13](#p13) stage 2 — NULL ordering~~ | ✅ **Fixed 2026-09-05** — 141 → **152 AGREE**, eleven cases in one change. Both halves were the same comparator's NULL rule, so they were taken as one slice. The two sorts that disagreed with each other now *share* one function (`compare_for_order_by`), which is the part that stops the divergence recurring; the window site turned out to be sorting NULL as the **maximum** via a derived `PartialOrd`, not merely following a different rule | | ~~9d~~ | ~~[P37](#p37) window in `WHERE` returns 0 rows~~ | ✅ **Fixed 2026-09-05** — corpus count unchanged, and that is the finding: the case is `OURS_ONLY` before *and* after, so the harness cannot see this fix or a future regression of it (first entry of that kind — the regression test is a Rust module). The filed root cause was wrong: `ExpressionLifter` *does* lift from `WHERE`. The real defect was one arm in the WHERE evaluator answering FALSE for any bare value used as a predicate — `WHERE true` returned zero rows too. It did **not** close [P15](#p15), which needs the opposite change | | ~~9e~~ | ~~[P41](#p41) `MODE` tie-break is random per run~~ | ✅ **Fixed 2026-09-06** — 152 → **156 AGREE** (four new cases). Small, as predicted, but not where it was filed: the named `ModeState` was a *shadowed* implementation and fixing it moved nothing. Reference does specify a rule and it is **first-occurrence**, not the "smallest value wins" this row proposed. Unblocked both example files, now FORMAL. Spun off [P42](#p42), [P43](#p43), [R12](ENGINE_REFACTORING.md#r12) | -| **NEXT** | [P14](#p14), [P20](#p20), [P23](#p23) | Smaller, self-contained, decisions already taken. Was row 9b | +| **NEXT** | [P46](#p46) column-vs-column `WHERE` returns 0 rows, [P47](#p47) `MIN`/`MAX` ranked by type, [P48](#p48) NULL comparison projects `false` | All three found 2026-09-12 writing one examples file, all three silent. P46 is the [P30](#p30) disguise again — zero rows reads as "no data" — and P47 hands back a number wrong by a factor of half a million from an ordinary `GROUP BY`. Both are cheap: P47's correct comparator is already in the tree, and P48 is the [P18](#p18)/[P19](#p19) rule at one more site. P46 and P48 sit in neighbouring code, so take them together | +| then | [P14](#p14), [P20](#p20), [P23](#p23) | Smaller, self-contained, decisions already taken. Was row 9b, then NEXT until the 2026-09-12 findings displaced it | | 9f | [P42](#p42) `MODE` is numeric-only | Companion to [R12](ENGINE_REFACTORING.md#r12), and cheap if taken with it: the shadowed implementation already handles non-numerics and preserves type, so the fix is largely to stop the live path throwing away what it knows. Also buys the corpus its clearest tie-break case | | 10 | [P22](#p22), [P25](#p25), [P26](#p26), [P15](#p15), [P32](#p32), [P38](#p38) | Hard errors — visible, so less urgent than any of the above | | 10b | [P39](#p39) `x/0` errors, voiding the whole statement | Hard error like row 10, but the only one whose blast radius is the *query* rather than the cell. Settle the four inconsistent call sites as one decision; it currently has no live probe (see the entry) | @@ -118,6 +119,16 @@ the fix; the entry describes the symptom that was looked for, not necessarily th defect. The same session's sweep for other consumers of the same route turned up [P31](#p31), a live zero-rows bug in the `--limit` flag that nobody had reported. +**A fourth, from P46–P48 (2026-09-12): writing examples is discovery.** Those +three came out of one afternoon writing `examples/countries.sql` — not from the +corpus, not from a fix, but from asking a real dataset ordinary questions +("which borders cross a continent", "largest country per region"). Two were +silent wrong answers in shapes the corpus did not cover, and the corpus grew by +seven cases as a result. The examples directory is cheap discovery surface and +worth mining deliberately when the open list runs short: unlike the corpus, it +is written to be *interesting* rather than to be exhaustive, which is exactly how +it wanders outside the tested envelope. + **A fourth lesson, from closing P41 (2026-09-06): the entry named a fix site, and the fix site was dead code.** `ModeState` in `src/sql/aggregates/mod.rs` is exactly what you find by grepping for MODE, and it is shadowed — a second, newer @@ -1762,8 +1773,33 @@ out of date. emits `value,index`, i.e. `WITH ORDINALITY` for free, which is worth keeping. - **What works today:** `SPLIT_PART(path,'/',n)` — scalar, resolves columns - normally, fine for fixed positions. There is no way to handle - arbitrary-depth paths. + normally, fine for fixed positions. + +- **A *bounded* explode also works today, and this entry undersold it + (added 2026-09-12).** `CROSS JOIN` a table of slot numbers onto the rows and + take one part per slot: + + ```sql + WITH slots AS (SELECT value AS n FROM RANGE(1, 20)), + c AS (SELECT * FROM #countries) + SELECT cca3, SPLIT_PART(borders, ',', n) AS neighbour + FROM c + CROSS JOIN slots + WHERE borders <> '' AND SPLIT_PART(borders, ',', n) <> ''; + ``` + + That turns 250 country rows into 649 border pairs. Nothing here needs a + per-row generator call — the generator runs **once**, for the slot numbers, + and the *scalar* `SPLIT_PART` does the per-row work, so it sidesteps both + defects above rather than working around them. `examples/countries.sql` uses + it twice (`borders`, then `languages`) and it is the form to reach for until + piece 3 lands. + + Two costs, which are why this is a workaround and not the feature: the bound + is hand-picked (20 slots for a real maximum of 16 — too few silently truncates, + too many just costs a scan), and the source is scanned once per slot. But + "there is no way to handle arbitrary-depth paths" overstated the gap: any + depth you are willing to bound is reachable today. - **Found:** 2026-09-04, from real use rather than the corpus — a TeamCity API response piped into `READ_JSON('-', …)`, wrapped in a CTE to extract an @@ -1969,6 +2005,7 @@ out of date. |---|---|---| | `SELECT * FROM range(1,3)` | 1,2,3 ✅ | ✅ | | `SELECT * FROM t JOIN range(1,3) ON t.id = value` | **Parse error: Expected ON keyword after JOIN table** 🚫 | joins | + | `SELECT * FROM t CROSS JOIN range(1,3)` | **Parse error: Unexpected '(' after end of statement** 🚫 | joins | | `WITH a AS (SELECT * FROM range(1,3)), b AS (...) SELECT * FROM a JOIN b ON a.value = b.value` | ✅ | ✅ | - **Found:** 2026-09-12, designing the system tables @@ -1980,7 +2017,12 @@ out of date. inconvenience. A family of system tables is specifically meant to be joined to one another, so this becomes the common case rather than an edge. - **Workaround, and it is a decent one:** wrap each side in a CTE, which parses - and executes correctly today. `SYSTEM_TABLES.md` documents that form. + and executes correctly today. `SYSTEM_TABLES.md` documents that form, and + `examples/countries.sql` leans on it — the bounded explode in + [P40](#p40) is `CROSS JOIN RANGE(1, 20)` with the generator wrapped in a CTE + for exactly this reason. Note the `CROSS JOIN` spelling fails with a *different* + message (the parser stops at the `(` and reports trailing tokens), which is + the same grammar hole seen from the other side. - **Decision:** fix in the parser — the joined source should accept the same productions as the `FROM` source, which is the general shape of the bug: two places that should parse "a table source" and only one of them knows the full @@ -1989,6 +2031,164 @@ out of date. --- +### P46 — A column-to-column comparison in `WHERE` returns zero rows +- **Status:** 🔴 OPEN — silent, and it returns the emptiest possible wrong answer +- **Corpus:** `02_where.toml :: where_col_lt_col`, `where_col_eq_col_string` + (both `expect = "DIFFER"`); the contrast anchor is + `01_select.toml :: select_col_vs_col_projected`, which AGREEs. +- **Observed:** if both sides of a comparison are columns, the predicate never + matches. Not one operator, not one type — the shape: + + | Query | Ours | DuckDB 1.5.5 | + |---|---|---| + | `WHERE id < partner_id` (`null_edges.csv`) | **0 rows** 🚫 | ids 1, 4, 7, 9 | + | `WHERE currency = trading_currency` (`instruments.csv`) | **0 rows** 🚫 | 57 of 200 | + | `WHERE bid_price > ask_price` (`instruments.csv`) | **0 rows** 🚫 | 108 of 200 | + | `WHERE region = region` — *the same column on both sides* | **0 rows** 🚫 | every row | + | `SELECT id < partner_id AS lt` — same comparison, projected | ✅ correct | ✅ | + + The `region = region` row is the tell: no data, no type, no NULL and no + coercion is involved. Any `Column op Column` predicate is false. +- **Not the parser.** `--query-plan` shows the WHERE clause as + `BinaryOp { left: Column(region), op: "=", right: Column(subregion) }` — the + AST is exactly right, so this lives in the WHERE evaluator. And the same + comparison evaluated in the `SELECT` list is correct, which localises it + further: comparison works, *this site's* comparison does not. +- **Found:** 2026-09-12, writing `examples/countries.sql`. "Which land borders + cross a continent boundary" is `WHERE region <> nb_region` after joining the + border pairs back to the countries, and it returned nothing. The shape is + ordinary in joined or staged data: bid against ask, claimed against settled, + this row's region against its neighbour's. +- **Why it bites:** zero rows reads as *no such data*, the same disguise that + made [P30](#p30) dangerous — and the natural next move is to loosen the query + rather than to doubt the engine. It was only caught here because the answer was + known in advance (Russia borders China, so the result cannot be empty). +- **Workaround, and it is the one `examples/countries.sql` uses:** do the + comparison in the `SELECT` list, where it works, and filter on *that* column + downstream — `CASE WHEN a = b THEN 'internal' ELSE 'cross' END AS frontier` + into a CTE or `#temp`, then `WHERE frontier = 'cross'`. Arithmetic serves the + same purpose when the columns are numeric: project + `COUNT('*') - SUM(flag) AS remainder`, then `WHERE remainder = 0`. +- **Decision:** fix. While in there, check the other predicate sites for the + same hole — `JOIN ... ON` and `HAVING` both compare operands of their own — + and re-probe from a second angle first, per the [P28](#p28) lesson: this entry + was written from `WHERE` alone. +- **Related:** [P30](#p30) (same zero-rows disguise), [P7](#p7) (a + multi-condition join evaluating two-column operands by position — possibly the + same root, worth checking together), [P48](#p48) (the NULL half of the same + comparison story, found while pinning this one). + +--- + +### P47 — `MIN`/`MAX` compare by type variant before value, so one float outranks every integer +- **Status:** 🔴 OPEN — silent, and wrong by orders of magnitude +- **Corpus:** `10_aggregate_nulls.toml :: max_mixed_int_float`, + `min_mixed_int_float`, `max_mixed_int_float_from_csv` (all `expect = "DIFFER"`). +- **Observed:** in a column holding both `Integer` and `Float` values, `MAX` + returns the largest *float* and `MIN` the smallest *integer*, whatever the + actual numbers are: + + | Query | Ours | DuckDB 1.5.5 | + |---|---|---| + | `MAX(v)` over `1, 2.5, 10` | **2.5** 🚫 | 10 | + | `MIN(v)` over `5, 2.5, 10` | **5** 🚫 | 2.5 | + | `MAX(area)` over `countries.csv` | **34.2** 🚫 | 17098242 | + | `MAX(CAST(area AS FLOAT))` | 17098242 ✅ | ✅ | + | `ORDER BY area DESC LIMIT 1` | Russia ✅ | ✅ | + | `MAX(v) OVER (PARTITION BY g)` | 10 ✅ | 10 | + | the same window with `SQL_CLI_BATCH_WINDOW=0` | **2.5** 🚫 | 10 | + + The third row is the one to feel: the largest country in the world, reported + as a 34 km² scattering of Pacific islets, by a `GROUP BY` that looks entirely + ordinary. +- **Root cause, and it is small.** `DataValue` derives `PartialOrd` + (`src/data/datatable.rs:204`), so a comparison between two *different* variants + is decided by the variant order in the enum declaration — + `String < InternedString < Integer < Float < Boolean < DateTime < Vector < Null` + — before any value is looked at. `MinMaxState` then compares with bare `<`/`>`: + `src/sql/aggregate_functions/mod.rs:439` in the current registry, + `src/sql/aggregates/mod.rs:180` in the older one. So every `Float` beats every + `Integer` for `MAX`, and every `Integer` beats every `Float` for `MIN`. +- **The correct comparator is already in the tree.** `compare_datavalues` + (`src/data/datavalue_compare.rs`) has explicit `Integer`↔`Float` arms that + compare `*i as f64` against the float — which is why `ORDER BY` and the batched + window path get this right while the aggregate does not. Same shape as + [P17](#p17): two sites that should share one rule, and only one of them knows + it. +- **A performance flag changes the answer.** The row-wise window path + (`src/sql/window_functions/aggregates.rs:198,259`) carries the same bare + comparison, so `SQL_CLI_BATCH_WINDOW=0` — documented in `CLAUDE.md` as an + opt-out for a *speed* optimisation — silently switches window `MAX` from right + to wrong. Whatever else the fix does, it should leave the two window paths + agreeing. +- **How a plain CSV ends up with a mixed column** (this is why the bug is + reachable without literals): type inference samples early rows. In + `countries.csv` only three of 250 areas carry decimals — Monaco 2.02, US Minor + Outlying Islands 34.2, Vatican City 0.44 — and they sit at rows 140, 233 and + 237, so the column was typed `Integer` and those three stayed `Float`. +- **Wider than numbers.** The same variant order puts every `Boolean` above + every `Float` and every `String` below every number, so `MIN`/`MAX` over any + mixed-type column ranks by type first. NULLs are not affected — the + accumulator skips them explicitly before comparing. +- **Found:** 2026-09-12, writing `examples/countries.sql` — a per-region + `MAX(area)` gave Europe 2.02 (Monaco) and the Americas 34.2, which is the kind + of wrong that only stands out because the right answer happens to be famous. +- **Decision:** fix by routing both `MinMaxState` implementations through + `compare_datavalues`, and sweep the other bare `DataValue` comparisons while + there — `src/data/arithmetic_evaluator.rs:1272,1311` and the two window sites + above are the same code shape. **Confirm which implementation is live before + changing it** ([P41](#p41)'s lesson: the named one was a shadow, and fixing it + moved nothing); there are two registries plus a third MIN/MAX path in the + evaluator. +- **Workaround until then:** `CAST(col AS FLOAT)` inside the aggregate, which is + what `examples/countries.sql` does, with the reason in a comment so it does not + read as noise. +- **Related:** [P17](#p17) (a derived `PartialOrd` at the window ORDER BY site — + same root class), [P41](#p41)/[R12](ENGINE_REFACTORING.md#r12) (shadowed + aggregate implementations), [P42](#p42) (`MODE`'s type handling). + +--- + +### P48 — A comparison with a NULL operand projects `false` instead of NULL +- **Status:** 🔴 OPEN — silent, and it disagrees with our own `WHERE` evaluator +- **Corpus:** `01_select.toml :: select_col_vs_col_null_operand` + (`expect = "DIFFER"`). +- **Observed:** a projected comparison coerces a NULL operand to a definite + answer: + + | Query (`null_edges.csv`, row 3, where `partner_id` is NULL) | Ours | DuckDB 1.5.5 | + |---|---|---| + | `SELECT id < partner_id AS lt` | **false** 🚫 | NULL | + | `SELECT id = partner_id AS eq` | **false** 🚫 | NULL | + | `CASE WHEN id < partner_id THEN 'y' ELSE 'n' END` | 'n' ✅ | 'n' ✅ | + | `WHERE id < partner_id` (for contrast) | row excluded ✅ | excluded ✅ | + + The `CASE` row is correct in both engines and for the same reason — UNKNOWN + takes the `ELSE` branch — so the defect is specifically the *value* a + comparison yields when it is projected rather than branched on. +- **This is the [P18](#p18)/[P19](#p19) rule at a site that fix did not reach.** + That change (2026-08-22) gave the WHERE evaluator three-valued logic; a + projected comparison goes through the arithmetic evaluator instead, which still + returns a definite `false`. So the same predicate means two different things + depending on where it is written — and a flag projected in one statement then + filtered in the next (the [P46](#p46) workaround, a pattern the examples now + use deliberately) quietly turns "unknown" into "no". +- **Found:** 2026-09-12, and worth a method note: this case was added as the + *anchor* for [P46](#p46) — the control that was supposed to AGREE and so prove + the comparison evaluator itself was fine. It came back DIFFER for an unrelated + reason. A case added to localise one bug found a second; the anchor was then + narrowed with `WHERE partner_id IS NOT NULL` so it can still do its original + job, and the unguarded form kept as this entry's case. +- **Decision:** fix — the comparison arms of the arithmetic evaluator should + yield NULL when either operand is NULL, matching the WHERE evaluator rather + than diverging from it. Small, and it wants doing alongside [P46](#p46), which + is in neighbouring code. +- **Related:** [P18](#p18), [P19](#p19), [P20](#p20) (`||` treating NULL as an + empty string — the same coercion instinct in the string operators), + [P46](#p46). + +--- + ## Deferred / won't fix (intentional) ### D1 — Recursive CTEs (`WITH RECURSIVE`) diff --git a/examples/countries.sql b/examples/countries.sql index bddb8a9b..fd28b66f 100644 --- a/examples/countries.sql +++ b/examples/countries.sql @@ -1,4 +1,6 @@ +-- stage all countries in a tmp table + WITH countries AS ( SELECT * @@ -10,6 +12,8 @@ INTO #countries FROM countries; go +-- select single country - note the borders + SELECT "name.common" AS name, region, @@ -25,6 +29,8 @@ FROM #countries WHERE "name.common" = 'Russia'; go +-- for Russia inflate borders join back to countries + WITH all AS ( SELECT * @@ -51,11 +57,328 @@ INNER JOIN codes ON all.cca3 = codes.code ORDER BY name ASC; GO +-- counts within each region + SELECT region, + landlocked, COUNT('*') AS n FROM #countries -GROUP BY region -ORDER BY n desc, region ASC; +GROUP BY region,landlocked +ORDER BY region, landlocked, n desc; +GO + + +-- --------------------------------------------------------------------------- +-- borders is a comma-packed list, so unpack it for EVERY country at once: +-- CROSS JOIN a slot number onto each row and pull the nth item out with +-- SPLIT_PART. 20 slots is enough for China, the record holder with 16. +-- This is the generic version of the hand-typed SPLIT list above. +-- --------------------------------------------------------------------------- + +WITH + slots AS ( + SELECT value AS n + FROM RANGE(1, 20) + ), + c AS ( + SELECT * + FROM #countries + ) +SELECT + cca3 AS iso, + "name.common" AS name, + region, + landlocked, + CAST(SPLIT_PART(latlng, ',', 1) AS FLOAT) AS lat, + CAST(SPLIT_PART(latlng, ',', 2) AS FLOAT) AS lng, + SPLIT_PART(borders, ',', n) AS neighbour_iso +INTO #border_pairs +FROM c +CROSS JOIN slots +WHERE borders <> '' + AND SPLIT_PART(borders, ',', n) <> ''; +GO + +-- who has the most neighbours? + +SELECT + name, + iso, + region, + COUNT('*') AS neighbours +FROM #border_pairs +GROUP BY name, iso, region +ORDER BY neighbours DESC, name ASC +LIMIT 15; +GO + +-- --------------------------------------------------------------------------- +-- join the pairs back to #countries to pick up the neighbour's attributes. +-- NOTE: the engine cannot compare two columns in a WHERE clause yet (P46 in +-- docs/SQL_PARITY.md - it returns zero rows), so the region comparison is done +-- here in the SELECT list, where it works, and filtered in the next statement. +-- --------------------------------------------------------------------------- + +WITH + bp AS ( + SELECT * + FROM #border_pairs + ), + nb AS ( + SELECT + cca3, + "name.common" AS nb_name, + region AS nb_region, + CAST(landlocked AS INT) AS nb_landlocked, + CAST(SPLIT_PART(latlng, ',', 1) AS FLOAT) AS nb_lat, + CAST(SPLIT_PART(latlng, ',', 2) AS FLOAT) AS nb_lng + FROM #countries + ) +SELECT + bp.name, + bp.region, + bp.lat, + bp.lng, + CAST(bp.landlocked AS INT) AS landlocked, + nb.nb_name, + nb.nb_region, + nb.nb_landlocked, + nb.nb_lat, + nb.nb_lng, + CASE WHEN bp.region = nb.nb_region THEN 'internal' ELSE 'cross-region' END AS frontier, + CASE WHEN bp.name < nb.nb_name THEN 'keep' ELSE 'mirror' END AS pair_dedupe +INTO #edges +FROM bp +INNER JOIN nb ON bp.neighbour_iso = nb.cca3; +GO + +-- the land borders that leave their own continent + +SELECT + name, + region, + nb_name, + nb_region +FROM #edges +WHERE frontier = 'cross-region' +ORDER BY region ASC, name ASC; +GO + +-- --------------------------------------------------------------------------- +-- doubly landlocked: landlocked, and every neighbour is landlocked too. +-- there are exactly two in the world. +-- --------------------------------------------------------------------------- + +SELECT + name, + COUNT('*') AS neighbours, + SUM(nb_landlocked) AS landlocked_neighbours, + COUNT('*') - SUM(nb_landlocked) AS coastal_neighbours +INTO #landlocked_summary +FROM #edges +WHERE landlocked = 1 +GROUP BY name; +GO + +SELECT + name, + neighbours, + landlocked_neighbours +FROM #landlocked_summary +WHERE coastal_neighbours = 0 +ORDER BY name ASC; +GO + +-- --------------------------------------------------------------------------- +-- data quality: borders ought to be symmetric. Build a forward key and a +-- reverse key for every pair, then LEFT JOIN the reverse key back onto the +-- forward keys - a miss is a one-sided border in the source data. +-- --------------------------------------------------------------------------- + +WITH + a AS ( + SELECT + iso, + neighbour_iso, + iso || '>' || neighbour_iso AS fwd, + neighbour_iso || '>' || iso AS rev + FROM #border_pairs + ), + b AS ( + SELECT iso || '>' || neighbour_iso AS fwd + FROM #border_pairs + ) +SELECT + a.iso AS claims, + a.neighbour_iso AS but_not_listed_by, + a.fwd AS one_sided_border +FROM a +LEFT JOIN b ON a.rev = b.fwd +WHERE b.fwd IS NULL +ORDER BY a.iso ASC; GO +-- --------------------------------------------------------------------------- +-- no geo functions required - haversine straight off latlng with +-- RADIANS/SIN/COS/ACOS, then CONVERT for the imperial column. +-- --------------------------------------------------------------------------- + +WITH + c AS ( + SELECT + "name.common" AS name, + region, + CAST(SPLIT_PART(latlng, ',', 1) AS FLOAT) AS lat, + CAST(SPLIT_PART(latlng, ',', 2) AS FLOAT) AS lng + FROM #countries + WHERE latlng <> '' + ) +SELECT + name, + region, + lat, + lng, + ROUND(6371 * ACOS( + SIN(RADIANS(51.5)) * SIN(RADIANS(lat)) + + COS(RADIANS(51.5)) * COS(RADIANS(lat)) * COS(RADIANS(lng - (-0.13))) + ), 0) AS km_from_london, + ROUND(CONVERT(6371 * ACOS( + SIN(RADIANS(51.5)) * SIN(RADIANS(lat)) + + COS(RADIANS(51.5)) * COS(RADIANS(lat)) * COS(RADIANS(lng - (-0.13))) + ), 'km', 'miles'), 0) AS miles_from_london +FROM c +ORDER BY km_from_london DESC +LIMIT 10; +GO + +-- the widest land borders: neighbours whose centroids are furthest apart. +-- every border appears twice in #edges (A>B and B>A), so pair_dedupe keeps +-- one direction only. + +SELECT + name, + nb_name, + region, + nb_region, + ROUND(6371 * ACOS( + SIN(RADIANS(lat)) * SIN(RADIANS(nb_lat)) + + COS(RADIANS(lat)) * COS(RADIANS(nb_lat)) * COS(RADIANS(nb_lng - lng)) + ), 0) AS centroid_km +FROM #edges +WHERE pair_dedupe = 'keep' +ORDER BY centroid_km DESC +LIMIT 10; +GO + +-- and the tightest ones - microstates and enclaves + +SELECT + name, + nb_name, + ROUND(6371 * ACOS( + SIN(RADIANS(lat)) * SIN(RADIANS(nb_lat)) + + COS(RADIANS(lat)) * COS(RADIANS(nb_lat)) * COS(RADIANS(nb_lng - lng)) + ), 0) AS centroid_km +FROM #edges +WHERE pair_dedupe = 'keep' +ORDER BY centroid_km ASC +LIMIT 10; +GO + +-- --------------------------------------------------------------------------- +-- currency unions - one currency code, many countries +-- --------------------------------------------------------------------------- + +SELECT + currencies, + COUNT('*') AS members, + STRING_AGG("name.common", ',') AS countries +FROM #countries +WHERE currencies <> '' +GROUP BY currencies +ORDER BY members DESC, currencies ASC +LIMIT 10; +GO + +-- --------------------------------------------------------------------------- +-- languages is another packed list - same slot trick, then count the reach +-- --------------------------------------------------------------------------- + +WITH + slots AS ( + SELECT value AS n + FROM RANGE(1, 6) + ), + c AS ( + SELECT * + FROM #countries + ) +SELECT + SPLIT_PART(languages, ',', n) AS language, + "name.common" AS name, + region +INTO #languages +FROM c +CROSS JOIN slots +WHERE languages <> '' + AND SPLIT_PART(languages, ',', n) <> ''; +GO + +SELECT + language, + COUNT('*') AS countries, + COUNT(DISTINCT region) AS regions +FROM #languages +GROUP BY language +ORDER BY countries DESC, language ASC +LIMIT 15; +GO + +-- --------------------------------------------------------------------------- +-- island nations: no land borders, and not landlocked either +-- --------------------------------------------------------------------------- + +SELECT + flag, + "name.common" AS name, + region, + subregion, + capital, + RENDER_NUMBER(area, 'compact') AS area_km2 +FROM #countries +WHERE borders = '' + AND landlocked = 0 + AND area > 10000 +ORDER BY area DESC +LIMIT 20; +GO + +-- --------------------------------------------------------------------------- +-- window functions plus a text bar chart. CAST(area AS FLOAT) is deliberate: +-- area mixes integers and floats, and raw MIN/MAX rank by type before value +-- (P47 in docs/SQL_PARITY.md - MAX(area) answers 34.2, not Russia). +-- --------------------------------------------------------------------------- + +WITH + c AS ( + SELECT + "name.common" AS name, + region, + CAST(area AS FLOAT) AS area + FROM #countries + WHERE area > 0 + ) +SELECT + ROW_NUMBER() OVER (ORDER BY area DESC) AS rank_world, + ROW_NUMBER() OVER (PARTITION BY region ORDER BY area DESC) AS rank_in_region, + name, + region, + RENDER_NUMBER(area, 'compact') AS area_km2, + ROUND(100.0 * area / SUM(area) OVER (), 2) AS pct_of_world, + ROUND(100.0 * area / SUM(area) OVER (PARTITION BY region), 2) AS pct_of_region, + REPEAT('#', CAST(area / 500000 AS INT)) AS bar +FROM c +ORDER BY area DESC +LIMIT 20; +GO diff --git a/tests/comparison/corpus/01_select.toml b/tests/comparison/corpus/01_select.toml index 52c2343c..0f8c01b5 100644 --- a/tests/comparison/corpus/01_select.toml +++ b/tests/comparison/corpus/01_select.toml @@ -61,3 +61,25 @@ sql = "SELECT symbol, price FROM trades ORDER BY price DESC LIMIT 5" id = "order_by_expr" data = "trades.csv" sql = "SELECT symbol, price, volume FROM trades ORDER BY price * volume DESC" + +# --- P46 contrast: the comparison that fails in WHERE works in projection --- + +[[case]] +id = "select_col_vs_col_projected" +data = "null_edges.csv" +sql = "SELECT id, partner_id, id < partner_id AS lt FROM null_edges WHERE partner_id IS NOT NULL ORDER BY id" +# The anchor for P46. Column-vs-column is evaluated correctly HERE, which is +# what localises that bug to the WHERE evaluator rather than to comparison in +# general. If this case ever starts failing, P46 is bigger than filed. +# The NULL rows are excluded on purpose - they are P48's business, not P46's, +# and they are pinned separately as select_col_vs_col_null_operand below. + +[[case]] +id = "select_col_vs_col_null_operand" +data = "null_edges.csv" +sql = "SELECT id, partner_id, id < partner_id AS lt FROM null_edges ORDER BY id" +expect = "DIFFER" +# P48, found 2026-09-12 while building the P46 anchor above. Same query without +# the IS NOT NULL guard: for a NULL operand we project false, DuckDB projects +# NULL. The three-valued-logic fix of P18/P19 landed in the WHERE evaluator; +# this is the same rule missing at the projection site. diff --git a/tests/comparison/corpus/02_where.toml b/tests/comparison/corpus/02_where.toml index 3e003492..d563f828 100644 --- a/tests/comparison/corpus/02_where.toml +++ b/tests/comparison/corpus/02_where.toml @@ -178,3 +178,25 @@ expect = "GAP" # the same UNKNOWN-under-NOT rule as P19. So when the parse gap is closed, this # doubles as coverage that NOT LIKE inherits the fixed NULL semantics; the # evaluator side is already implemented and returns UNKNOWN for a NULL operand. + +# --- P46: a column-to-column comparison in WHERE returns zero rows --- + +[[case]] +id = "where_col_lt_col" +data = "null_edges.csv" +sql = "SELECT id, partner_id FROM null_edges WHERE id < partner_id ORDER BY id" +expect = "DIFFER" +# P46, found 2026-09-12 writing examples/countries.sql. Both operands are +# columns, and that alone is enough: we return NO rows, DuckDB returns +# 1,4,7,9 (the NULL-partner rows are correctly excluded by both). +# The same comparison in the SELECT list is right - see +# 01_select.toml :: select_col_vs_col_projected. + +[[case]] +id = "where_col_eq_col_string" +data = "instruments.csv" +sql = "SELECT instrument_id, currency, trading_currency FROM instruments WHERE currency = trading_currency ORDER BY instrument_id LIMIT 5" +expect = "DIFFER" +# P46 again, on strings and on equality, to pin that the defect is the +# column-vs-column *shape* rather than one operator or one type. DuckDB matches +# 57 of the 200 rows; we match none. diff --git a/tests/comparison/corpus/10_aggregate_nulls.toml b/tests/comparison/corpus/10_aggregate_nulls.toml index 7cec14c0..a730e98d 100644 --- a/tests/comparison/corpus/10_aggregate_nulls.toml +++ b/tests/comparison/corpus/10_aggregate_nulls.toml @@ -205,3 +205,37 @@ sql = "SELECT team, MODE(score) AS m FROM null_edges GROUP BY team ORDER BY team # Per-group state: each group must tally independently, and the all-NULL group # ('delta') must still produce its row with NULL. Catches a fix applied to the # ungrouped path only. + +# --- P47: MIN/MAX compare by type variant before value --- + +[[case]] +id = "max_mixed_int_float" +data = "null_edges.csv" +sql = "WITH t AS (SELECT 1 AS v UNION ALL SELECT 2.5 AS v UNION ALL SELECT 10 AS v) SELECT MAX(v) AS mx FROM t" +expect = "DIFFER" +# P47, found 2026-09-12 writing examples/countries.sql. One column holding both +# Integer and Float values: MAX returns 2.5 because DataValue's derived +# PartialOrd ranks the Float variant above Integer regardless of the numbers. +# DuckDB returns 10. The table is irrelevant to the query - the literal CTE is +# the smallest shape that produces a mixed-type column deterministically. + +[[case]] +id = "min_mixed_int_float" +data = "null_edges.csv" +sql = "WITH t AS (SELECT 5 AS v UNION ALL SELECT 2.5 AS v UNION ALL SELECT 10 AS v) SELECT MIN(v) AS mn FROM t" +expect = "DIFFER" +# The mirror of the case above, and it needs the true minimum to BE the float: +# we return 5 (an Integer beats any Float for MIN), DuckDB returns 2.5. With +# the values of the MAX case this would pass for the wrong reason. + +[[case]] +id = "max_mixed_int_float_from_csv" +data = "countries.csv" +sql = "SELECT MAX(area) AS mx FROM countries" +expect = "DIFFER" +# P47 as it is actually met. Three of the 250 areas carry decimals (Monaco +# 2.02, US Minor Outlying Islands 34.2, Vatican City 0.44) and they appear late +# in the file, so CSV type inference leaves the column mixed Integer/Float. +# MAX(area) is then 34.2 instead of 17098242 - the largest country in the world +# by a factor of half a million. Keep this alongside the literal-CTE cases: a +# fix that only handles literals would leave this one wrong.