Skip to content

feat: support @rocicorp/zero 1.11.3 - #39

Merged
jnsdls merged 7 commits into
mainfrom
feat/zero-1.11.3
Oct 28, 2026
Merged

jnsdls merged 7 commits into
mainfrom
feat/zero-1.11.3

Conversation

@jnsdls

@jnsdls jnsdls commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Summary

Pins @rocicorp/zero 1.11.3 (from 1.10.0) and ports what changed in the server SDK between them.

 query builder
-  AST as built, normalized only when zero-cache hashes it
+  AST normalized at every step: related sorted by alias, where flattened + sorted
+  values ordered null < bool < number < string < array; static params comparable
+  empty and/or kept (always true / always false)
 z2s
+  LIKE patterns bound as text (non-string patterns match as their string)
+  char(n) compared as ::bpchar; bare `character` column named bpchar
-  $n::text::text  ->  $n::text
 mutate
+  mutator return value -> result.data, written to zero_0.mutations
+  tx.retry_on(predicate): re-run in a fresh tx, at most 5 runs per push
+  tx unusable once its transaction has ended
+  adapters take isolation_level=
 ZQLDatabase.run
+  bare statement when the connection supports it, no BEGIN/COMMIT
 query endpoint
+  transform requests parsed in strip mode (unknown fields ignored)
 compat proof
+  valita 0.5.5 (`dist/index.js`), scope remapped to 1.11.3 line numbers

Not ported: upstream's LogSink (Python logs through stdlib logging), and query interning, which only saves work.

Upstream now interns built queries, and a few type-violating inputs share a key: a flip of 'yes' with true, and any orderBy direction other than 'asc' with 'desc'. That made upstream's output depend on earlier programs. The harness now builds each program from a fresh schema, and docs/compat.md records it as a known difference.

Evidence

  • Before (pin bumped, port unchanged): pytest: 53 failed, 661 passed (AST normalization, query bodies, SQL tier).
    After: pytest: 782 passed, zero_server at 100%; python -m tests.compat: 54 upstream modules and every port module fully exercised differentially.

  • New differential tests:

    • Mutate: return values, every retry_on outcome, using a tx after it ended, isolation level per adapter, run with and without a bare connection.
    • SQL tier: char(3) and bpchar columns, compared and matched with LIKE.
    • Pinned builder programs: each new sort rule.
  • Not run locally: e2e (port 55432 was held by an unrelated container). CI runs it.

  • Wall-clock bench, Python side vs main:

    Case main This PR Change
    build.simple 3.6 µs 4.0 µs about 13% slower
    build.complex 20.1 µs 23.3 µs about 16% slower
    query cases 7–16% slower
    z2s unchanged

    The cost is the per-step normalization upstream added. TS got faster at rebuilding the same query thanks to interning, so expect CodSpeed to flag build/query. Porting interning is the follow-up that wins this back.

Merge Danger

Door: two-way

Blast Radius: wire

Every query AST sent to zero-cache changes shape: key order, sorted conditions and relationships, and empty and/or now kept. This now matches what a 1.11.3 TypeScript server sends. A deployment still on zero-cache 1.10 would see different query hashes, which re-runs its queries.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

jnsdls and others added 7 commits October 28, 2026 13:13
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Mutator return values reach the client as result data and are written to
the mutations table. tx.retry_on() re-runs a failed mutator in a fresh
transaction, at most 5 runs per push, and the mutator's tx stops working
once its transaction ends. Adapters take an isolation level.
ZQLDatabase.run reads without a transaction when the connection has a
bare query, and the server schema fetch is shared while in flight and
not cached on failure. Transform requests parse in strip mode.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednpm/​@​rocicorp/​zero@​1.10.0 ⏵ 1.11.366 -710087 +1100100

View full report

@codspeed

codspeed Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 3.99%

1 warning, 1 note

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

ℹ️ 3 benchmarks have no flamegraph

The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze.

See affected benchmarks

⚡ 4 improved benchmarks
❌ 10 (👁 10) regressed benchmarks
✅ 32 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ build.simple 131.3 µs 77 µs +70.52%
⚡ map.complex 265.9 µs 159.2 µs +67.01%
⚡ map.simple 107.9 µs 69 µs +56.4%
⚡ build.complex 454 µs 341.6 µs +32.87%
👁 parse.push.m1000 45.4 ms 51.4 ms -11.57%
👁 query_json.complex.b1 1.3 ms 1.5 ms -10.31%
👁 query_json.complex.b100 61.4 ms 84.8 ms -27.56%
👁 query_json.simple.b1 747.3 µs 866.9 µs -13.8%
👁 query_json.simple.b100 14 ms 19.5 ms -28.01%
👁 query.complex.b1 1.1 ms 1.3 ms -10.84%
👁 query.complex.b100 42.9 ms 70.2 ms -38.92%
👁 query.simple.b100 10.9 ms 16.6 ms -34.43%
👁 test_cpu[build.complex] 678.4 µs 772.1 µs -12.14%
👁 test_cpu[build.simple] 175.2 µs 205.2 µs -14.6%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing feat/zero-1.11.3 (81709d4) with main (a6ad73e)

Open in CodSpeed

@jnsdls
jnsdls merged commit 9e95bb1 into main Oct 28, 2026
9 checks passed
@jnsdls
jnsdls deleted the feat/zero-1.11.3 branch October 28, 2026 21:14
jnsdls added a commit that referenced this pull request Oct 28, 2026
)

## Summary

Ports upstream 1.11.3's query interning to the builder. When a step
repeats one taken before, the builder returns the query it built then
instead of building it again.

```text
query.where(field, op, value)
  key = (step, every argument, type-tagged)       # 1 / 1.0 / True / "1" stay apart, floats by hex
  if parent._children[key] is alive: return it    # children held weakly, parent strongly
  build the child, remember it under key
  hold it in _RECENT (last 4096 built)            # CPython frees a dropped chain at once; V8 waits for GC
```

Subqueries (`related`, `exists`) are keyed by the identity of their
query's AST. The new `CorrelatedSubquery.source` field keeps that query
alive. A two-hop `exists` builds its junction AST new every time, so
it's keyed by its junction condition instead. Python's keys are exact
where upstream's aren't: upstream folds a `flip` of `'yes'` into `true`.

Stacked on #39, which this wins back the builder slowdown for.

## Evidence

Wall-clock, Python median, same machine:

| case | `main` (1.10.0) | #39 | this PR |
| --- | --- | --- | --- |
| build.simple | 3.5 µs | 4.6 µs | 1.3 µs |
| build.complex | 20.8 µs | 25.2 µs | 13.7 µs |
| query.simple.b100 | 834 µs | 907 µs | 547 µs |
| query.complex.b100 | 3.69 ms | 4.17 ms | 2.78 ms |

- `test_rebuilt_queries_match_upstream`: the same programs three times
in one batch through one builder, compared with the npm package. They
cover every step kind: scalar and float `where`, `whereFn` with a static
parameter and `exists`, two-hop `whereExists`, a raw condition,
`related` with and without a callback, `orderBy`, nested `start`,
`limit` and `one`.
- **Before** the two-hop key fix (0d4781b): fails. `scalar: true` then
`false` returned the first query.
  - **After:** passes.
- `uv run pytest`: 784 passed, 100% coverage. `uv run python -m
tests.compat`: fully exercised.

## Merge Danger

**Door:** two-way

**Blast Radius:** builder

Queries built from the same builder now share objects. Their ASTs equal
what fresh building gives, including when a caller later mutates a list
it passed in, which upstream does the same way. Memory is bounded:
children are held weakly, plus at most 4096 recently built queries.

Reviewed with sol-loop: 2 rounds with gpt-6.1-sol. Sol verdict: approved

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/agent-labs-dev/codesmith/zero-python/pr/40?autoLogin=true&ref=codesmith_pr_footer"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v3.svg"><img
alt="View with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png"
width="162" height="28"></picture></a> <a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1795813402&installation_model_id=434381&pr_number=40&ref=codesmith_pr_footer&repository=agent-labs-dev%2Fzero-python&return_to=https%3A%2F%2Fgithub.com%2Fagent-labs-dev%2Fzero-python%2Fpull%2F40&signature=2340a83be99f9f3903c8c5f9ca8c50963658bc75b1f75129e8997f65aa18a38e"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png"
width="190" height="28"></picture></a>
<sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you
need.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
jnsdls added a commit that referenced this pull request Oct 28, 2026
… their JSON (#43)

## Summary

Follow-up to #39/#40. Upstream 1.11.3 normalizes ASTs at every builder
step, which made the query path slower. This wins it back on the query
endpoint, mostly by not doing the same work twice for a query the
builder already interned.

```diff
 handle_query_request
   handler(name, args)            # interned query, often the same object as last time
-  map_ast(query.ast, mapper)
+  mapped_ast(query, mapper)      # kept on the Query, keyed by mapper identity
 starlette /query
-  js_stringify_bytes(response.to_json())          # orjson, then _may_differ scans ~200 KB
+  query_response_bytes(response)
+    QuerySuccess.to_json_bytes()
+      TransformedQuery.to_json_bytes()
+        AST.to_json_bytes()      # js_stringify_bytes(to_json()) once, kept on the AST
 js_stringify_bytes._may_differ
-  10 substring scans, one per digit, for an array-index key
+  digits deleted with translate, one scan for '"":'
 builder
-  _derive(("related_base"/"exists_base", rel), lambda: ...)
+  _find, then _keep only on a miss (no closure per call)
+  _condition_key: column-vs-scalar condition keyed as one flat tuple
```

ASTs are already immutable by contract (`ast.py`: "no code changes a
node once built"), so caching their mapped form and their text is safe.
`AST._text` is a `compare=False`, `init=False` slot and is reset in
`_copy`.

## Evidence

Wall clock, `python -m bench`, same machine, this branch against `main`
(ca665da):

| case | main | this PR | Py/TS main | Py/TS now |
|---|---:|---:|---:|---:|
| query.simple.b1 | 6.9 µs | 5.2 µs | 1.71 | 1.26 |
| query.simple.b100 | 541.5 µs | 388.0 µs | 2.96 | 2.15 |
| query.complex.b1 | 31.9 µs | 20.3 µs | 3.12 | 1.97 |
| query.complex.b100 | 2.86 ms | 1.81 ms | 3.94 | 2.33 |
| query_json.simple.b1 | 15.4 µs | 8.0 µs | 2.56 | 1.34 |
| query_json.simple.b100 | 1.05 ms | 543.8 µs | 3.53 | 1.64 |
| query_json.complex.b1 | 68.8 µs | 23.5 µs | 3.82 | 1.31 |
| query_json.complex.b100 | 6.14 ms | 2.05 ms | 4.63 | 1.57 |
| build.complex | 14.6 µs | 13.7 µs | 6.54 | 5.68 |
| mutate_cpu.put.n1 | 16.3 µs | 14.6 µs | 4.63 | 3.84 |
| parse, parse_json, map, z2s | | | | unchanged (noise) |

`query_json` now times `query_response_bytes`, which is what the
Starlette route sends.

- New differential assertion in `test_queries_match_upstream`:
`query_response_bytes(actual) == JSON.stringify(upstream response)`,
byte for byte, over the Hypothesis program corpus (floats, surrogates,
errors, `userID` set/unset).
- `uv run pytest`: 784 passed, 100% coverage. `python -m tests.compat`:
54 upstream modules, every port module fully exercised. ruff, ty clean.

The caches pay off when a query repeats (same name, same args), which
interning already makes the common case. A query with args that vary per
request still maps and stringifies once per distinct query.

## Merge Danger

**Door:** two-way

**Blast Radius:** query endpoint

Response bytes are unchanged (proven against upstream). Mapped ASTs are
now shared between responses for the same interned query, so code that
mutates a returned AST would corrupt later responses. Nothing in the SDK
does that.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/agent-labs-dev/codesmith/zero-python/pr/43?autoLogin=true&ref=codesmith_pr_footer"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v3.svg"><img
alt="View with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png"
width="162" height="28"></picture></a> <a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1795815534&installation_model_id=434381&pr_number=43&ref=codesmith_pr_footer&repository=agent-labs-dev%2Fzero-python&return_to=https%3A%2F%2Fgithub.com%2Fagent-labs-dev%2Fzero-python%2Fpull%2F43&signature=f86f3e3910688eb52df613c24891cd33d35dd6ca56f408c4b37e894ede9e5955"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png"
width="190" height="28"></picture></a>
<sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you
need.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

Co-authored-by: Claude Opus 5.5 <[email protected]>
jnsdls added a commit that referenced this pull request Oct 28, 2026
…benchmarks (#46)

## Summary

The docs had fallen behind #39 (1.11.3), #40 (interning) and #43 (cached
mapped ASTs). This brings them level and adds current benchmark ratios.

```diff
 README.md
+  "Queries are shared": interned queries, mapped AST and JSON reused; treat as immutable
-  dangling "wire and SQL tiers run against every adapter" line
+  **Adapters.** bullet in the compatibility list
+  build/map in the CPU case list
+  Py/TS ratio table at ec0768a, 1.11.3, Apple Silicon laptop
 docs/compat.md
+  known difference: logging (stdlib `logging`, not upstream's LogSink)
   re-wrap the interned-queries paragraph
 docs/plan.md
-  "small bun runner"            -> Node runner in harness/
-  "first pin is 1.10.0"         -> was 1.10.0; current pin in ZERO_VERSION
+  status: phases 1-7 done; phase 8 publishes on tag, npm watch not built
 CONTRIBUTING.md "Bumping Zero"
+  README version, docstring tag/commit, compat_scope line remap,
+  `python -m bench.capture`, record new differences
 bench/__init__.py
+  build and map groups
```

## Evidence

Nothing on screen changes; docs only.

- Every API name the README names is exported from `zero_server`. The
version is 1.11.3 everywhere it's pinned (`ZERO_VERSION`, both
`package.json` files, the compose image, the docstrings).
- The ratio table comes from a full local `uv run python -m bench` (exit
0) at ec0768a. Postgres cases: Python within 10% of npm or faster. Query
endpoint with JSON: 1.3–1.7x. `build`/`map`: 6–14x, but microseconds per
call.
- `ruff check` and `ruff format --check` pass on `bench/`.

## Merge Danger

**Door:** two-way

**Blast Radius:** docs

README is the PyPI long description, so the next release's page shows
the new table. The ratios are from one laptop and will drift; the table
names the commit and the machine.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/agent-labs-dev/codesmith/zero-python/pr/46?autoLogin=true&ref=codesmith_pr_footer"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v3.svg"><img
alt="View with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png"
width="162" height="28"></picture></a> <a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1795816999&installation_model_id=434381&pr_number=46&ref=codesmith_pr_footer&repository=agent-labs-dev%2Fzero-python&return_to=https%3A%2F%2Fgithub.com%2Fagent-labs-dev%2Fzero-python%2Fpull%2F46&signature=c4188e5be40817be60bd25370789884b1853df2c8efe7ef6c8707f41d6f4f79f"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png"
width="190" height="28"></picture></a>
<sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you
need.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

Co-authored-by: Claude Opus 5.5 <[email protected]>
@jnsdls jnsdls mentioned this pull request Oct 28, 2026
jnsdls added a commit that referenced this pull request Oct 28, 2026
Bumps the version to 0.0.4 in `pyproject.toml`, `uv.lock` and
`zero_server.__version__`. Nothing else changes.

Since 0.0.3, `main` has gained support for `@rocicorp/zero` 1.11.3
(#39), orjson for request parsing and response writing (#37), query
reuse that builds queries up to 2.7x faster (#40), reused mapped ASTs
and JSON that send query responses up to 3x faster (#43), updated docs
(#46) and dependency bumps.

After merge, tag `v0.0.4` on main to publish to PyPI.

Local run: 786 passed, 100% coverage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/agent-labs-dev/codesmith/zero-python/pr/47?autoLogin=true&ref=codesmith_pr_footer"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v3.svg"><img
alt="View with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png"
width="162" height="28"></picture></a> <a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1795817623&installation_model_id=434381&pr_number=47&ref=codesmith_pr_footer&repository=agent-labs-dev%2Fzero-python&return_to=https%3A%2F%2Fgithub.com%2Fagent-labs-dev%2Fzero-python%2Fpull%2F47&signature=bf55280b5bd9ccc8d24e9874ca54b3b0973db3ee2b748438e15b920b024f233f"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with [code]smith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png"
width="190" height="28"></picture></a>
<sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you
need.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

Co-authored-by: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant