Repository navigation
Commit cd74c80
fix: match upstream on surrogates, wire keys, raw bodies and query results (#30)
## Summary
Fixes divergences B4, B6, B7 and B8 from the 1.10.0 accuracy
investigation, plus three smaller ones found along the way: handler
concurrency, `details: null`, and response key order and number
spelling.
| Finding | Before | After (matches `@rocicorp/zero` 1.10.0) |
| --- | --- | --- |
| B4: lone surrogate in an app error message | PushFailed `database`,
nothing recorded | app error, stored as `JSON.stringify` writes it
(`\udc00` escaped) |
| B4: lone surrogate in a bound string (`item.put`, where literal) |
driver refuses, app error or query error | bound as U+FFFD, as
postgres.js sends it |
| B4: lone surrogate in a Starlette response | `UnicodeEncodeError`, 500
| `JSON.stringify` text, UTF-8 |
| B6: snake_case keys (`client_group_id`, `order_by`, `parent_field`,
`app_id`, ...) | accepted | refused, as valita refuses them |
| B6: extra `"schema_version": null` in a push body | refused | accepted
(passthrough) |
| B7: `/query` body with NaN or Infinity | accepted | TransformFailed
`parse`, `SyntaxError` |
| B7: body with bytes that are not UTF-8, or one or two BOMs |
`UnicodeDecodeError` or parse failure | decoded as Node's fetch decodes
it (U+FFFD, up to two BOMs dropped) |
| B8: query handler returns a primitive | whole batch fails,
TransformFailed `internal` | that query fails alone: `Cannot use 'in'
operator to search for 'query' in X`, `{name: "TypeError"}` |
| B8: `defineQuery` function returns a non-query | passed through |
`asQueryInternals` error, as upstream |
| Query handlers | run one after another | run together, as
`Promise.all` runs them |
| Error with `details: null` | `{name}` reported | no details, as
upstream |
| Response key order | Python declaration order | upstream's order,
including its second order for response mutation IDs, negated simple
conditions, a condition's subquery, the unsupported-push-version failure
and start rows (`Object.entries`) |
| A number past a double (`1e400`) in a response | crash in `js_number`
| `null`, as `JSON.stringify` writes Infinity |
The fast paths from #29 keep their speed and follow these rules. The
fast `js_stringify` escapes surrogates, the cached optional keys are
aliases only, the hand-built mutate response writes upstream's key
order, and every transaction a session opens goes through the
surrogate-cleaning wrapper.
Not mirrored:
- The deprecated `{query: q}` unwrap. A dict with a `"query"` key fails
the batch instead.
- A query name with a `~` part, or a JS `Object.prototype` name.
Upstream's registry lookup finds the registry's type tag, and what fails
after that is the caller's own code.
- Promise tick ordering between sync and async handlers when two of them
fail in one batch.
Open divergences the explore runs found that this PR leaves alone: the
builder accepts a direction like `"�asc"` and a non-`$param` object as a
literal upstream, and Python's builder refuses both.
## Evidence
Each fix began as a failing differential test against the published
package:
- `tests/wire/test_bodies.py` (new) sends raw bytes through upstream's
`Request` path and through Python's bytes path and Starlette, then
compares the response bytes. Cases: NaN, Infinity, `1e400`, `-0`, huge
ints, BOMs, UTF-16, truncated bodies and bytes that are not UTF-8.
- `tests/wire/test_js.py`: `js_stringify` against `JSON.stringify` over
JSON values that include lone surrogates.
- `tests/wire/test_requests.py` and `test_ast.py`: `corrupted()` now
renames keys to their Python names and adds a null under that name.
- `tests/wire/test_query.py`: handlers that return each kind of
primitive and object, a gate test that deadlocks unless handlers run
together, `ApplicationError` details, and response comparison as JSON
text.
- `tests/wire/test_mutate.py`: lone surrogates in values and messages,
`details` null or absent, and response comparison as JSON text.
- `tests/strategies.py`: `strings()` replaces `st.text` and gives about
one character in eight as a lone surrogate.
Gate: ruff, format and ty clean. pytest 460 passed, 100% branch
coverage. CodSpeed against main 8994dca: 5 benchmarks faster
(query_json.complex.b100 by 52%), 33 unchanged, none slower. One explore
run of `test_mutate.py` and `test_requests.py`: 17 passed. Earlier full
explore runs of `tests/wire` before the rebase onto #29: 309 passed.
## Merge Danger
Door: two-way. One commit, nothing persisted changes shape, and a revert
restores the old behaviour.
Blast radius:
- Wire parsing is stricter: a snake_case key is now an unknown key.
zero-cache never sends one.
- Response key order changes. zero-cache hashes the ASTs it receives,
and its normalization keeps a simple condition's key order. A Python
server's query hashes should now match a TypeScript server's. A running
Python deployment may rehydrate queries once after the upgrade.
- Query handlers now run concurrently. A handler that relied on running
after the previous one finished has to do its own locking.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent 8994dca commit cd74c80
33 files changed
Lines changed: 815 additions & 162 deletions
File tree
- harness
- src/zero_server
- integrations
- protocol
- query
- server
- adapters
- tests
- unit
- wire
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
83 | 88 | | |
84 | 89 | | |
85 | 90 | | |
| |||
147 | 152 | | |
148 | 153 | | |
149 | 154 | | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
150 | 173 | | |
151 | 174 | | |
152 | 175 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | | - | |
| 12 | + | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
| 30 | + | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
| |||
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
39 | 46 | | |
40 | 47 | | |
41 | 48 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
109 | 113 | | |
110 | 114 | | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
111 | 139 | | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
120 | 149 | | |
121 | 150 | | |
122 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
33 | 41 | | |
34 | 42 | | |
35 | 43 | | |
| |||
64 | 72 | | |
65 | 73 | | |
66 | 74 | | |
67 | | - | |
| 75 | + | |
68 | 76 | | |
69 | 77 | | |
70 | 78 | | |
| |||
80 | 88 | | |
81 | 89 | | |
82 | 90 | | |
83 | | - | |
| 91 | + | |
84 | 92 | | |
85 | 93 | | |
86 | 94 | | |
| |||
108 | 116 | | |
109 | 117 | | |
110 | 118 | | |
111 | | - | |
| 119 | + | |
112 | 120 | | |
113 | 121 | | |
114 | 122 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| |||
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
76 | | - | |
77 | 76 | | |
78 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
79 | 89 | | |
80 | 90 | | |
81 | 91 | | |
| |||
95 | 105 | | |
96 | 106 | | |
97 | 107 | | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
98 | 114 | | |
99 | 115 | | |
100 | 116 | | |
| |||
108 | 124 | | |
109 | 125 | | |
110 | 126 | | |
111 | | - | |
112 | | - | |
113 | 127 | | |
114 | 128 | | |
| 129 | + | |
| 130 | + | |
115 | 131 | | |
116 | 132 | | |
117 | 133 | | |
| |||
126 | 142 | | |
127 | 143 | | |
128 | 144 | | |
| 145 | + | |
129 | 146 | | |
130 | 147 | | |
131 | | - | |
132 | 148 | | |
133 | 149 | | |
134 | 150 | | |
| |||
201 | 217 | | |
202 | 218 | | |
203 | 219 | | |
204 | | - | |
| 220 | + | |
205 | 221 | | |
206 | 222 | | |
207 | | - | |
| 223 | + | |
208 | 224 | | |
209 | 225 | | |
210 | 226 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| 68 | + | |
| 69 | + | |
68 | 70 | | |
69 | | - | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
70 | 74 | | |
71 | 75 | | |
72 | | - | |
| 76 | + | |
73 | 77 | | |
74 | 78 | | |
75 | 79 | | |
76 | 80 | | |
77 | 81 | | |
78 | 82 | | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
79 | 89 | | |
80 | 90 | | |
81 | 91 | | |
| |||
93 | 103 | | |
94 | 104 | | |
95 | 105 | | |
| 106 | + | |
| 107 | + | |
96 | 108 | | |
97 | | - | |
98 | | - | |
| 109 | + | |
| 110 | + | |
99 | 111 | | |
| 112 | + | |
| 113 | + | |
100 | 114 | | |
101 | 115 | | |
102 | 116 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
24 | 23 | | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
| 27 | + | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
31 | 30 | | |
32 | 31 | | |
33 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
130 | 130 | | |
131 | 131 | | |
132 | 132 | | |
133 | | - | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
26 | 25 | | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
0 commit comments