Skip to content

encode: view bodies through Vec<u8>; de-frame in place; gzip buffer sizing - #300

Merged
iainmcgin merged 2 commits into
mainfrom
encode-buffers
Sep 11, 2026
Merged

iainmcgin merged 2 commits into
mainfrom
encode-buffers

Conversation

@rpb-ant

@rpb-ant rpb-ant commented Sep 11, 2026 •

Copy link
Copy Markdown
Collaborator

Three small buffer-management fixes on the request/response path, plus a bench that isolates them from HTTP. Additive API (one new Envelope fn); no wire change. Reviewed with the repo's rust-code-reviewer and rust-api-ergonomics-reviewer agents; their Medium findings (keep decode* signatures concrete, changelog accuracy) are applied.

What and why

  • encode_view_body / encode_view_body_with_min_segment write through a Vec<u8>, not a BytesMut. These size their own buffer and encode the view into it (the path every generated impl Encodable<M> for MView takes). bytes does not inline <BytesMut as BufMut>::put_slice, so every tag and varint byte written through a BytesMut is an out-of-line call; a pre-sized Vec<u8> + Bytes::from is zero-copy, allocation-free at len == capacity, and 3–4× faster. Owned-message paths (Encodable for M, PreEncoded, client request encoding) call buffa's encode_to_bytes, which buffa ≥ 0.9.3 already routes through a Vec (encode: encode_to_bytes writes through Vec<u8>, not BytesMut anthropics/buffa#437), so they're left alone — verified below. Upstream bytes fix: Inline BytesMut's specialized BufMut methods and add put_u8 fast paths tokio-rs/bytes#850.
  • New Envelope::decode_bytes_with_limit(&mut Bytes, max) — decode_with_limit for a body already held as Bytes; the payload is split off by reference count (Buf::copy_to_bytes is zero-copy on Bytes) rather than the body first being copied into a BytesMut. The three places that did that copy just to de-frame — the gRPC unary and server-streaming handlers on the server, and the Connect client-streaming response scan on the client — use it: one allocation and one body-sized memcpy less per message. decode / decode_with_limit keep their &mut BytesMut signatures and now share a private body with the new function.
  • gzip compress_inner no longer doubles its pre-sized output buffer before the first write; it grows only when spare capacity is exhausted (3 allocations / ~3× input bytes → 2 / ~1× per compressed response).
  • benches/rpc/benches/encode_frame.rs: view_encode/* (view → Bytes via buffa's two entry points and encode_view_body) and deframe/* (decode_with_limit on a BytesMut copy vs decode_bytes_with_limit on the Bytes), over a 346 B, 4.4 KB and 271 KB log-record message and a 256 KB four-string blob.

Numbers

criterion medians, one pinned core (Xeon 8488C), main → this branch:

bench main this branch
view_encode/small_300B/connectrpc_encode_view_body 628 ns 166 ns
view_encode/logs_4KB/connectrpc_encode_view_body 4.56 µs 1.21 µs
view_encode/logs_256KB/connectrpc_encode_view_body 264 µs 75.4 µs
view_encode/blob_256KB/connectrpc_encode_view_body (memcpy-bound) 5.33 µs 5.59 µs
deframe/small_300B copy → in place 50 ns 25 ns
deframe/logs_4KB copy → in place 103 ns 25 ns
deframe/logs_256KB copy → in place 5.42 µs 25 ns
gzip, 256 KB text (allocs / bytes allocated, one-off count) 3 / 792 KB 2 / 265 KB

End-to-end (rpc_bench, loopback, ~70 µs floor): unary −1…−8 %, gRPC server-streaming −6 %, other arms within noise — these are per-message constant costs, visible as CPU more than latency.

Why the owned paths are not touched: on main with only the buffa dependency swapped (0.9.2 → 0.9.2 + anthropics/buffa#437), Encodable::encode goes 647 → 184 ns, 4.87 → 1.38 µs, 282 → 86 µs (= the Vec reference), while encode_view_body stays at 563 ns / 4.16 µs / 237 µs — the buffa bump fixes everything except the sites changed here.

Testing

cargo test --workspace (937 passed), cargo test -p connectrpc --no-default-features (514), cargo clippy --workspace --all-features --all-targets -- -D warnings, cargo fmt --check; conformance server and client suites pass. New unit test that decode_bytes_with_limit aliases its input, leaves incomplete input untouched, and rejects over-limit frames.

Follow-ups (not here)

  • Client request framing without the copy (fused Vec when uncompressed, or header + payload as two body frames ≥ 16 KiB): saves one alloc + memcpy per request but makes HTTP/1.1 request bodies chunked; wants its own discussion.
  • zstd output overhang: compress returns a Vec with cap = compress_bound(N); a tiny result pins an N-sized buffer until sent.

@rpb-ant rpb-ant changed the title encode: view bodies through Vec<u8>; de-frame gRPC requests without a copy; gzip buffer sizing encode: view bodies through Vec<u8>; de-frame in place; gzip buffer sizing Sep 11, 2026
@rpb-ant
rpb-ant force-pushed the encode-buffers branch 2 times, most recently from a532266 to 5e62ae5 Compare September 11, 2026 16:23
rpb-ant and others added 2 commits September 11, 2026 20:39
…izing

- `encode_view_body` / `encode_view_body_with_min_segment` sized their own
  `BytesMut` and encoded the view into it. `bytes` does not inline
  `<BytesMut as BufMut>::put_slice`, so every tag/varint byte was an
  out-of-line call; a pre-sized `Vec<u8>` + `Bytes::from` is zero-copy and
  3-4x faster (encode_frame view_encode/*: 628 -> 166 ns at 346 B,
  4.6 -> 1.2 us at 4.4 KB, 264 -> 75 us at 271 KB). Owned-message paths go
  through buffa's `encode_to_bytes`, which buffa >= 0.9.3 already routes
  through a `Vec` (anthropics/buffa#437), so they are left alone.
- `Envelope::decode` / `decode_with_limit` take `&mut impl Buf` instead of
  `&mut BytesMut`; `Buf::copy_to_bytes` is a zero-copy split on both `Bytes`
  and `BytesMut`. The gRPC unary and server-streaming handlers and the
  client's gRPC-web unary response path pass their collected `Bytes`
  directly instead of copying it into a `BytesMut` first (deframe/*:
  50 -> 25 ns at 346 B, 5.4 us -> 25 ns at 271 KB; -1 alloc, -1 memcpy).
- gzip `compress_inner` reserved `2 * len` on top of an already pre-sized
  output buffer before the first write; it now grows only when spare capacity
  is exhausted (3 allocs / ~3x input bytes -> 2 / ~1x per compressed response).

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Ryan Brewster <[email protected]>
@iainmcgin
iainmcgin added this pull request to the merge queue Sep 11, 2026
Merged via the queue into main with commit 758bd3a Sep 11, 2026
14 checks passed
@iainmcgin
iainmcgin deleted the encode-buffers branch September 11, 2026 21:33
iainmcgin added a commit that referenced this pull request Sep 22, 2026
main's Envelope::decode_bytes_with_limit (#300) replaces this branch's
decode_from_bytes_with_limit; the service keeps its call sites and the
decoder agreement test now covers main's function.

Signed-off-by: Iain McGinniss <[email protected]>
iainmcgin added a commit to allada/connect-rust that referenced this pull request Sep 24, 2026
…onnectrpc#291)

Every queued compressed response frame keeps its encoder's whole output
buffer alive: the gzip and zstd paths size that `Vec` to the input
length or a little over (`compress_bound` for zstd), and it becomes the
frame's `Bytes` with the capacity intact, so a compressible frame pins
about its uncompressed size until HTTP/2 writes it. The gzip sizing on
main (connectrpc#300) is also too small for incompressible input at this
provider's default fast level, which grows such input by a few percent:
a 1.1 MB frame of noise ends up in a 2.1 MB buffer after the reserve
doubles it.

Both paths now hand the buffer over through `compressed_output`, which
gives back the unused capacity when the compressed frame is under half
of it, and gzip sizes its buffer from zlib-rs's conservative deflate
bound (about 1.14× the input). A queued compressible frame now pins at
most about twice its compressed size; an incompressible one, about its
input size.

Signed-off-by: Iain McGinniss <[email protected]>
iainmcgin added a commit to allada/connect-rust that referenced this pull request Sep 24, 2026
Profiling the small-unary gRPC benchmark (`cross/unary_small_grpc`, the
~300-byte `small_request()` payload) showed the server making 63 heap
allocations per request, about 26 of them in connect-rust's own request
path rather than in decode or the handler. This removed nine of them (63
→ 54); connectrpc#300 has since landed one of the nine, the copy of the request
body before de-framing, and this PR keeps the other eight:

- `RequestMetadata::from_headers` cloned the request `HeaderMap` for the
handler context although the request is consumed a few lines later. The
three entry points now take the request apart first and move the map.
- Constant header values (content-type, encodings, `te`,
`connect-protocol-version`) on server responses (unary and streaming
alike) and client requests use `HeaderValue::from_static`, and the
accept-encoding value is built once per `CompressionRegistry` (now
exposed as `accept_encoding_value()`), instead of each going through
`HeaderValue::from_str`.

`from_static` on a negotiated encoding name is only safe if provider
names are valid header values, so `CompressionRegistry::register` now
panics when `CompressionProvider::name()` is not an RFC 9110 token. The
alternative was caching a `HeaderValue` per provider and keeping
`register` infallible.

Signed-off-by: Iain McGinniss <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants