Repository navigation
encode: view bodies through Vec<u8>; de-frame in place; gzip buffer sizing - #300
Merged
Merged
Conversation
rpb-ant
force-pushed
the
encode-buffers
branch
from
September 11, 2026 15:48
129fcc8 to
3477c32
Compare
Vec<u8>; de-frame gRPC requests without a copy; gzip buffer sizingVec<u8>; de-frame in place; gzip buffer sizing
rpb-ant
force-pushed
the
encode-buffers
branch
2 times, most recently
from
September 11, 2026 16:23
a532266 to
5e62ae5
Compare
…rom HTTP Signed-off-by: Ryan Brewster <[email protected]>
…izing - `encode_view_body` / `encode_view_body_with_min_segment` sized their own `BytesMut` and encoded the view into it. `bytes` does not inline `<BytesMut as BufMut>::put_slice`, so every tag/varint byte was an out-of-line call; a pre-sized `Vec<u8>` + `Bytes::from` is zero-copy and 3-4x faster (encode_frame view_encode/*: 628 -> 166 ns at 346 B, 4.6 -> 1.2 us at 4.4 KB, 264 -> 75 us at 271 KB). Owned-message paths go through buffa's `encode_to_bytes`, which buffa >= 0.9.3 already routes through a `Vec` (anthropics/buffa#437), so they are left alone. - `Envelope::decode` / `decode_with_limit` take `&mut impl Buf` instead of `&mut BytesMut`; `Buf::copy_to_bytes` is a zero-copy split on both `Bytes` and `BytesMut`. The gRPC unary and server-streaming handlers and the client's gRPC-web unary response path pass their collected `Bytes` directly instead of copying it into a `BytesMut` first (deframe/*: 50 -> 25 ns at 346 B, 5.4 us -> 25 ns at 271 KB; -1 alloc, -1 memcpy). - gzip `compress_inner` reserved `2 * len` on top of an already pre-sized output buffer before the first write; it now grows only when spare capacity is exhausted (3 allocs / ~3x input bytes -> 2 / ~1x per compressed response). Co-Authored-By: Claude <[email protected]> Signed-off-by: Ryan Brewster <[email protected]>
rpb-ant
force-pushed
the
encode-buffers
branch
from
September 11, 2026 20:40
5e62ae5 to
cf9e1e0
Compare
iainmcgin
approved these changes
Sep 11, 2026
This was referenced Sep 21, 2026
iainmcgin
added a commit
that referenced
this pull request
Sep 22, 2026
main's Envelope::decode_bytes_with_limit (#300) replaces this branch's decode_from_bytes_with_limit; the service keeps its call sites and the decoder agreement test now covers main's function. Signed-off-by: Iain McGinniss <[email protected]>
iainmcgin
added a commit
to allada/connect-rust
that referenced
this pull request
Sep 24, 2026
…onnectrpc#291) Every queued compressed response frame keeps its encoder's whole output buffer alive: the gzip and zstd paths size that `Vec` to the input length or a little over (`compress_bound` for zstd), and it becomes the frame's `Bytes` with the capacity intact, so a compressible frame pins about its uncompressed size until HTTP/2 writes it. The gzip sizing on main (connectrpc#300) is also too small for incompressible input at this provider's default fast level, which grows such input by a few percent: a 1.1 MB frame of noise ends up in a 2.1 MB buffer after the reserve doubles it. Both paths now hand the buffer over through `compressed_output`, which gives back the unused capacity when the compressed frame is under half of it, and gzip sizes its buffer from zlib-rs's conservative deflate bound (about 1.14× the input). A queued compressible frame now pins at most about twice its compressed size; an incompressible one, about its input size. Signed-off-by: Iain McGinniss <[email protected]>
iainmcgin
added a commit
to allada/connect-rust
that referenced
this pull request
Sep 24, 2026
Profiling the small-unary gRPC benchmark (`cross/unary_small_grpc`, the ~300-byte `small_request()` payload) showed the server making 63 heap allocations per request, about 26 of them in connect-rust's own request path rather than in decode or the handler. This removed nine of them (63 → 54); connectrpc#300 has since landed one of the nine, the copy of the request body before de-framing, and this PR keeps the other eight: - `RequestMetadata::from_headers` cloned the request `HeaderMap` for the handler context although the request is consumed a few lines later. The three entry points now take the request apart first and move the map. - Constant header values (content-type, encodings, `te`, `connect-protocol-version`) on server responses (unary and streaming alike) and client requests use `HeaderValue::from_static`, and the accept-encoding value is built once per `CompressionRegistry` (now exposed as `accept_encoding_value()`), instead of each going through `HeaderValue::from_str`. `from_static` on a negotiated encoding name is only safe if provider names are valid header values, so `CompressionRegistry::register` now panics when `CompressionProvider::name()` is not an RFC 9110 token. The alternative was caching a `HeaderValue` per provider and keeping `register` infallible. Signed-off-by: Iain McGinniss <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three small buffer-management fixes on the request/response path, plus a bench that isolates them from HTTP. Additive API (one new
Envelopefn); no wire change. Reviewed with the repo'srust-code-reviewerandrust-api-ergonomics-revieweragents; their Medium findings (keepdecode*signatures concrete, changelog accuracy) are applied.What and why
encode_view_body/encode_view_body_with_min_segmentwrite through aVec<u8>, not aBytesMut. These size their own buffer and encode the view into it (the path every generatedimpl Encodable<M> for MViewtakes).bytesdoes not inline<BytesMut as BufMut>::put_slice, so every tag and varint byte written through aBytesMutis an out-of-line call; a pre-sizedVec<u8>+Bytes::fromis zero-copy, allocation-free atlen == capacity, and 3–4× faster. Owned-message paths (Encodable for M,PreEncoded, client request encoding) call buffa'sencode_to_bytes, which buffa ≥ 0.9.3 already routes through aVec(encode:encode_to_byteswrites throughVec<u8>, notBytesMutanthropics/buffa#437), so they're left alone — verified below. Upstreambytesfix: InlineBytesMut's specializedBufMutmethods and addput_u8fast paths tokio-rs/bytes#850.Envelope::decode_bytes_with_limit(&mut Bytes, max)—decode_with_limitfor a body already held asBytes; the payload is split off by reference count (Buf::copy_to_bytesis zero-copy onBytes) rather than the body first being copied into aBytesMut. The three places that did that copy just to de-frame — the gRPC unary and server-streaming handlers on the server, and the Connect client-streaming response scan on the client — use it: one allocation and one body-sized memcpy less per message.decode/decode_with_limitkeep their&mut BytesMutsignatures and now share a private body with the new function.compress_innerno longer doubles its pre-sized output buffer before the first write; it grows only when spare capacity is exhausted (3 allocations / ~3× input bytes → 2 / ~1× per compressed response).benches/rpc/benches/encode_frame.rs:view_encode/*(view →Bytesvia buffa's two entry points andencode_view_body) anddeframe/*(decode_with_limiton aBytesMutcopy vsdecode_bytes_with_limiton theBytes), over a 346 B, 4.4 KB and 271 KB log-record message and a 256 KB four-string blob.Numbers
criterion medians, one pinned core (Xeon 8488C),
main→ this branch:view_encode/small_300B/connectrpc_encode_view_bodyview_encode/logs_4KB/connectrpc_encode_view_bodyview_encode/logs_256KB/connectrpc_encode_view_bodyview_encode/blob_256KB/connectrpc_encode_view_body(memcpy-bound)deframe/small_300Bcopy → in placedeframe/logs_4KBcopy → in placedeframe/logs_256KBcopy → in placeEnd-to-end (
rpc_bench, loopback, ~70 µs floor): unary −1…−8 %, gRPC server-streaming −6 %, other arms within noise — these are per-message constant costs, visible as CPU more than latency.Why the owned paths are not touched: on
mainwith only the buffa dependency swapped (0.9.2 → 0.9.2 + anthropics/buffa#437),Encodable::encodegoes 647 → 184 ns, 4.87 → 1.38 µs, 282 → 86 µs (= theVecreference), whileencode_view_bodystays at 563 ns / 4.16 µs / 237 µs — the buffa bump fixes everything except the sites changed here.Testing
cargo test --workspace(937 passed),cargo test -p connectrpc --no-default-features(514),cargo clippy --workspace --all-features --all-targets -- -D warnings,cargo fmt --check; conformance server and client suites pass. New unit test thatdecode_bytes_with_limitaliases its input, leaves incomplete input untouched, and rejects over-limit frames.Follow-ups (not here)
Vecwhen uncompressed, or header + payload as two body frames ≥ 16 KiB): saves one alloc + memcpy per request but makes HTTP/1.1 request bodies chunked; wants its own discussion.compressreturns aVecwithcap = compress_bound(N); a tiny result pins an N-sized buffer until sent.