Skip to content

client: gRPC bidirectional streaming mode (--grpc-mode bidi-stream) - #1603

Open
bpalermo wants to merge 3 commits into
envoyproxy:mainfrom
bpalermo:up/grpc-stream
Open

client: gRPC bidirectional streaming mode (--grpc-mode bidi-stream)#1603
bpalermo wants to merge 3 commits into
envoyproxy:mainfrom
bpalermo:up/grpc-stream

Conversation

@bpalermo

@bpalermo bpalermo commented Sep 7, 2026

Copy link
Copy Markdown

Description

This PR is related to #24

Adds --grpc-mode bidi-stream, gRPC bidirectional streaming load with per-message latency. It opens --streams long-lived bidi streams to the URI path (spread evenly over the workers) and sends the --request-body-file message on them at an aggregate --rps message rate, round-robin over the streams, on the sequencer's absolute schedule: late sends fire immediately and are never rescheduled, which keeps the run free of coordinated omission. The server is expected to echo one message per message, in order, on the same stream (gRPC guarantees per-stream ordering), so echoes are correlated FIFO per stream and the send-to-echo time of every message is recorded in benchmark_stream.message_latency.

A send scheduled for a stream that already has --max-inflight-per-stream unanswered messages (default 256), or whose write buffer is above Envoy's high watermark, is dropped and counted in benchmark.stream_deferred rather than queued, which makes that counter a clean saturation signal. At the end of the run every stream is half-closed and echoes are collected for --stream-drain-duration (default 0.5s) before the counters are snapshotted; the grpc-status of each closed stream lands in benchmark.stream_grpc_status.<code>, and streams still open when the window ends are counted in benchmark.stream_drain_incomplete with their unanswered messages in benchmark.stream_inflight_lost.

Other counters: benchmark.stream_messages_sent / stream_messages_received, streams_opened, stream_open_failures, stream_unavailable, stream_resets, stream_early_close, stream_unexpected_message, stream_write_blocked.

Notes for Reviewers

  • Stacked on client: add --grpc-mode unary for gRPC load with grpc-status scoring #1602 (which is stacked on client: add --request-body-file to send a file's bytes as the request body #1601): only the last commit is new here; will be rebased as those merge.
  • bidi-stream is the second value of the GrpcMode enum that client: add --grpc-mode unary for gRPC load with grpc-status scoring #1602 introduces with --grpc-mode unary, so streaming needs no separate boolean flag.
  • Design: a new GrpcStreamBenchmarkClientImpl (source/client/grpc_stream_client_impl.*) owns the streams and implements BenchmarkClient; the existing sequencer, rate limiter and termination predicates drive it unchanged (tryStartRequest() sends one message and completes on its echo). Envoy's Grpc::Decoder reassembles echoed frames across DATA frames. BenchmarkClient gains prepare() (open the streams before the sequencer starts) and finish() (half-close and drain before the worker snapshots counters), both no-ops for the HTTP client.
  • In this mode --rps is the aggregate message rate divided over the workers, --concurrency must be numeric and divide both --streams and --rps, and --max-pending-requests is raised to hold all of a worker's opening streams; --max-active-requests must cover the streams per worker. Not combined with --simple-warmup, request source plugins or user-defined output plugins.
  • Testing: //test:grpc_stream_client_test with a mocked pool/encoder (sync and async opens, round-robin and echo completion with latency, frames split across DATA chunks, in-flight bound and write-watermark deferral, early server close, resets, half-close + drain, drain timeout and terminate); //test:options_test, //test:factories_test, //test:client_worker_test for the new hooks. Verified end to end against a grpc-java bidi echo service (arm64): 20 streams at 400 to 4,800 msg/s and 40 streams at 4,000 to 16,000 msg/s, every message echoed, zero deferred, all streams closed with grpc-status 0. README usage regenerated; version history updated.

… body

Nighthawk could only send a synthetic body ('a' repeated --request-body-size
times) or, through a request source plugin, a UTF-8 json_body string. Add
--request-body-file, which reads the whole file and sends its bytes
verbatim on every request (binary safe), carried over the gRPC service API
in the new RequestOptions.request_body bytes field. Content-Length is set
from the body size; no Content-Type is assumed, supply one with
--request-header. Mutually exclusive with --request-body-size.

StaticRequestSourceImpl gains an optional body so the static request source
can yield it; the plugin and remote request sources are unchanged.

Signed-off-by: Bruno Palermo <[email protected]>
Nighthawk had no gRPC awareness: a request with a gRPC body had to be
assembled by hand and a failed RPC (HTTP 200 + grpc-status 13) counted
as a 2xx success because decodeTrailers() only kept trailers for
tracing.

--grpc-mode unary issues gRPC unary calls:
- implies --protocol http2 (h2c prior knowledge on http:// URIs) and
  --request-method POST; sets content-type: application/grpc and
  te: trailers; no content-length;
- wraps the --request-body-file bytes (a serialized protobuf message) in
  the gRPC length-prefixed frame, so users supply only the message;
- the method comes from the URI path (/pkg.Service/Method), or a :path
  request header;
- reads grpc-status from the trailers, or from the headers on
  trailers-only responses. Status 0 is a success and feeds the new
  benchmark_http_client.latency_grpc_ok statistic; any other or missing
  status increments benchmark.grpc_error and benchmark.grpc_status.<code>
  (.missing when absent) and is not counted as benchmark.http_2xx. HTTP
  level counters and latency_* statistics keep their meaning.

StreamDecoderCompletionCallback::onComplete()/exportLatency() now pass the
observed grpc-status through. Fixes envoyproxy#24 for unary calls; streaming is a
follow-up: --grpc-mode is an enum-valued flag (GrpcMode in options.proto)
so that streaming becomes --grpc-mode bidi-stream rather than a second
boolean flag.

Signed-off-by: Bruno Palermo <[email protected]>
Adds a streaming BenchmarkClient that opens --streams long-lived gRPC
bidi streams per run (spread evenly over the workers) and sends the
--request-body-file message on them at an aggregate --rps message rate,
round-robin over the streams on the sequencer's absolute schedule, so
late sends fire immediately and are never rescheduled. Echoes are
correlated FIFO per stream (gRPC delivers in order) and the send-to-echo
time is recorded in benchmark_stream.message_latency. A send scheduled
for a stream that already has --max-inflight-per-stream unanswered
messages, or whose write buffer is above the high watermark, is dropped
and counted in benchmark.stream_deferred rather than queued, making that
counter a clean saturation signal. At the end of the run the streams are
half-closed and echoes are collected for --stream-drain-duration
(default 500ms) before the counters are snapshotted; the grpc-status of
every closed stream lands in benchmark.stream_grpc_status.<code>.

Other counters: stream_messages_sent/received, streams_opened,
stream_open_failures, stream_unavailable, stream_resets,
stream_early_close, stream_unexpected_message, stream_inflight_lost,
stream_write_blocked.

To support this the BenchmarkClient interface gains prepare() and
finish() hooks that the worker calls around the sequencer run, the
pending-request circuit breaker is raised to hold all of a worker's
opening streams, and --rps is divided over the workers in this mode
(--concurrency must be numeric and divide --streams and --rps).

BIDI_STREAM is the second value of the GrpcMode enum introduced with
--grpc-mode unary, so streaming needs no separate boolean flag.

Streams the server does not close within --stream-drain-duration are
counted in benchmark.stream_drain_incomplete and their unanswered
messages moved into benchmark.stream_inflight_lost by finish(), so
sent - received is always explained in the output.

Signed-off-by: Bruno Palermo <[email protected]>
@bpalermo bpalermo changed the title client: gRPC bidirectional streaming mode (--grpc-stream) client: gRPC bidirectional streaming mode (--grpc-mode bidi-stream) Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant