From 638793ec71e9871ccbd82a8d448368cacadde233 Mon Sep 17 00:00:00 2001 From: Ethan Arrowood Date: Fri, 4 Sep 2026 10:21:32 -0600 Subject: [PATCH 1/2] stream: add dump()/dumpSync() for stream/iter Every consumer in node:stream/iter retains what it reads, so there is no way to read a streamable to completion while keeping nothing. To clear a stream, callers write `for await (const _ of source) {}`. This is apparent throughout multiple test suites, including the stream/iter tests themselves. These are especially useful for QUIC where a receiver doesn't want the payload, but still has to read it to completion to relieve backpressure. bytes() does that too, but allocates the whole payload to discard it. dump() pulls every batch and drops it, so peak memory is one batch regardless of volume. It takes the same signal and limit options as the other consumers, rejects if the source errors mid-stream, and fulfills with undefined. dumpSync() is the synchronous form. The name follows undici's body.dump(). drain() was the obvious alternative but this module already exports ondrain() and drainableProtocol() for writer-side backpressure, which is the opposite side of the pipe. Assisted-by: Claude Opus 5 Signed-off-by: Ethan Arrowood --- doc/api/quic.md | 4 + doc/api/stream_iter.md | 76 ++++ lib/internal/streams/iter/consumers.js | 77 ++++ lib/stream/iter.js | 6 + .../test-stream-iter-consumers-dump.mjs | 424 ++++++++++++++++++ test/parallel/test-stream-iter-namespace.js | 6 + 6 files changed, 593 insertions(+) create mode 100644 test/parallel/test-stream-iter-consumers-dump.mjs diff --git a/doc/api/quic.md b/doc/api/quic.md index a9e7414f4a3e..82304cba00ad 100644 --- a/doc/api/quic.md +++ b/doc/api/quic.md @@ -343,6 +343,10 @@ Only one async iterator can be obtained per stream. The stream is also compatible with `node:stream/iter` utilities such as `Stream.bytes()`, `Stream.text()`, and `Stream.pipeTo()`. +Consuming a stream is what returns flow-control credit to the peer, so a +stream whose payload is not wanted should still be read to completion. Use +`Stream.dump()` to read the stream without retaining any of it. + ### Datagrams In addition to streams, QUIC supports unreliable datagrams ([RFC 9221][]) for diff --git a/doc/api/stream_iter.md b/doc/api/stream_iter.md index c1ebacd3dfad..bd8d9a36f5ab 100644 --- a/doc/api/stream_iter.md +++ b/doc/api/stream_iter.md @@ -1025,6 +1025,81 @@ added: Synchronous version of [`bytes()`][]. +### `dump(source[, options])` + + + +* `source` {AsyncIterable|Iterable} whose chunks must be {Uint8Array\[]} +* `options` {Object} + * `signal` {AbortSignal} + * `limit` {number} Maximum number of bytes to consume. If the total bytes + read exceeds limit, an `ERR_OUT_OF_RANGE` error is thrown +* Returns: {Promise} Fulfills with `undefined`. + +Read a source to completion, discarding every chunk. Unlike the other +consumers, `dump()` retains nothing. Memory tops-out at a single batch no +matter how much data the source produces. + +Use this to consume a stream when the content doesn't matter. For example, A +QUIC stream only returns flow-control credit to the peer as its data is +consumed, so a receiver that does not want the payload must still read it to +completion. + +If the source errors part-way through, the returned promise rejects with that +error. + +There is no default `limit`. `dump()` reads until the source is exhausted +unless a limit is specified. When a limit is configured and the source exceeds +it, the promise rejects and the source is cancelled. A partial dump is never +reported as success. + +```mjs +import { dump, from, pull, tap } from 'node:stream/iter'; + +// Count the bytes flowing through a stream without retaining any of them. +let bytesSeen = 0; +const counter = tap((chunks) => { + for (const chunk of chunks) bytesSeen += chunk.byteLength; +}); + +await dump(pull(from('hello world'), counter)); +console.log(bytesSeen); // 11 +``` + +```cjs +const { dump, from, pull, tap } = require('node:stream/iter'); + +async function run() { + // Count the bytes flowing through a stream without retaining any of them. + let bytesSeen = 0; + const counter = tap((chunks) => { + for (const chunk of chunks) bytesSeen += chunk.byteLength; + }); + + await dump(pull(from('hello world'), counter)); + console.log(bytesSeen); // 11 +} + +run().catch(console.error); +``` + +### `dumpSync(source[, options])` + + + +* `source` {Iterable} whose chunks must be {Uint8Array\[]} +* `options` {Object} + * `limit` {number} Maximum number of bytes to consume. If the total bytes + read exceeds limit, an `ERR_OUT_OF_RANGE` error is thrown +* Returns: {undefined} + +Synchronous version of [`dump()`][]. Throws `ERR_INVALID_ARG_TYPE` if `source` +is not synchronously iterable. + ### `text(source[, options])`