Remote file access: send file content as raw bytes instead of base64 (~+33% per stream) #175

Follow-up of #146 (remote file read). Makes every read terminal of client.file(path) (readBytes, readText, openStream, openReader) and the upcoming downloadTo (#147) about a third faster, on a single connection, with no API change.

Context

#146 reads a remote file by running a PowerShell script that writes each block of the file as one base64 line to stdout. Base64 was chosen to be safe against the remote console code page. It costs a 4/3 inflation, and that inflation now sets the speed limit.

Measurements made while tuning #146 (Windows Server 2008 R2 / PS 2.0, Server 2019 and Server 2022 / PS 5.1, all identical):

• The WinRM service reads a command's stdout pipe itself, at most 32 KiB per read and about 60 reads per second per output stream, so each stream carries about 2 MB/s of output at best, whatever the network, the client or MaxEnvelopeSize.
• The client is not the limit: it takes in 24–31 MB/s against a local fake server.
• With base64, the best line shape (49,149-byte blocks → 65,533-byte lines = exactly 2 reads) gives 1.4–1.5 MB/s of file data (20 MiB in 14–15 s). The theoretical ceiling for base64 is 24 KiB of file data per 32 KiB read ≈ 1.57 MB/s.
• Raw bytes written with [Console]::OpenStandardOutput().Write in 64 KiB blocks reached 2.07–2.10 MB/s on Server 2022 (synthetic output, not yet through the file reader). An earlier probe found raw output byte-exact on Server 2022 and about 35% faster than base64 for the same file.

Expected gain: about +33% per stream (ceiling 2.1 vs 1.57 MB/s).

Proposal

• The read script writes the file's bytes raw to [Console]::OpenStandardOutput(), in blocks that are a multiple of 32 KiB (e.g. 64 KiB), one Write per block.
• The client consumes stdout as bytes, not text: add a package-private raw-byte path next to RemoteProcess's character readers (the CommandCursor chunks are already raw bytes; ChunkDecoder must be bypassed).
• Errors stay on the exit code and stderr, exactly as today (exit code checked at the end of the output, a failure midway is never a silently short read).
• digest() is unchanged (its output is tiny).

Requirements / open questions

• Byte-exactness on every supported host, in particular Windows Server 2008 R2 / PowerShell 2.0: verify that the WinRS shell does not translate anything in stdout (CR/LF, the console code page 65001, a leading BOM — PowerShell 2.0 prefixes its text output with a UTF-8 BOM, which the raw stream must not get). Test every byte value 0–255 and a file ending in \r / \n.
• Keep base64 as a fallback only if some host turns out not to be byte-exact (detect, don't guess), or drop it entirely if all targets pass.
• The RemoteFilesScriptTest (runs the scripts in the local powershell.exe) and RemoteFileTest (FakeWsmanServer) move to raw bytes; add a test with an output chunk boundary in the middle of a block.
• Update the Performance section of src/site/markdown/files.md with the new measured figure on the three test hosts.

Acceptance criteria

• Every read terminal returns byte-exact content, verified by SHA-256 against the remote file, on 2008 R2, 2019 and 2022.
• A 20 MiB openStream() read is measurably faster than with base64 on all three hosts (target ≥ 1.9 MB/s).
• mvn verify site green.

🤖 Generated with Claude Code