Download remote files: downloadFile() and downloadTo(), digest-verified and atomic (#147)

[bertysentry]

Closes #147. This is the third step of the remote file access family, after #146 (merged in #177) and #145 (merged in #179). #148 (CLI) comes next.

What changed

Downloads: the counterpart of uploadFile(...), with the same guarantees.

client.downloadFile("C:\\Windows\\Temp\\collect.log", Path.of("collect.log"));

long bytes = client.file("D:\\exports\\big.csv")
    .timeout(Duration.ofMinutes(10))
    .downloadTo(Path.of("big.csv"));   // bytes transferred, 0 when the local copy was already identical

• Both return the number of bytes transferred, or 0 when the local file already had the same content.
• When the local path is an existing directory, the file is written into it under its remote name, like cp. A remote name with a colon (an alternate data stream, or a drive-relative path) is refused there: the caller names the local file. The destination's directory is created when needed.
• offset(...)/length(...)/maxBytes(...) do not apply, the same as for digest(...).

How it works

1. Probe. One PowerShell invocation (RemoteFiles.probeScript) reports the size and the SHA-256 digest of the file as one base64 line (an 8-byte size, then the digest). The size is the number of bytes hashed, so size and digest always describe the same bytes. It opens the file exactly like the reads, with a share mode that tolerates other writers.
2. Skip. If the local file has the same size and digest, nothing is transferred.
3. Transfer. The content streams through openStream() into <name>.<random>.part next to the destination, the name cut to 64 characters so a long one still fits the file-name limits. Memory stays bounded.
4. Publish. The received bytes must match the probed size and digest. The file is then fsync'ed (FileChannel.force) and moved onto the destination with ATOMIC_MOVE, which replaces an existing file on Windows too (MoveFileEx with MOVEFILE_REPLACE_EXISTING). A replaced file keeps its POSIX permissions: the staging file is created with them (a creation attribute, never a later chmod that a reader could race).

The destination is never seen truncated. On a digest mismatch, a read error or a timeout, it keeps its previous content and the .part file is deleted. The local copy is always exactly the bytes the probe hashed: one consistent version of the file, never a mix of two. A change that reaches bytes not transferred yet fails the integrity check.

The timeout is a wall-clock deadline for the whole download. When it fires, the message says how far it got:

Download of D:\exports\big.csv from server01 timed out after PT30S, 41943040 of 104857600 bytes
transferred: raise the timeout for large files

Downloads are not resumable. That was decided in the issue, and the Javadoc and file-transfers.md both say so.

Quota retry. It lives in RemoteFiles.start(), reusing isRetryableQuotaRejection, QUOTA_RETRIES and QUOTA_RETRY_DELAY_MILLIS. So every remote-file script (reads, info(), list(), the download's two scripts) retries a start that the host's operation quota rejected. A retry is only attempted when its pause ends before the timeout.

Where this departs from the issue

• The probe does not use certutil. I measured it: certutil -hashfile fails with ERROR_SHARING_VIOLATION on a file another process holds open for writing, while the read's File.Open(..., 'ReadWrite') opens it fine. A certutil probe would make downloads fail on exactly the service logs openStream() can read. So ShellFileCopy.parseAnyDigest is not used. digestHex isn't either: it takes a byte[], which would break the bounded-memory requirement for the local digest.
• No README snippet. AGENTS.md restricts README changes, and Codex flagged the same snippet on Read remote files: client.file(path) with byte ranges, streaming and digests (#146) #177, where it was reverted. The feature is documented in file-transfers.md and files.md.

Worth a reviewer's eye

• The deadline and the final move exclude each other. The blocking terminals run in a worker that Utils.execute cancels when the deadline fires, and the caller gets the WinRMTimeoutException right away. A small RemoteFile.Publication handshake (two synchronized methods) decides between the worker's move and the caller's timeout:

  • a timeout is only reported when the destination was left untouched;
  • a deadline that fires mid-move waits for that move, and the download succeeds.

  The worker, usually blocked in a socket read, notices the cancellation at its next step and deletes its .part file, so that cleanup happens after the call returns.

• @SuppressFBWarnings("AT_NONATOMIC_64BIT_PRIMITIVE") on RemoteFile. SpotBugs treats any class with a java.util.concurrent.atomic local as multithreaded code, and then flags the offset/length/maxBytes setters. The AtomicLong progress counters are needed for a race-free read of the progress in the timeout message. RemoteFile is documented as not thread-safe.

• Permissions on replace. POSIX permissions are carried over, so a 0600 file stays 0600. Windows ACLs are not: the new file gets the directory's inherited ACL, as documented. Java has no ReplaceFile, and setAcl would turn inherited entries into explicit ones.

• Two local-filesystem guards.

  • Files.createDirectories is only called when the directory is missing: on JDK 11 and 17 it rejects an existing symbolic link to a directory, such as /tmp on macOS. Checked in the JDK sources; fixed in 21.
  • A root with no parent (for example a drive with no media) fails with a NoSuchFileException, wrapped in the usual WinRMClientException.

Tests

• RemoteFileTest (FakeWsmanServer):
  • verified download;
  • the range settings do not apply;
  • identical local file → probe only, returns 0;
  • directory destination;
  • digest mismatch → failure, no file at the destination;
  • timeout mid-transfer → message with 600 of 1000 bytes transferred, the destination keeps its previous content, and the .part file disappears once the abandoned transfer stops;
  • malformed probe;
  • local write failure;
  • a start rejected by the operation quota is retried, and no retry starts after the timeout;
  • an existing, different local file is replaced (the refresh case);
  • a size that disagrees with the probe fails even when the digest matches;
  • a destination name close to the 255-character limit still gets a valid staging file;
  • Publication: abandoning and publishing exclude each other;
  • a replaced file keeps its POSIX permissions (skipped on Windows; passed on Linux, WSL Ubuntu 22.04 with JDK 17);
  • a stream path (a.txt:meta) or a drive-relative path is refused for a directory destination, with nothing sent.
• RemoteFilesScriptTest (local powershell.exe): the probe reports the size and digest, including for an empty file and for a file held open for writing by another process.
• WinRMLiveTest:
  • uploadFile → downloadFile round trip, with every byte value under a non-ASCII name, into a directory, then skipped on a second download;
  • a 20 MiB (configurable) download checked against the host's digest.

Verification

• mvn clean verify site on JDK 17: 305 tests pass; 0 checkstyle, PMD and SpotBugs findings.
• Live, over HTTP with NTLM encryption, on tc-win2022 (Server 2022) and anaxagore (Server 2008 R2, PowerShell 2.0), both green:

Measurement	Result
20 MiB download	14.7–15.6 s (1.35–1.46 MB/s), probe included
64 MiB download, JVM limited to 32 MiB (-DargLine=-Xmx32m -Dwinrm.live.download.mib=64)	46 s
1 KiB download	~0.82 s
Skipping an identical copy	0.34–0.40 s

Docs

• file-transfers.md: new Downloading a file section covering how it works, the guarantees, "not resumable", the timeout, the measured throughput, and SMB for bulk data.
• files.md: short Downloading to a local file section, downloadTo in the timeout list, and the quota retry.
• Short mentions in index.md, preparing-the-host.md, timeouts-and-errors.md and migrating-from-winrm4j.md.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-24T23:22:09.572417Z	39e5c2d	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 620f5b8b90

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bertysentry]

@codex please review again

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 468e032db9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bertysentry]

@codex please review again

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6dbe72199b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bertysentry]

@codex please review again

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 090dd7833f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bertysentry]

@codex please review again

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39e5c2d41e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".