Skip to content

CLI: ls, stat, cat and get subcommands for remote files (#148) - #181

Merged
bertysentry merged 3 commits into
mainfrom
148-remote-file-access-44-cli-ls-stat-cat-and-get-subcommands
Sep 26, 2026
Merged

bertysentry merged 3 commits into
mainfrom
148-remote-file-access-44-cli-ls-stat-cat-and-get-subcommands

Conversation

@bertysentry

@bertysentry bertysentry commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Closes #148. This is the last step of the remote file access family, after #146 (merged in #177), #145 (#179) and #147 (#180).

What changed

Four CLI subcommands, each a thin mapping onto a library terminal:

Subcommand Library terminal
ls <directory> file(dir).list()...stream()
stat <path> file(path).info()
cat <file> file(path).openStream(), copied to stdout
get <file> [<local path>] file(path).downloadTo(...)
winrm-java -h server -u 'DOMAIN\user' -pf pw.txt ls 'C:\inetpub\logs' --glob '*.log' --recursive --depth 3
winrm-java ... stat 'C:\Windows\Temp\collect.log'
winrm-java ... cat 'D:\logs\huge.log' --offset -8192        # the last 8 KiB
winrm-java ... cat 'C:\legacy\report.txt' --charset windows-1252
winrm-java ... get 'C:\Windows\Temp\collect.log' ./collect.log
  • Options come after the subcommand, before or after the path. Remote paths are passed through untouched: the library quotes them.
  • ls writes one fixed, locale-independent line per entry, flushed as the host walks the tree:
    -a----      1048576 2026-01-02T03:04:05.6789012Z C:\inetpub\logs\LogFiles\u_ex260101.log
    
    The mode follows the Mode column of Windows PowerShell (darhsl). Then the size, the last write time, and the path. The time is ISO-8601 UTC with the 100 ns precision of Windows file times, always 28 characters, so it aligns and sorts as a string.
    Options: --glob, --recursive, --depth, --files-only, --directories-only, --modified-after <ISO-8601 date or date-time>, --min-size, --json.
  • stat prints the same fields, one name: value per line, or --json.
  • --json reuses JsonLinesWriter, which now writes numbers (size, attributes) as JSON numbers. WQL rows carry strings only, so the wql output is unchanged.
  • cat copies the bytes to stdout: no charset round trip, no newline translation. --offset (negative: from the end) and --length map to the ranged read. --charset decodes the file and prints it as text in the local console encoding.
  • get is the digest-verified download. Without a local path, the file goes into the current directory under its remote name. Nothing is printed on success.

Exit codes

New, documented in cli.md:

  • 1: ls listed the tree but could not read some directories; each one is reported on stderr.
  • 66 (EX_NOINPUT): remote path not found. stat gets it from the empty info(). ls, cat and get get it from the library's "Remote path not found" message, the only thing that tells this failure apart.
  • 74 (EX_IOERR): local I/O failure. Either stdout is closed or unwritable, or get hits a file-system error on its local file (a FileSystemException: access denied, a missing drive), which used to come out as a connection failure (69). A plain IOException such as a full disk keeps its message but still exits 69.

When stdout is closed early (cat ... | head), cat and ls stop the remote transfer instead of finishing it for nobody.

A library bug this found: closing a remote read early

Live, cat ... | head took the whole timeout (121 s with -t 120000), then exited 69 with "Read timed out".

The cause, measured on Windows Server 2022: closing a command early sends the terminate Signal. The service kills the process at once. But when the process is blocked writing a block larger than its output pipe, the service answers the Signal only when the Signal's OperationTimeout expires (WSManFault 2150858793). Every RemoteFile read writes 64 KB lines, for throughput. So openStream(), openReader() and a listing's stream() took the whole timeout to close early, then threw from close(), masking what the caller had read. A plain command writing 4,000-byte blocks closes in 18 ms; with 65,533-byte blocks, it takes 30 s.

Fix, in WsmanClient, which every caller goes through (separate commit, 9271931): the early-close Signal asks for a 1 s hold instead of the inactivity timeout. The expiry of that hold is not a failure: it is a complete exchange, the connection stays in sync, and the process is gone (checked from a second client). After the fix, an early close takes 1.0 s, and cat | head exits 74 in 1.1 s (0.98 s on anaxagore).

FakeWsmanServer answers every Signal instantly, which is why no test could catch this before.

Where this departs from the issue

  • --length without --offset is accepted: it reads from the start of the file, like head -c. The issue listed it as a usage error, but nothing about it is ambiguous.
  • get shows no progress. The library's downloadTo has no progress hook, and adding one means new public API. Java also cannot tell whether stderr is a terminal: System.console() looks at stdin and stdout. So get is silent, like cp. This is an easy follow-up if wanted.

Tests

  • CliArgumentsTest covers every new subcommand and option, and the usage errors: bad --depth, malformed --modified-after, unknown charset, a blank --glob, an option on the wrong subcommand, --files-only with --directories-only, a missing or extra path, an invalid local path, and -d/--env on a file subcommand.
  • WinRmCliTest runs end to end through the real connect factory against FakeWsmanServer:
    • ls streams: the first entry reaches stdout before the second Receive is even sent. The test also checks the long format, and that an inaccessible directory goes to stderr with exit 1.
    • The filters reach the host script, and the --json line is checked exactly.
    • stat text and JSON. A missing path exits 66 for stat, cat, ls and get.
    • cat is byte-exact: every byte value, CRLF, a lone LF and CR, invalid UTF-8, a UTF-16 BOM and Ctrl+Z, across two blocks and two Receives.
    • cat with a range and a charset.
    • A closed stdout exits 74 with a single Receive sent. The Signal is answered by the expiry fault, as a real host does.
    • get into a directory; an unwritable local destination exits 74.
    • The streaming and the closed-stdout tests were mutation-checked: buffering the listing, or not stopping on a closed output, makes them fail.
  • StreamingApiTest: an early terminate answered by the expiry of its 1 s hold is not a failure, and the connection stays usable.
  • JsonLinesWriterTest: numbers.

Verification

  • mvn clean verify site on JDK 17: 316 tests pass; 0 Checkstyle, PMD, CPD and SpotBugs findings.
  • Live over HTTP with NTLM on tc-win2022 (Server 2022) and anaxagore (Server 2008 R2, PowerShell 2.0), both green:
    • cat of a 3 MiB binary file (every byte value, then random data) redirected to a local file: identical SHA-256, in 3.5 s and 2.8 s.
    • Every ls filter, recursion and depth. --json with a non-ASCII file name comes out as exact UTF-8.
    • stat of a file, a directory, and a missing path (66). cat of a range, a tail, with a charset, and of a path with a space.
    • cat of a directory and ls of a file exit 70. ls and get of a missing path exit 66.
    • get without a local path, into a directory, and to a name with a space. A second get is skipped (about 1 s).
    • Acceptance: a recursive ls over a directory that another process holds open with no sharing reports it on stderr, leaves out its content, and exits 1. That lock produces a sharing violation even for an admin session, which holds the backup privilege.
    • cat ... | head -c 1000 exits 74 in about 1 s.

Docs

  • cli.md, the single source of truth, now has:

    • the synopsis and the subcommands table;
    • a File options table;
    • a Remote files section: output formats, streaming, byte-exactness, the --json fields, get, and the local shell quoting of Windows paths;
    • the timeout semantics and the exit codes.

    It also fixes four #Capitalized_Underscore anchors that are broken under Doxia 2.

  • The quoting notes were checked in Git Bash, cmd.exe and Windows PowerShell 5.1. That includes the trailing-backslash trap: in cmd.exe and PowerShell 5.1, "C:\my logs\" swallows its closing quote and merges the next arguments into it. It also includes that PowerShell 5.1's > re-encodes binary output (256 bytes became 519, with a BOM). The one line on PowerShell 7.4+ keeping the bytes comes from its release notes: PowerShell 7 isn't installed here.

  • --help: the four synopsis lines and one line per option. The details stay in the manual.

  • README is unchanged, per AGENTS.md (the example the issue asked for was removed after the Codex review). files.md gets a pointer to the manual.

🤖 Generated with Claude Code

bertysentry and others added 2 commits September 26, 2026 01:26
Closing a command early sends the terminate Signal. The service kills the
process at once, but when the process is blocked writing a block larger than
its output pipe, it answers the Signal only when the Signal's OperationTimeout
expires (WSManFault 2150858793, measured on Windows Server 2022). Every remote
file read writes 64 KB lines, so closing openStream(), openReader() or a
listing's stream() early took the whole timeout, then threw from close().

The early-close Signal now asks for a 1 s hold, and the expiry of that hold is
not a failure: it is a complete exchange, the connection stays in sync, and
the process is gone. Early close of a remote read: 30 s and an exception
before, 1 s after.

Found with the CLI's "cat ... | head" (#148).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Thin mappings onto the remote file access of the library:

- ls <dir>: file(dir).list()...stream(), one fixed line per entry (mode,
  size, ISO-8601 UTC time with 100 ns precision, path), written as the host
  walks the tree. --glob, --recursive, --depth, --files-only,
  --directories-only, --modified-after, --min-size, --json.
- stat <path>: file(path).info(), one field per line, or --json.
- cat <file>: file(path).openStream() copied to stdout byte for byte;
  --offset (negative: from the end), --length, --charset.
- get <file> [<local path>]: file(path).downloadTo(...), into the current
  directory under the remote name by default.

Options follow the subcommand; remote paths are passed through untouched.
JsonLinesWriter now writes numbers as JSON numbers (WQL rows carry strings
only, so the wql output is unchanged).

New exit codes: 1 when ls could not read some directories (each reported on
stderr), 66 for a remote path not found, 74 for a local I/O failure (stdout
closed, or the local file of get, previously reported as a connection
failure). A closed stdout ("| head") stops the remote transfer.

cli.md documents the subcommands, their options, output formats, exit codes
and the local shell quoting of Windows paths; it also fixes four anchors
broken under Doxia 2. --help lists the new options, README gets one example.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@bertysentry bertysentry linked an issue Sep 25, 2026 that may be closed by this pull request
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-26T00:12:44.999772Z f9bd2dc Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@bertysentry

Copy link
Copy Markdown
Contributor Author

@codex please review again

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@bertysentry

Copy link
Copy Markdown
Contributor Author

@codex please review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d02741cd1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread README.md Outdated
Comment thread src/main/java/org/metricshub/winrm/cli/CliArguments.java
Comment thread src/main/java/org/metricshub/winrm/cli/CliArguments.java
Comment thread src/main/java/org/metricshub/winrm/cli/WinRmCli.java
- README: the new example is removed. README changes only when a change
  alters what it already shows (AGENTS.md); cli.md covers the subcommands.
- A blank --glob value (e.g. an empty shell variable) is now a usage error
  (64), like every other file option, instead of reaching the library and
  exiting 70.
- cli.md: exit code 74 covers file-system errors on the local file of get;
  a plain IOException such as a full disk keeps its message but exits 69.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@bertysentry

Copy link
Copy Markdown
Contributor Author

@codex please review again

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: f9bd2dc84b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@bertysentry
bertysentry merged commit 4a5ac61 into main Sep 26, 2026
5 checks passed
@bertysentry
bertysentry deleted the 148-remote-file-access-44-cli-ls-stat-cat-and-get-subcommands branch September 26, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remote file access (4/4): CLI ls, stat, cat and get subcommands

1 participant