From a24f8bc4e25d77cb15003036fac44ce4ef434c3e Mon Sep 17 00:00:00 2001 From: Bertrand Martin Date: Thu, 24 Sep 2026 20:18:51 +0200 Subject: [PATCH 1/2] File properties and directory listing: info(), exists(), list() (#145) Second step of the remote file access family (#146, #145, #147, #148): client.file(path) gains info() (Optional, empty for a missing path), exists(), and list(), which returns a RemoteDirectoryListing with glob, recursive, maxDepth, filesOnly/directoriesOnly, modifiedAfter/modifiedBefore, minSize/maxSize, onInaccessible and timeout, ending with execute() (RemoteFileList: entries + inaccessible) or stream(). The host walks the tree with a small PowerShell script built on the .NET DirectoryInfo API and an explicit stack. It uses EnumerateFileSystemInfos where available and GetFileSystemInfos on PowerShell 2.0 / .NET 2.0. Every filter is evaluated on the host. Reparse points are reported but never descended into, so a junction loop terminates. A subdirectory that cannot be read becomes a "!" record and the walk goes on. Each entry is one ASCII line: plain integers for attributes, size and FileTimes, and only the path base64-encoded (UTF-8), so non-ASCII names do not depend on the console code page. A malformed record fails with a clear exception. - Long paths: absolute paths get the \\?\ (or \\?\UNC\) prefix where .NET accepts it (4.6.2+). The prefix is stripped from the reported paths. - The per-entry record is inlined instead of calling helper functions. A PowerShell function call costs about 45 us, which made large listings about 10 times slower (System32 went from 9.5 s to 2.3 s on 2022). - The script is kept compact so it fits the command line with paths up to about 450 characters. Like reads, a script that does not fit is refused rather than uploaded. - RemoteFiles: shared start/finish/blocking helpers for reads and metadata. PowerShell CLIXML progress records are filtered out of failure messages. - Docs: files.md (properties, listing, shell vs WMI, limitations), plus index.md, preparing-the-host.md, timeouts-and-errors.md, migrating-from-winrm4j.md and file-transfers.md. Tests: record parser and FakeWsmanServer tests (RemoteFileListingTest). The scripts run in local PowerShell (RemoteFilesScriptTest): depth, glob, filters, junction loop, a path over 300 characters, an access-denied subdirectory, non-ASCII names. WinRMLiveTest passes on Windows Server 2022 and 2008 R2. Co-Authored-By: Claude Opus 5.5 --- .../winrm/RemoteDirectoryListing.java | 315 ++++++++++++++ .../java/org/metricshub/winrm/RemoteFile.java | 96 +++-- .../org/metricshub/winrm/RemoteFileInfo.java | 231 +++++++++++ .../org/metricshub/winrm/RemoteFileList.java | 69 +++ .../org/metricshub/winrm/RemoteFiles.java | 392 ++++++++++++++++-- .../org/metricshub/winrm/WinRMClient.java | 11 +- src/site/markdown/file-transfers.md | 2 +- src/site/markdown/files.md | 173 +++++++- src/site/markdown/index.md | 25 +- src/site/markdown/migrating-from-winrm4j.md | 6 +- src/site/markdown/preparing-the-host.md | 13 +- src/site/markdown/timeouts-and-errors.md | 6 +- .../winrm/RemoteFileListingTest.java | 391 +++++++++++++++++ .../winrm/RemoteFilesScriptTest.java | 253 ++++++++++- .../org/metricshub/winrm/WinRMLiveTest.java | 111 +++++ 15 files changed, 1981 insertions(+), 113 deletions(-) create mode 100644 src/main/java/org/metricshub/winrm/RemoteDirectoryListing.java create mode 100644 src/main/java/org/metricshub/winrm/RemoteFileInfo.java create mode 100644 src/main/java/org/metricshub/winrm/RemoteFileList.java create mode 100644 src/test/java/org/metricshub/winrm/RemoteFileListingTest.java diff --git a/src/main/java/org/metricshub/winrm/RemoteDirectoryListing.java b/src/main/java/org/metricshub/winrm/RemoteDirectoryListing.java new file mode 100644 index 0000000..76f8570 --- /dev/null +++ b/src/main/java/org/metricshub/winrm/RemoteDirectoryListing.java @@ -0,0 +1,315 @@ +package org.metricshub.winrm; + +/*- + * ╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲ + * WinRM Java Client + * ჻჻჻჻჻჻ + * Copyright (C) 2023 - 2026 MetricsHub + * ჻჻჻჻჻჻ + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * ╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱ + */ + +import java.io.BufferedReader; +import java.time.Duration; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.Spliterator; +import java.util.Spliterators; +import java.util.function.Consumer; +import java.util.stream.Collectors; +import java.util.stream.Stream; +import java.util.stream.StreamSupport; +import org.metricshub.winrm.exceptions.WinRMClientException; +import org.metricshub.winrm.exceptions.WinRMTimeoutException; + +/** + * A listing of a remote directory being prepared, created by {@link RemoteFile#list()}: set the + * filters, then collect the result with {@link #execute()} or stream it with {@link #stream()}. + * + *
{@code
+ * RemoteFileList logs = client.file("C:\\inetpub\\logs").list()
+ * 	.glob("*.log")
+ * 	.recursive()
+ * 	.filesOnly()
+ * 	.modifiedAfter(Instant.now().minus(Duration.ofDays(1)))
+ * 	.execute();
+ * }
+ *

+ * The directory is walked on the host by a small PowerShell script, and every filter is + * evaluated there: only the matching entries travel. Without {@link #recursive()}, only the + * directory's own entries are listed. With it, every subdirectory is traversed — whatever the + * filters, which select what is reported, not where the walk goes — except reparse points + * (junctions, symbolic links, mount points), which are reported but never descended into, so a + * junction looping back to its parent terminates. A subdirectory that cannot be read (typically + * access denied) does not stop the walk: its path is reported through + * {@link #onInaccessible(Consumer)} and {@link RemoteFileList#inaccessible()}. + *

+ * A request is not thread-safe; configure it and call its terminals from one thread. + */ +public final class RemoteDirectoryListing { + + private static final int ALL = 0; + private static final int FILES = 1; + private static final int DIRECTORIES = 2; + + private final WinRMClient client; + private final String path; + private Duration timeout; + private String glob; + private boolean recursive; + private int maxDepth = Integer.MAX_VALUE; + private int type = ALL; + private long minSize; + private long maxSize = Long.MAX_VALUE; + private long after = -1; + private long before = Long.MAX_VALUE; + private Consumer onInaccessible = p -> {}; + + /** + * Create the request. + * + * @param client the client the directory is listed through + * @param path the path of the directory on the remote host + */ + RemoteDirectoryListing(final WinRMClient client, final String path) { + this.client = client; + this.path = path; + this.timeout = client.defaultTimeout(); + } + + /** + * Report only the entries whose name (not path) matches the given Windows wildcard + * pattern: {@code *} matches any sequence of characters (including none), {@code ?} exactly one + * character, and every other character only itself, case-insensitively. The whole name must + * match: {@code *.log} matches {@code app.log}, not {@code app.log.1} (unlike {@code dir}, + * which also matches 8.3 short names). With {@link #recursive()}, every directory is still + * traversed; the pattern only selects what is reported. + * + * @param glob the pattern, e.g. {@code *.log} or {@code u_ex??????.log} + * @return this request + */ + public RemoteDirectoryListing glob(final String glob) { + Utils.checkNonBlank(glob, "glob"); + this.glob = glob; + return this; + } + + /** + * List the whole tree below the directory instead of its own entries only, depth-first. + * Reparse points are reported but never descended into. + * + * @return this request + */ + public RemoteDirectoryListing recursive() { + this.recursive = true; + return this; + } + + /** + * List recursively, down to the given depth: 1 is the directory's own entries (the same as a + * non-recursive listing), 2 adds the entries of its subdirectories, and so on. Implies + * {@link #recursive()}. + * + * @param maxDepth the deepest level listed (at least 1) + * @return this request + */ + public RemoteDirectoryListing maxDepth(final int maxDepth) { + Utils.checkArgumentNotZeroOrNegative(maxDepth, "maxDepth"); + this.maxDepth = maxDepth; + this.recursive = true; + return this; + } + + /** + * Report files only (directories are still traversed with {@link #recursive()}). Replaces + * {@link #directoriesOnly()}. + * + * @return this request + */ + public RemoteDirectoryListing filesOnly() { + this.type = FILES; + return this; + } + + /** + * Report directories only. Replaces {@link #filesOnly()}. + * + * @return this request + */ + public RemoteDirectoryListing directoriesOnly() { + this.type = DIRECTORIES; + return this; + } + + /** + * Report only the entries last modified strictly after the given instant. + * + * @param instant the exclusive lower bound + * @return this request + */ + public RemoteDirectoryListing modifiedAfter(final Instant instant) { + Utils.checkNonNull(instant, "instant"); + this.after = RemoteFiles.toFileTime(instant); + return this; + } + + /** + * Report only the entries last modified strictly before the given instant. + * + * @param instant the exclusive upper bound + * @return this request + */ + public RemoteDirectoryListing modifiedBefore(final Instant instant) { + Utils.checkNonNull(instant, "instant"); + // Rounded up to the 100 ns FileTime unit, so a file time just below the bound still passes. + this.before = RemoteFiles.toFileTime(instant.plusNanos(99)); + return this; + } + + /** + * Report only the files of at least the given size. Directories have no size and are not + * filtered by it: combine with {@link #filesOnly()} to leave them out. + * + * @param bytes the smallest size, inclusive + * @return this request + */ + public RemoteDirectoryListing minSize(final long bytes) { + this.minSize = bytes; + return this; + } + + /** + * Report only the files of at most the given size. Directories have no size and are not + * filtered by it: combine with {@link #filesOnly()} to leave them out. + * + * @param bytes the largest size, inclusive + * @return this request + */ + public RemoteDirectoryListing maxSize(final long bytes) { + this.maxSize = bytes; + return this; + } + + /** + * Be told of each directory that could not be read (typically access denied): its path is + * given to the consumer as soon as the host reports it, and the walk goes on. Applies to both + * terminals; {@link #execute()} also collects the paths in + * {@link RemoteFileList#inaccessible()}. The directory being listed is not concerned: when it + * cannot be read, the listing fails. + * + * @param consumer receives the path of each directory that could not be read + * @return this request + */ + public RemoteDirectoryListing onInaccessible(final Consumer consumer) { + Utils.checkNonNull(consumer, "consumer"); + this.onInaccessible = consumer; + return this; + } + + /** + * Override the client's timeout for this request. For {@link #execute()} it is a wall-clock + * deadline for the whole listing; for {@link #stream()} it is an inactivity timeout. + * The host signals it is alive every second while it walks, so the inactivity timeout only + * trips when the host is unresponsive — or when reading a single directory takes longer. + * + * @param timeout the timeout (at least one millisecond) + * @return this request + */ + public RemoteDirectoryListing timeout(final Duration timeout) { + this.timeout = WinRMClient.checkPositive(timeout, "timeout"); + return this; + } + + /** + * List the directory and collect the result. + * + * @return the matching entries and the directories that could not be read + * @throws WinRMClientException when the directory cannot be listed (not found, a file, access + * denied, PowerShell unavailable or constrained) + * @throws WinRMTimeoutException when the timeout elapses first + */ + public RemoteFileList execute() { + return RemoteFiles.blocking(client, path, timeout, () -> { + final List inaccessible = new ArrayList<>(); + final Consumer collect = inaccessible::add; + try (Stream entries = stream(collect.andThen(onInaccessible))) { + return new RemoteFileList(entries.collect(Collectors.toList()), inaccessible); + } + }); + } + + /** + * List the directory and stream the entries as the host reports them: memory stays bounded + * whatever the size of the tree. + * + *

{@code
+	 * try (Stream tree = client.file("D:\\data").list().recursive().stream()) {
+	 * 	tree.filter(RemoteFileInfo::isDirectory).forEach(System.out::println);
+	 * }
+	 * }
+ *

+ * The stream must be closed — use try-with-resources. It holds the client's connection + * until it is exhausted or closed; closing it early stops the remote walk. The listing starts + * here, and the failures below are thrown from the stream's operations, when the host reports + * them. The timeout is an inactivity timeout (see {@link #timeout(Duration)}). + * + * @return a lazy, sequential stream of entries, to use with try-with-resources + * @throws WinRMClientException when the directory cannot be listed + * @throws WinRMTimeoutException when the host does not answer in time + */ + public Stream stream() { + return stream(onInaccessible); + } + + private Stream stream(final Consumer inaccessible) { + final String script = RemoteFiles.listScript( + path, + glob, + type, + minSize, + maxSize, + after, + before, + recursive ? maxDepth : 1 + ); + final RemoteProcess process = RemoteFiles.start(client, path, script, timeout); + final BufferedReader stdout = process.stdout(); + final String hostname = client.hostname(); + final Spliterator spliterator = new Spliterators.AbstractSpliterator( + Long.MAX_VALUE, + Spliterator.ORDERED | Spliterator.NONNULL | Spliterator.IMMUTABLE + ) { + private boolean ended; + + @Override + public boolean tryAdvance(final Consumer action) { + while (!ended) { + final String line = RemoteFiles.readLine(stdout); + if (line == null) { + ended = true; + RemoteFiles.finish(process, path, hostname, false); + } else if (line.startsWith("!")) { + inaccessible.accept(RemoteFiles.parseInaccessible(line.strip())); + } else if (!line.isBlank()) { + action.accept(RemoteFiles.parseEntry(line.strip())); + return true; + } + } + return false; + } + }; + return StreamSupport.stream(spliterator, false).onClose(process::close); + } +} diff --git a/src/main/java/org/metricshub/winrm/RemoteFile.java b/src/main/java/org/metricshub/winrm/RemoteFile.java index 79200cb..ab17a96 100644 --- a/src/main/java/org/metricshub/winrm/RemoteFile.java +++ b/src/main/java/org/metricshub/winrm/RemoteFile.java @@ -21,23 +21,23 @@ */ import java.io.BufferedReader; -import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.nio.charset.Charset; import java.time.Duration; +import java.util.Optional; import java.util.concurrent.Callable; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.TimeoutException; import org.metricshub.winrm.exceptions.WinRMClientException; import org.metricshub.winrm.exceptions.WinRMTimeoutException; /** - * A file on the remote host, obtained with {@link WinRMClient#file(String)}: read its content — - * whole, as a byte range, or as a stream — or compute its digest. Nothing is sent until a - * terminal ({@link #readBytes()}, {@link #readText(Charset)}, {@link #openStream()}, - * {@link #openReader(Charset)}, {@link #digest(String)}) is called. + * A file or directory on the remote host, obtained with {@link WinRMClient#file(String)}: read a + * file's content — whole, as a byte range, or as a stream — or compute its digest, get its + * properties ({@link #info()}, {@link #exists()}), or {@link #list()} a directory. Nothing is sent + * until a terminal ({@link #readBytes()}, {@link #readText(Charset)}, {@link #openStream()}, + * {@link #openReader(Charset)}, {@link #digest(String)}, {@link #info()}, {@link #exists()}) is + * called. * *

{@code
  * byte[] content = client.file("C:\\Windows\\Temp\\collect.bin").readBytes();
@@ -284,35 +284,69 @@ public String digest(final String algorithm) {
 		});
 	}
 
+	/**
+	 * Get the properties of the file or directory: size, timestamps, attributes. A path that does
+	 * not exist is not an error: the result is empty. The {@link #offset(long)} and
+	 * {@link #length(long)} settings do not apply.
+	 *
+	 * @return the properties, or empty when the path does not exist
+	 * @throws WinRMClientException when the properties cannot be read (access denied, invalid
+	 *         path, PowerShell unavailable or constrained)
+	 * @throws WinRMTimeoutException when the timeout elapses first
+	 */
+	public Optional info() {
+		final String script = RemoteFiles.infoScript(path);
+		return blocking(() -> {
+			final RemoteProcess process = RemoteFiles.start(client, path, script, timeout);
+			try {
+				final RemoteFileInfo info = process
+					.stdout()
+					.lines()
+					.filter(line -> !line.isBlank())
+					.map(line -> RemoteFiles.parseEntry(line.strip()))
+					.reduce((first, last) -> last)
+					.orElse(null);
+				return RemoteFiles.finish(process, path, client.hostname(), true) == 0
+					? Optional.ofNullable(info)
+					: Optional.empty();
+			} finally {
+				process.close();
+			}
+		});
+	}
+
+	/**
+	 * Tell whether the file or directory exists: {@code info().isPresent()}.
+	 *
+	 * @return {@code true} when the path exists
+	 * @throws WinRMClientException when the path cannot be checked (e.g. access denied)
+	 * @throws WinRMTimeoutException when the timeout elapses first
+	 */
+	public boolean exists() {
+		return info().isPresent();
+	}
+
+	/**
+	 * Prepare the listing of this path, a directory: set its filters, then call
+	 * {@link RemoteDirectoryListing#execute()} or {@link RemoteDirectoryListing#stream()}. The
+	 * timeout of this request, when set, carries over.
+	 *
+	 * 
{@code
+	 * RemoteFileList logs = client.file("C:\\inetpub\\logs").list().glob("*.log").recursive().execute();
+	 * }
+ * + * @return the listing request + */ + public RemoteDirectoryListing list() { + return new RemoteDirectoryListing(client, path).timeout(timeout); + } + /** Start the remote read of the configured offset and the given length ({@code -1}: to the end). */ private InputStream open(final long readLength) { return RemoteFiles.open(client, path, RemoteFiles.readScript(path, offset, readLength), timeout); } - /** - * Run a blocking terminal under the wall-clock deadline: a worker runs the exchange and is - * cancelled when the deadline fires, exactly like {@link CommandRequest#execute()}. - */ private T blocking(final Callable task) { - try { - return Utils.execute(task, WinRMClient.toMillis(timeout)); - } catch (final TimeoutException e) { - throw new WinRMTimeoutException( - String.format("Reading remote file %s timed out after %s on %s", path, timeout, client.hostname()), - e - ); - } catch (final InterruptedException e) { - Thread.currentThread().interrupt(); - throw new WinRMClientException(e.getMessage(), e); - } catch (final ExecutionException e) { - final Throwable cause = e.getCause() != null ? e.getCause() : e; - if (cause instanceof RuntimeException) { - throw (RuntimeException) cause; - } - if (cause instanceof IOException) { - throw new WinRMClientException(cause.getMessage(), cause); - } - throw new WinRMClientException(String.valueOf(cause.getMessage()), cause); - } + return RemoteFiles.blocking(client, path, timeout, task); } } diff --git a/src/main/java/org/metricshub/winrm/RemoteFileInfo.java b/src/main/java/org/metricshub/winrm/RemoteFileInfo.java new file mode 100644 index 0000000..719588d --- /dev/null +++ b/src/main/java/org/metricshub/winrm/RemoteFileInfo.java @@ -0,0 +1,231 @@ +package org.metricshub.winrm; + +/*- + * ╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲ + * WinRM Java Client + * ჻჻჻჻჻჻ + * Copyright (C) 2023 - 2026 MetricsHub + * ჻჻჻჻჻჻ + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * ╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱ + */ + +import java.time.Instant; +import java.util.Objects; + +/** + * The properties of a file or directory on the remote host, as returned by + * {@link RemoteFile#info()} and {@link RemoteDirectoryListing}: an immutable value. + *

+ * Timestamps come from the host's UTC file times, with their 100-nanosecond precision. The + * attribute accessors decode the Windows {@code FileAttributes} bit flags; {@link #attributes()} + * returns the raw value for anything else (e.g. {@code 0x800} compressed, {@code 0x4000} + * encrypted). + */ +public final class RemoteFileInfo { + + private static final int READ_ONLY = 0x1; + private static final int HIDDEN = 0x2; + private static final int SYSTEM = 0x4; + private static final int DIRECTORY = 0x10; + private static final int ARCHIVE = 0x20; + private static final int REPARSE_POINT = 0x400; + + private final String path; + private final int attributes; + private final long size; + private final Instant lastModified; + private final Instant created; + private final Instant lastAccessed; + + /** + * Create the value. + * + * @param path the full path, as reported by the host + * @param attributes the raw {@code FileAttributes} value + * @param size the size in bytes (0 for a directory) + * @param lastModified the last write time + * @param created the creation time + * @param lastAccessed the last access time + */ + RemoteFileInfo( + final String path, + final int attributes, + final long size, + final Instant lastModified, + final Instant created, + final Instant lastAccessed + ) { + this.path = path; + this.attributes = attributes; + this.size = size; + this.lastModified = lastModified; + this.created = created; + this.lastAccessed = lastAccessed; + } + + /** + * Get the full path of the entry, as reported by the host (absolute, with backslashes). + * + * @return the path + */ + public String path() { + return path; + } + + /** + * Get the last element of the path: the file or directory name ({@code C:} for a drive root). + * + * @return the name + */ + public String name() { + int end = path.length(); + while (end > 1 && path.charAt(end - 1) == '\\') { + end--; + } + return path.substring(path.lastIndexOf('\\', end - 1) + 1, end); + } + + /** + * Tell whether the entry is a directory. + * + * @return {@code true} for a directory (including a junction or a directory symbolic link) + */ + public boolean isDirectory() { + return (attributes & DIRECTORY) != 0; + } + + /** + * Get the size of the file. + * + * @return the size in bytes, 0 for a directory + */ + public long size() { + return size; + } + + /** + * Get the time of the last write. + * + * @return the last modification time + */ + public Instant lastModified() { + return lastModified; + } + + /** + * Get the creation time. + * + * @return the creation time + */ + public Instant created() { + return created; + } + + /** + * Get the time of the last access. Windows updates it lazily, or not at all when last access + * updates are disabled (the default on many versions): do not rely on it. + * + * @return the last access time + */ + public Instant lastAccessed() { + return lastAccessed; + } + + /** + * Tell whether the entry is hidden. + * + * @return {@code true} when the hidden attribute is set + */ + public boolean isHidden() { + return (attributes & HIDDEN) != 0; + } + + /** + * Tell whether the entry is a system file or directory. + * + * @return {@code true} when the system attribute is set + */ + public boolean isSystem() { + return (attributes & SYSTEM) != 0; + } + + /** + * Tell whether the entry is read-only. + * + * @return {@code true} when the read-only attribute is set + */ + public boolean isReadOnly() { + return (attributes & READ_ONLY) != 0; + } + + /** + * Tell whether the entry is marked for archiving (backup). + * + * @return {@code true} when the archive attribute is set + */ + public boolean isArchive() { + return (attributes & ARCHIVE) != 0; + } + + /** + * Tell whether the entry is a reparse point: a junction, a symbolic link, a mount point, or a + * cloud placeholder. A listing reports reparse points but never descends into them. + * + * @return {@code true} when the reparse point attribute is set + */ + public boolean isReparsePoint() { + return (attributes & REPARSE_POINT) != 0; + } + + /** + * Get the raw Windows {@code FileAttributes} bit flags. + * + * @return the attributes + */ + public int attributes() { + return attributes; + } + + @Override + public boolean equals(final Object other) { + if (this == other) { + return true; + } + if (!(other instanceof RemoteFileInfo)) { + return false; + } + final RemoteFileInfo that = (RemoteFileInfo) other; + return attributes == that.attributes + && + size == that.size + && + path.equals(that.path) + && + lastModified.equals(that.lastModified) + && + created.equals(that.created) + && + lastAccessed.equals(that.lastAccessed); + } + + @Override + public int hashCode() { + return Objects.hash(path, attributes, size, lastModified, created, lastAccessed); + } + + @Override + public String toString() { + return String.format("%s %12d %s %s", isDirectory() ? "d" : "-", size, lastModified, path); + } +} diff --git a/src/main/java/org/metricshub/winrm/RemoteFileList.java b/src/main/java/org/metricshub/winrm/RemoteFileList.java new file mode 100644 index 0000000..083b54f --- /dev/null +++ b/src/main/java/org/metricshub/winrm/RemoteFileList.java @@ -0,0 +1,69 @@ +package org.metricshub.winrm; + +/*- + * ╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲ + * WinRM Java Client + * ჻჻჻჻჻჻ + * Copyright (C) 2023 - 2026 MetricsHub + * ჻჻჻჻჻჻ + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * ╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱ + */ + +import java.util.List; + +/** + * The result of {@link RemoteDirectoryListing#execute()}: the matching entries, and the + * directories that could not be read. + */ +public final class RemoteFileList { + + private final List entries; + private final List inaccessible; + + /** + * Create the result. + * + * @param entries the matching entries + * @param inaccessible the paths of the directories that could not be read + */ + RemoteFileList(final List entries, final List inaccessible) { + this.entries = List.copyOf(entries); + this.inaccessible = List.copyOf(inaccessible); + } + + /** + * Get the matching entries, in the order the host walked them: the entries of a directory + * together, depth-first. + * + * @return an unmodifiable list, empty for an empty directory or when nothing matches + */ + public List entries() { + return entries; + } + + /** + * Get the directories that could not be read (typically access denied): their content is + * missing from {@link #entries()}, while the walk went on elsewhere. + * + * @return an unmodifiable list of paths, empty when the whole tree was read + */ + public List inaccessible() { + return inaccessible; + } + + @Override + public String toString() { + return entries.size() + " entries" + (inaccessible.isEmpty() ? "" : ", " + inaccessible.size() + " inaccessible"); + } +} diff --git a/src/main/java/org/metricshub/winrm/RemoteFiles.java b/src/main/java/org/metricshub/winrm/RemoteFiles.java index fa35a6b..5344e71 100644 --- a/src/main/java/org/metricshub/winrm/RemoteFiles.java +++ b/src/main/java/org/metricshub/winrm/RemoteFiles.java @@ -25,16 +25,23 @@ import java.io.InputStream; import java.nio.charset.StandardCharsets; import java.time.Duration; +import java.time.Instant; import java.util.Base64; import java.util.Objects; import java.util.Set; +import java.util.concurrent.Callable; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeoutException; import java.util.stream.Collectors; import org.metricshub.winrm.exceptions.WinRMClientException; +import org.metricshub.winrm.exceptions.WinRMTimeoutException; /** * The remote file access primitive: small PowerShell scripts that open a remote file and write * what the caller asked for as base64 lines on stdout, and the {@link InputStream} that - * decodes those lines as the output chunks arrive. + * decodes those lines as the output chunks arrive. The metadata scripts ({@link #infoScript}, + * {@link #listScript}) write one ASCII record per entry instead, parsed by {@link #parseEntry} + * and {@link #parseInaccessible}: plain integers, and only the path base64-encoded. *

* Base64 is what makes a text console a binary-safe channel: every byte value survives, and the * output is pure ASCII, so recovering the exact bytes never depends on the remote console code @@ -63,6 +70,9 @@ private RemoteFiles() {} /** The path is a directory, not a file. */ static final int EXIT_IS_DIRECTORY = 6; + /** The path is a file, not a directory (listing). */ + static final int EXIT_NOT_DIRECTORY = 7; + /** The exit code of {@code cmd.exe} when {@code powershell.exe} cannot be found. */ static final int EXIT_COMMAND_NOT_FOUND = 9009; @@ -86,13 +96,11 @@ private RemoteFiles() {} /** * The opening shared by every script: check the language mode, define {@code fail}, which maps * a .NET failure to the documented exit code (a sharing violation is HRESULT - * {@code 0x80070020}, 32; a byte-range lock violation {@code 0x80070021}, 33), then open the - * file with a share mode that tolerates other writers ({@code ReadWrite}) — a log written by a - * running service is the most common thing to read, and {@code File.OpenRead} fails on it. - * {@code exit} in a function exits the whole script. {@code %s} is the base64 of the path's - * UTF-8 bytes. + * {@code 0x80070020}, 32; a byte-range lock violation {@code 0x80070021}, 33), and decode the + * path into {@code $p}. {@code exit} in a function exits the whole script. {@code %s} is the + * base64 of the path's UTF-8 bytes. */ - private static final String OPEN_FILE = "if($ExecutionContext.SessionState.LanguageMode -ne 'FullLanguage'){exit 5};" + private static final String PREAMBLE = "if($ExecutionContext.SessionState.LanguageMode -ne 'FullLanguage'){exit 5};" + "$ErrorActionPreference='Stop';" + "function fail($e){if($e.InnerException){$e=$e.InnerException};" + @@ -102,10 +110,84 @@ private RemoteFiles() {} "$h=[Runtime.InteropServices.Marshal]::GetHRForException($e) -band 0xFFFF;" + "if($h -eq 32 -or $h -eq 33){exit 3};" + "exit 1};" + - "$p=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('%s'));" + + "$p=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('%s'));"; + + /** + * Open the file with a share mode that tolerates other writers ({@code ReadWrite}) — a log + * written by a running service is the most common thing to read, and {@code File.OpenRead} + * fails on it. + */ + private static final String OPEN_FILE = PREAMBLE + "if([IO.Directory]::Exists($p)){exit 6};" + "try{$f=[IO.File]::Open($p,'Open','Read','ReadWrite')}catch{fail $_.Exception};"; + /** + * The opening of the metadata scripts, after {@link #PREAMBLE}: the record writer and the path + * resolution. Kept terse: the script must fit the command line with room for long paths. + *

    + *
  • {@code out} writes the buffered records and a newline to the raw stdout stream as one + * write (see {@link #READ_RANGE} for why); with nothing buffered, the bare newline is a + * keepalive the parser skips, so a long walk that matches nothing does not trip the inactivity + * timeout; + *
  • the path is made absolute, then given the {@code \\?\} prefix ({@code \\?\UNC\} for a + * UNC path) that lifts the 260-character {@code MAX_PATH} limit — where .NET accepts it (4.6.2 + * and later: older versions reject {@code GetFullPath('\\?\...')}, and paths stay limited + * there); {@code $pn} and {@code $pa} strip the prefix from the reported paths again; + *
  • {@code $a} gets the attributes of the path, failing with the documented exit codes. + *
+ */ + private static final String METADATA = PREAMBLE + + "$o=[Console]::OpenStandardOutput();$u=[Text.Encoding]::UTF8;$w=New-Object Text.StringBuilder;$t=0;" + + "function out{$y=$u.GetBytes(\"$w`n\");$o.Write($y,0,$y.Length);$o.Flush();$w.Length=0;" + + "$script:t=[Environment]::TickCount};" + + "try{$q=[IO.Path]::GetFullPath($p);" + + "try{$q=[IO.Path]::GetFullPath((($q-replace'^\\\\\\\\(?=[^\\\\?.])','\\\\?\\UNC\\')" + + "-replace'^(?=[A-Za-z]:\\\\)','\\\\?\\'))}catch{};" + + "$a=[int][IO.File]::GetAttributes($q)}catch{fail $_.Exception};" + + "$pn=0;$pa='';if($q -match '^\\\\\\\\\\?\\\\(UNC)?'){$pn=$matches[0].Length;if($matches[1]){$pa='\\'}};"; + + /** + * Buffer the {@code F} record of the {@code FileSystemInfo} in {@code $e}: plain integers + * (PowerShell expands them with the invariant culture) and the base64 of the UTF-8 path. + * Inlined wherever it is used: a PowerShell function call costs more than the record itself + * (about 45 µs, measured), which would make a large listing 10 times slower. + */ + private static final String RECORD = "$l=0;if($e -is [IO.FileInfo]){$l=$e.Length};" + + "[void]$w.Append(\"F $([int]$e.Attributes) $l $($e.LastWriteTimeUtc.ToFileTimeUtc()) " + + "$($e.CreationTimeUtc.ToFileTimeUtc()) $($e.LastAccessTimeUtc.ToFileTimeUtc()) " + + "$([Convert]::ToBase64String($u.GetBytes($pa+$e.FullName.Substring($pn))))`n\")"; + + /** Write the record of the path itself, a file or a directory. */ + private static final String INFO = "try{if($a -band 16){$e=New-Object IO.DirectoryInfo $q}" + + "else{$e=New-Object IO.FileInfo $q};" + RECORD + ";out}catch{fail $_.Exception}"; + + /** + * Walk the directory with an explicit stack of {@code (DirectoryInfo, depth)} pairs: + * {@code EnumerateFileSystemInfos} where it exists (.NET 4), {@code GetFileSystemInfos} + * otherwise (PowerShell 2.0 runs on .NET 2.0). Every entry is evaluated against the filters + * here, on the host; a directory is pushed for traversal whatever the filters, unless it is a + * reparse point (junction, symbolic link: never followed, so a junction loop terminates) or + * at the maximum depth. A directory that cannot be read becomes a {@code !} record and the walk + * goes on — except the root itself, which fails the listing. {@code %s} and {@code %d} are the + * base64 name regex (see {@link #globRegex(String)}; empty: any name), the type (0: all, 1: + * files, 2: directories), the size bounds, the exclusive FileTime bounds and the maximum depth. + */ + private static final String LIST = "if(!($a -band 16)){exit 7};" + + "$g=$u.GetString([Convert]::FromBase64String('%s'));$k=%d;$mn=%d;$mx=%d;$ta=%d;$tb=%d;$md=%d;" + + "$m=[IO.DirectoryInfo].GetMethod('EnumerateFileSystemInfos',[Type[]]@());" + + "$s=New-Object Collections.Stack;$s.Push(@((New-Object IO.DirectoryInfo $q),1));" + + "while($s.Count){$d,$n=$s.Pop();" + + "try{if($m){$c=$d.EnumerateFileSystemInfos()}else{$c=$d.GetFileSystemInfos()};" + + "foreach($e in $c){$i=$e -is [IO.DirectoryInfo];" + + "if($i -and !([int]$e.Attributes -band 1024) -and $n -lt $md){$s.Push(@($e,($n+1)))};" + + "$f=$e.LastWriteTimeUtc.ToFileTimeUtc();" + + "if(($k -eq 0 -or ($k -eq 2) -eq $i) -and ($i -or ($e.Length -ge $mn -and $e.Length -le $mx)) -and " + + "$f -gt $ta -and $f -lt $tb -and $e.Name -match $g){" + RECORD + "};" + + "if($w.Length -gt 32000 -or [Environment]::TickCount-$t -gt 1000){out}}}" + + "catch{if($n -eq 1){fail $_.Exception};$x=$_.Exception;if($x.InnerException){$x=$x.InnerException};" + + "[void]$w.Append(\"! $([Convert]::ToBase64String($u.GetBytes($pa+$d.FullName.Substring($pn)))) " + + "$([Convert]::ToBase64String($u.GetBytes($x.Message)))`n\")}};out"; + /** * Read a byte range: resolve a negative offset from the size of the open stream (so a * growing log is tailed from its current end), seek, and write at most {@code length} bytes @@ -160,7 +242,167 @@ static String digestScript(final String path, final String algorithm) { } private static String openFile(final String path) { - return String.format(OPEN_FILE, Base64.getEncoder().encodeToString(path.getBytes(StandardCharsets.UTF_8))); + return String.format(OPEN_FILE, base64(path)); + } + + private static String base64(final String text) { + return Base64.getEncoder().encodeToString(text.getBytes(StandardCharsets.UTF_8)); + } + + /** + * Build the script writing the {@code F} record of the path itself. + * + * @param path the remote file or directory + * @return the PowerShell script + */ + static String infoScript(final String path) { + return String.format(METADATA, base64(path)) + INFO; + } + + /** + * Build the script listing a directory: one {@code F} record per matching entry, one {@code !} + * record per directory that could not be read. + * + * @param path the remote directory + * @param glob the wildcard pattern the entry names must match (see {@link #globRegex}), or + * {@code null} + * @param type 0: files and directories, 1: files only, 2: directories only + * @param minSize the smallest file size, inclusive + * @param maxSize the largest file size, inclusive + * @param after the exclusive lower bound of the last write time, as a FileTime + * @param before the exclusive upper bound of the last write time, as a FileTime + * @param maxDepth the deepest level listed, 1 being the directory's own entries + * @return the PowerShell script + */ + static String listScript( + final String path, + final String glob, + final int type, + final long minSize, + final long maxSize, + final long after, + final long before, + final int maxDepth + ) { + return String.format(METADATA, base64(path)) + + String.format(LIST, glob == null ? "" : base64(globRegex(glob)), type, minSize, maxSize, after, before, maxDepth); + } + + /** + * Translate a Windows wildcard pattern into the anchored .NET regex the listing script matches + * names with (PowerShell's {@code -match}: case-insensitive): {@code *} is any sequence, + * {@code ?} any one character, and every other character is literal. ASCII punctuation is + * escaped; letters, digits, {@code _} and non-ASCII characters are never special, and .NET + * rejects escaping them. + * + * @param glob the wildcard pattern + * @return the regex + */ + static String globRegex(final String glob) { + final StringBuilder regex = new StringBuilder("^"); + for (final char c : glob.toCharArray()) { + if (c == '*') { + regex.append(".*"); + } else if (c == '?') { + regex.append('.'); + } else if (c < 128 && !Character.isLetterOrDigit(c) && c != '_') { + regex.append('\\').append(c); + } else { + regex.append(c); + } + } + return regex.append('$').toString(); + } + + /** The FileTime of the Unix epoch: 100-nanosecond intervals since 1601-01-01 UTC. */ + private static final long FILETIME_EPOCH = 116_444_736_000_000_000L; + + /** 100-nanosecond intervals per second. */ + private static final long FILETIME_PER_SECOND = 10_000_000L; + + /** + * Convert a Windows FileTime to an {@link Instant}. + * + * @param fileTime 100-nanosecond intervals since 1601-01-01 UTC + * @return the instant + */ + static Instant fromFileTime(final long fileTime) { + final long sinceEpoch = fileTime - FILETIME_EPOCH; + return Instant.ofEpochSecond( + Math.floorDiv(sinceEpoch, FILETIME_PER_SECOND), + Math.floorMod(sinceEpoch, FILETIME_PER_SECOND) * 100 + ); + } + + /** + * Convert an {@link Instant} to a Windows FileTime, truncated to 100 nanoseconds. + * + * @param instant the instant + * @return 100-nanosecond intervals since 1601-01-01 UTC + */ + static long toFileTime(final Instant instant) { + return FILETIME_EPOCH + instant.getEpochSecond() * FILETIME_PER_SECOND + instant.getNano() / 100; + } + + /** + * Parse an {@code F} record: + * {@code F }. + * + * @param line the record + * @return the entry + * @throws WinRMClientException when the record is truncated or malformed + */ + static RemoteFileInfo parseEntry(final String line) { + final String[] fields = line.split(" ", 7); + if (fields.length != 7 || !"F".equals(fields[0])) { + throw malformed(line, null); + } + try { + return new RemoteFileInfo( + decode(fields[6]), + Integer.parseInt(fields[1]), + Long.parseLong(fields[2]), + fromFileTime(Long.parseLong(fields[3])), + fromFileTime(Long.parseLong(fields[4])), + fromFileTime(Long.parseLong(fields[5])) + ); + } catch (final IllegalArgumentException e) { + throw malformed(line, e); + } + } + + /** + * Parse a {@code !} record: {@code ! }. + * + * @param line the record + * @return the path of the directory that could not be read + * @throws WinRMClientException when the record is truncated or malformed + */ + static String parseInaccessible(final String line) { + final String[] fields = line.split(" ", 3); + if (fields.length != 3 || !"!".equals(fields[0])) { + throw malformed(line, null); + } + try { + decode(fields[2]); + return decode(fields[1]); + } catch (final IllegalArgumentException e) { + throw malformed(line, e); + } + } + + private static String decode(final String base64) { + if (base64.isEmpty()) { + throw new IllegalArgumentException("empty field"); + } + return new String(Base64.getDecoder().decode(base64), StandardCharsets.UTF_8); + } + + private static WinRMClientException malformed(final String line, final Throwable cause) { + return new WinRMClientException( + "Malformed remote file record: " + (line.length() > 120 ? line.substring(0, 120) + "..." : line), + cause + ); } /** @@ -174,23 +416,115 @@ private static String openFile(final String path) { * @return the decoded stream; it must be closed */ static InputStream open(final WinRMClient client, final String path, final String script, final Duration timeout) { + final RemoteProcess process = start(client, path, script, timeout); + try { + return new DecodingStream(process, path, client.hostname()); + } catch (final RuntimeException e) { + process.close(); + throw e; + } + } + + /** + * Start a script. It must fit the command line: {@code powerShell(...)} would transparently + * fall back to uploading it as a file, and file access must stay read-only on the host (no + * files written, no certutil), so a script too long is refused instead. + * + * @param client the client to run the script on + * @param path the remote path, for the error messages + * @param script the script + * @param timeout the inactivity timeout of the process + * @return the running process; it must be closed + */ + static RemoteProcess start(final WinRMClient client, final String path, final String script, final Duration timeout) { if (CommandRequest.encodePowerShell(script) == null) { - // powerShell(...) would transparently fall back to uploading the script as a file: a read - // must stay read-only on the host (no files written, no certutil), so refuse instead. throw new WinRMClientException( String.format( - "Remote path too long to read on %s (%d characters): the reader script must fit the remote command line", + "Remote path too long to access on %s (%d characters): the script must fit the remote command line", client.hostname(), path.length() ) ); } - final RemoteProcess process = client.powerShell(script).timeout(timeout).start(); + return client.powerShell(script).timeout(timeout).start(); + } + + /** + * Read the next stdout line of a script. + * + * @param stdout the script's stdout + * @return the line, or {@code null} at the end of the output + */ + static String readLine(final BufferedReader stdout) { try { - return new DecodingStream(process, path, client.hostname()); - } catch (final RuntimeException e) { - process.close(); - throw e; + return stdout.readLine(); + } catch (final IOException e) { + // Unreachable: the reader reports failures unchecked (see RemoteProcess). + throw new WinRMClientException(e.getMessage(), e); + } + } + + /** + * The output of a script ended: collect the exit code and release the connection. + * + * @param process the script's process + * @param path the remote path, for the error messages + * @param hostname the remote host, for the error messages + * @param notFoundIsEmpty whether {@link #EXIT_NOT_FOUND} is a normal outcome, not a failure + * @return the exit code, 0 or {@link #EXIT_NOT_FOUND} when {@code notFoundIsEmpty} + * @throws WinRMClientException for any other non-zero exit code + */ + static int finish( + final RemoteProcess process, + final String path, + final String hostname, + final boolean notFoundIsEmpty + ) { + final int exitCode = process.waitFor(); + // PowerShell serializes its progress records to a redirected stderr as CLIXML (e.g. "Preparing + // modules for first use"): noise, not the error message. + final String stderr = exitCode == 0 + ? "" + : process + .stderr() + .lines() + .filter(line -> !line.startsWith("#< CLIXML") && !line.startsWith(" the result type + * @param client the client, for the error messages + * @param path the remote path, for the error messages + * @param timeout the deadline + * @param task the exchange + * @return the result of the task + */ + static T blocking(final WinRMClient client, final String path, final Duration timeout, final Callable task) { + try { + return Utils.execute(task, WinRMClient.toMillis(timeout)); + } catch (final TimeoutException e) { + throw new WinRMTimeoutException( + String.format("Accessing remote path %s timed out after %s on %s", path, timeout, client.hostname()), + e + ); + } catch (final InterruptedException e) { + Thread.currentThread().interrupt(); + throw new WinRMClientException(e.getMessage(), e); + } catch (final ExecutionException e) { + final Throwable cause = e.getCause() != null ? e.getCause() : e; + if (cause instanceof RuntimeException) { + throw (RuntimeException) cause; + } + throw new WinRMClientException(String.valueOf(cause.getMessage()), cause); } } @@ -211,7 +545,7 @@ static WinRMClientException failure( ) { switch (exitCode) { case EXIT_NOT_FOUND: - return new WinRMClientException(String.format("Remote file not found on %s: %s", hostname, path)); + return new WinRMClientException(String.format("Remote path not found on %s: %s", hostname, path)); case EXIT_SHARING_VIOLATION: return new WinRMClientException( String.format( @@ -229,6 +563,8 @@ static WinRMClientException failure( hostname ) ); + case EXIT_NOT_DIRECTORY: + return new WinRMClientException(String.format("Remote path %s on %s is a file, not a directory", path, hostname)); case EXIT_IS_DIRECTORY: return new WinRMClientException(String.format("Remote path %s on %s is a directory, not a file", path, hostname)); case EXIT_COMMAND_NOT_FOUND: @@ -238,7 +574,7 @@ static WinRMClientException failure( default: return new WinRMClientException( String.format( - "Failed to read remote file %s on %s (exit code %d)%s", + "Failed to access remote path %s on %s (exit code %d)%s", path, hostname, exitCode, @@ -302,7 +638,7 @@ private boolean fill() { if (ended) { return false; } - final String line = readLine(); + final String line = readLine(stdout); if (line == null) { end(); return false; @@ -329,24 +665,10 @@ private boolean fill() { return true; } - private String readLine() { - try { - return stdout.readLine(); - } catch (final IOException e) { - // Unreachable: the reader reports failures unchecked (see RemoteProcess). - throw new WinRMClientException(e.getMessage(), e); - } - } - /** The output ended: collect the exit code, release the connection, report a failure. */ private void end() { ended = true; - final int exitCode = process.waitFor(); - final String stderr = exitCode == 0 ? "" : process.stderr().lines().collect(Collectors.joining("\n")); - process.close(); - if (exitCode != 0) { - throw failure(exitCode, path, hostname, stderr); - } + finish(process, path, hostname, false); } @Override diff --git a/src/main/java/org/metricshub/winrm/WinRMClient.java b/src/main/java/org/metricshub/winrm/WinRMClient.java index 8503f1d..c17611c 100644 --- a/src/main/java/org/metricshub/winrm/WinRMClient.java +++ b/src/main/java/org/metricshub/winrm/WinRMClient.java @@ -171,16 +171,19 @@ public CommandRequest powerShell(final String script) { } /** - * Prepare access to a file on the remote host: read its content (whole, as a byte range, or as - * a stream) or compute its digest, through the WinRM connection itself. Nothing is sent until - * a terminal of the returned {@link RemoteFile} is called. + * Prepare access to a file or directory on the remote host, through the WinRM connection + * itself: read a file's content (whole, as a byte range, or as a stream), compute its digest, + * get its properties, or list a directory. Nothing is sent until a terminal of the returned + * {@link RemoteFile} is called. * *
{@code
 	 * byte[] content = client.file("C:\\Windows\\Temp\\collect.bin").readBytes();
 	 * String tail = client.file("D:\\logs\\huge.log").offset(-8192).readText(StandardCharsets.UTF_8);
+	 * Optional info = client.file("C:\\Windows\\Temp\\collect.log").info();
+	 * RemoteFileList logs = client.file("C:\\inetpub\\logs").list().glob("*.log").recursive().execute();
 	 * }
* - * @param path the absolute path of the file on the remote host, e.g. + * @param path the absolute path of the file or directory on the remote host, e.g. * {@code C:\Windows\Temp\collect.log} * @return the remote file, to configure and read * @throws IllegalArgumentException when the path is blank diff --git a/src/site/markdown/file-transfers.md b/src/site/markdown/file-transfers.md index 10bba1d..8bc5f73 100644 --- a/src/site/markdown/file-transfers.md +++ b/src/site/markdown/file-transfers.md @@ -177,6 +177,6 @@ Notes: ## See also -* [Remote Files](files.html) — the other direction: reading remote files through the WinRM channel +* [Remote Files](files.html) — the other direction: reading and listing remote files through the WinRM channel * [Remote Commands](commands.html) — the command builder that carries the transfer * [Preparing the Windows Host](preparing-the-host.html) — the privileges a transfer needs diff --git a/src/site/markdown/files.md b/src/site/markdown/files.md index 13dcc5e..ec3e65f 100644 --- a/src/site/markdown/files.md +++ b/src/site/markdown/files.md @@ -1,14 +1,14 @@ -keywords: remote file, read file, byte range, tail, digest, base64, powershell, stream -description: How the WinRM Java Client reads files on the remote host through the WinRM channel itself — whole files, byte ranges, tails, streams, and digests. +keywords: remote file, read file, byte range, tail, digest, base64, powershell, stream, directory listing, file properties, exists, glob +description: How the WinRM Java Client reads files and lists directories on the remote host through the WinRM channel itself — whole files, byte ranges, tails, streams, digests, file properties, and filtered recursive listings. # Remote Files WinRM has no file-access operation of its own (nothing like SFTP's `READ`), so the client reads -remote files **through the WinRM command shell**: a small PowerShell script opens the file on the -host and writes the requested bytes base64-encoded, and the client decodes them as they arrive. -No SMB, no extra port, no share. This is the reverse direction of +remote files and lists directories **through the WinRM command shell**: a small PowerShell script +does the work on the host and writes the result in an encoding-proof form, and the client decodes +it as it arrives. No SMB, no extra port, no share. This is the reverse direction of [File Transfers](file-transfers.html). ## Reading a file @@ -77,31 +77,33 @@ a failure midway is reported by `read()`, never as a silently short read. ## Timeouts -The blocking terminals (`readBytes`, `readText`, `digest`) run under a **wall-clock deadline**: -the client's timeout, or `timeout(Duration)` on the request — raise it for large reads. -`openStream()`/`openReader()` use the **inactivity** semantics of the other streaming terminals: -the timeout bounds the silence between two blocks, not the whole read. See -[Timeouts and Errors](timeouts-and-errors.html). +The blocking terminals (`readBytes`, `readText`, `digest`, `info`, `exists`, and `execute()` on a +[directory listing](#listing-a-directory)) run under a **wall-clock deadline**: the client's +timeout, or `timeout(Duration)` on the request — raise it for large reads and big trees. +`openStream()`/`openReader()` and a listing's `stream()` use the **inactivity** semantics of the +other streaming terminals: the timeout bounds the silence between two blocks, not the whole +operation. See [Timeouts and Errors](timeouts-and-errors.html). -## Locked files, errors and requirements +## Errors and requirements * The file is opened with a share mode that tolerates other writers, so **a log being written by a running service can be read**. A file held with an exclusive lock (`pagefile.sys`, live registry hives) fails with a *sharing violation*. -* Every failure is a `WinRMClientException` naming its cause: file not found, the path is a - directory, access denied, sharing violation, PowerShell not available, or PowerShell in - Constrained Language Mode. +* Every failure is a `WinRMClientException` naming its cause: path not found, a directory where a + file is expected (or a file given to `list()`), access denied, sharing violation, PowerShell not + available, or PowerShell in Constrained Language Mode. * The host needs **PowerShell 2.0 or later in `FullLanguage` mode** (Windows Server 2008 R2 and later ship it). When AppLocker or WDAC puts PowerShell in Constrained Language Mode, the .NET - calls the reader relies on are blocked: remote file access is then not available. + calls the scripts rely on are blocked: remote file access is then not available. * Non-ASCII paths and content are safe: the path travels base64-encoded (UTF-8) inside the script and the content comes back base64-encoded, so neither depends on the remote console code page. -* A read never writes anything on the host: the reader script travels on the command line, which - limits the path to about **1,450 characters** (about 720 with accented letters, 480 with CJK - characters — well above the classic 260-character `MAX_PATH`). A longer path fails with a +* Remote file access never writes anything on the host: its scripts travel on the command line, + which limits the path to about **1,450 characters** for a read (about 720 with accented letters, + 480 with CJK characters) and about **450 characters** for `info()` and `list()`, whose script + is larger — both well above the classic 260-character `MAX_PATH`. A longer path fails with a `WinRMClientException` before anything is sent. -## Performance +## Read performance The mechanism is designed for **configuration files, logs and small data files — not bulk data**. Measured over HTTP with NTLM encryption, `openStream()` reads about **1.4–1.5 MB/s**: a 20 MiB @@ -121,7 +123,138 @@ line by line, like `Write-Output` or `type` — gets only what accumulated in th read, about 0.3 MB/s. Parallel commands, each on its own `WinRMClient`, add up: each command gets its own output pipeline. +## File properties + +`info()` returns the properties of a file or directory — or an empty `Optional` when the path +does not exist, which is not an error. `exists()` is `info().isPresent()`. + +```java +Optional info = client.file("C:\\Windows\\Temp\\collect.log").info(); +info.ifPresent(i -> System.out.println(i.size() + " bytes, modified " + i.lastModified())); + +boolean there = client.file("C:\\Windows\\Temp\\collect.log").exists(); +``` + +[`RemoteFileInfo`](apidocs/org/metricshub/winrm/RemoteFileInfo.html) is an immutable value: + +| Accessor | Content | +|---|---| +| `path()`, `name()` | the full path as reported by the host, and its last element | +| `isDirectory()` | a directory (including a junction or a directory symbolic link) | +| `size()` | the size in bytes, 0 for a directory | +| `lastModified()`, `created()`, `lastAccessed()` | UTC timestamps, 100 ns precision | +| `isHidden()`, `isSystem()`, `isReadOnly()`, `isArchive()`, `isReparsePoint()` | the attribute flags | +| `attributes()` | the raw Windows `FileAttributes` value, for anything else | + +Genuine failures still throw a `WinRMClientException`: access denied on the path itself, an +invalid path, PowerShell unavailable or constrained. + +## Listing a directory + +`list()` prepares the listing of a directory; set its filters, then `execute()` it or +`stream()` it: + +```java +RemoteFileList logs = client.file("C:\\inetpub\\logs").list() + .glob("*.log") + .recursive() // off by default + .maxDepth(3) + .filesOnly() // or directoriesOnly() + .modifiedAfter(Instant.now().minus(Duration.ofDays(1))) + .minSize(1024L) + .execute(); +logs.entries(); // List +logs.inaccessible(); // List: directories that could not be read + +// Large trees stream: memory stays bounded whatever the size of the tree +try (Stream tree = client.file("D:\\data").list() + .recursive() + .onInaccessible(path -> System.err.println("skipped " + path)) + .stream()) { + tree.filter(RemoteFileInfo::isDirectory).forEach(System.out::println); +} +``` + +| Setting | Effect | +|---|---| +| *(none)* | the directory's own entries | +| `recursive()` | the whole tree, depth-first | +| `maxDepth(n)` | the tree down to depth `n` (1 = the directory's own entries); implies `recursive()` | +| `glob(pattern)` | entries whose **name** matches: `*` any sequence, `?` one character, everything else literal, case-insensitive, whole name (`*.log` matches `app.log`, not `app.log.1`) | +| `filesOnly()`, `directoriesOnly()` | one kind of entry | +| `modifiedAfter(instant)`, `modifiedBefore(instant)` | last write time strictly after / before | +| `minSize(bytes)`, `maxSize(bytes)` | file size bounds, inclusive; directories have no size and are not filtered by them | +| `onInaccessible(consumer)` | told of each directory that could not be read, as the host reports it | +| `timeout(duration)` | wall-clock deadline for `execute()`, inactivity timeout for `stream()` | + +**Every filter is evaluated on the host**: a big tree is not shipped over the wire just to be +discarded locally. The filters select what is *reported*, not where the walk goes: with +`recursive()`, every subdirectory is traversed, whatever the glob or the type filter. + +Like the other streaming terminals, **a `stream()` must be closed** (try-with-resources): it +holds the client's connection until it is exhausted or closed, and closing it early stops the +remote walk. Failures are thrown from the stream's operations, when the host reports them. The +host signals it is alive every second while it walks, so the inactivity timeout only trips when +the host is unresponsive — or when reading a single directory takes longer than the timeout. + +### Links and inaccessible directories + +* **Reparse points** — junctions, symbolic links, mount points — are reported (with + `isReparsePoint()` true) but **never descended into**: a junction looping back to its parent, + like the legacy `C:\Documents and Settings`-style junctions, cannot make a walk run forever. + Listing a junction explicitly lists its target. +* A subdirectory that cannot be read (typically access denied) **does not stop the walk**: its + path is reported through `onInaccessible(...)` and `RemoteFileList.inaccessible()`, and its + content is missing from the entries. The directory being listed is different: when it cannot be + read, the listing fails. +* An administrator's WinRM session holds the *backup* privilege, enabled, and directory + enumeration uses backup semantics: permissions that deny reading a directory do not stop an + administrator. Inaccessible directories are mostly met with non-administrator accounts. + +### Listing performance + +The walk runs in the host's PowerShell and streams one short text line per reported entry. +Measured over HTTP with NTLM encryption: `C:\Windows\System32` (16,000 entries) in 2.3 seconds, +the whole of `C:\Windows` (126,000 entries) in 15 seconds on Windows Server 2022; the same +System32 in 5.5 seconds on Windows Server 2008 R2 (PowerShell 2.0). With selective filters, only +the walk itself counts: a recursive `*.dll` over 10 MB in System32 takes under a second on 2022. + +## How the metadata travels + +The listing does not parse `dir` output (locale-dependent columns, dates and decimal separators, +minute granularity), nor use `Get-ChildItem` (whose `-Depth`, `-File` and `-Directory` need +PowerShell 3 to 5, and whose object pipeline is slow on big trees). A small script walks the tree +with the .NET `DirectoryInfo` API and an explicit stack — on every PowerShell version from 2.0 — +and writes **one ASCII line per entry**: the attributes, size and timestamps as plain integers +(Windows file times), and only the path base64-encoded (UTF-8). Non-ASCII names therefore +round-trip exactly, whatever the remote console code page, and a truncated or malformed line +fails with a clear exception instead of producing a half-populated entry. + +WMI could serve metadata too (`CIM_DataFile`, `CIM_Directory` and `ASSOCIATORS OF` queries through +[`client.wql(...)`](wql.html)), and remains an alternative where PowerShell is constrained — but +the WMI file provider is notoriously slow (an unindexed `CIM_DataFile` query on a large tree can +take minutes), a recursive listing needs one query per directory, and timestamps come back as +DMTF strings. + +## Paths and limitations + +* **Long paths**: paths longer than the classic 260-character `MAX_PATH` work where PowerShell + runs on .NET Framework 4.6.2 or later (Windows Server 2016 and later out of the box; older + versions with WMF 5.1 and an updated .NET): the scripts use `\\?\`-prefixed paths there, and + report paths without the prefix. On older hosts (e.g. + Windows Server 2008 R2 with PowerShell 2.0), a path over 260 characters fails with an explicit + "path too long" error. The path *given* to `info()` or `list()` is limited to about 450 + characters (see [the requirements](#errors-and-requirements)); the entries a + listing reports can be of any length. +* **UNC paths** (`\\server\share\...`) are a *second hop*: the host must authenticate to the file + server with your credentials, which NTLM does not allow. They need Kerberos credential + delegation ([#141](https://github.com/MetricsHub/winrm-java/issues/141)), which this client does + not provide yet: until then, access to a UNC path typically fails with access denied. +* `lastAccessed()` is only as good as the host keeps it: many Windows versions disable or delay + last-access updates. + ## See also * [File Transfers](file-transfers.html) — the other direction: copying local files to the host -* [Remote Commands](commands.html) — the command shell the reads ride +* [Remote Commands](commands.html) — the command shell the reads and listings ride +* [WQL Queries](wql.html) — the WMI alternative for file metadata diff --git a/src/site/markdown/index.md b/src/site/markdown/index.md index ee31e31..060e4ab 100644 --- a/src/site/markdown/index.md +++ b/src/site/markdown/index.md @@ -1,5 +1,5 @@ keywords: winrm java client, windows remote management, wsman, dependency-free, overview -description: A dependency-free Java client for Windows Remote Management (WinRM): run WQL queries and remote commands over NTLM or Kerberos. +description: A dependency-free Java client for Windows Remote Management (WinRM): run WQL queries and remote commands, and read and list remote files, over NTLM or Kerberos. # WinRM Java Client @@ -11,15 +11,17 @@ The **WinRM Java Client** is a small library that talks to the Windows Remote Ma (WS-Management) service on a remote Windows host. It lets a Java application: * run **WQL / WMI queries** such as `SELECT Name, State FROM Win32_Service` and read the rows back - ([WQL Queries](wql.html)), and + ([WQL Queries](wql.html)), * **execute remote commands** — `cmd.exe` command lines or PowerShell scripts — capturing standard output, standard error and the exit code, optionally copying local script files to the host - first ([Remote Commands](commands.html)). + first ([Remote Commands](commands.html)), and +* **access remote files** — read a file (whole, a byte range, a tail), get its properties, or + list a directory with filters evaluated on the host ([Remote Files](files.html)). -Both operations can also **stream**: WQL rows are consumed page by page as they arrive -(`stream()`), and command output is consumed while the command is still running (`start()`, -returning a `java.lang.Process`-like handle) — memory stays bounded regardless of the result -size. +All of them can also **stream**: WQL rows are consumed page by page as they arrive +(`stream()`), command output is consumed while the command is still running (`start()`, +returning a `java.lang.Process`-like handle), and file contents and directory listings are +decoded as the host sends them — memory stays bounded regardless of the result size. It supports **NTLM** over HTTP (with message encryption) and HTTPS, **Kerberos (SPNEGO)** over HTTPS, and **HTTP Basic** over HTTPS ([Authentication](authentication.html)). @@ -111,6 +113,13 @@ quoting or escaping at all: CommandResult ps = client.powerShell("Get-Service | Where-Object Status -eq 'Running'").execute(); ``` +Remote files go through the same connection — no SMB, no extra port: + +```java +String tail = client.file("D:\\logs\\app.log").offset(-8192).readText(StandardCharsets.UTF_8); +RemoteFileList logs = client.file("D:\\logs").list().glob("*.log").recursive().execute(); +``` + Failures are reported through the unchecked [`WinRMClientException`](apidocs/org/metricshub/winrm/exceptions/WinRMClientException.html) hierarchy. The static one-shot helpers that predate `WinRMClient` @@ -126,7 +135,7 @@ remain available and unchanged, with their checked exceptions. * [WQL Queries](wql.html) — query WMI and read the result * [Remote Commands](commands.html) — run commands and copy files to the host * [File Transfers](file-transfers.html) — how files are copied through the WinRM channel -* [Remote Files](files.html) — read remote files: whole, byte ranges, tails, streams, digests +* [Remote Files](files.html) — read remote files (whole, byte ranges, tails, streams, digests), get file properties, list directories * [Command-Line Client](cli.html) — the standalone jar's manual page * [Authentication](authentication.html) — NTLM and Kerberos * [TLS / HTTPS](tls.html) — certificate validation and trust stores diff --git a/src/site/markdown/migrating-from-winrm4j.md b/src/site/markdown/migrating-from-winrm4j.md index 0d82657..23dba16 100644 --- a/src/site/markdown/migrating-from-winrm4j.md +++ b/src/site/markdown/migrating-from-winrm4j.md @@ -24,7 +24,8 @@ code base can switch in one sitting. winrm4j/CXF failure mode. * **Actively maintained**, with releases published on Maven Central. * **Features winrm4j never had:** [WQL / WMI queries](wql.html), [file transfers](file-transfers.html) - through the WinRM channel, [standard input](commands.html#standard-input), + and [remote file reads and directory listings](files.html) through the WinRM channel, + [standard input](commands.html#standard-input), [`Process`-style streaming](commands.html#streaming-the-output) of live output, and a full-featured [command-line client](cli.html). @@ -179,6 +180,9 @@ Once on the fluent API, features winrm4j never offered are one call away: typed rows and streaming ([WQL Queries](wql.html)). * **File transfers** — `upload(Path...)` copies local scripts to the host through the WinRM channel itself (no SMB, no port 445) before the command runs ([File Transfers](file-transfers.html)). +* **Remote files** — `client.file(path)` reads a file (whole, a byte range, a tail, or streamed), + returns its properties, or lists a directory with filters evaluated on the host + ([Remote Files](files.html)). * **Standard input** — `stdin(...)` feeds a remote command its input, with real EOF semantics ([Standard input](commands.html#standard-input)). * **Live streaming** — `start()` returns a `java.lang.Process`-shaped diff --git a/src/site/markdown/preparing-the-host.md b/src/site/markdown/preparing-the-host.md index 8ffbaf6..b2035a6 100644 --- a/src/site/markdown/preparing-the-host.md +++ b/src/site/markdown/preparing-the-host.md @@ -44,8 +44,9 @@ here: by the `winrs` command-line tool. A Java client never reads it, so you do not need to add anything to it on either machine. * **No DCOM (port 135) and no SMB (port 445).** WQL rides WinRM rather than DCOM, and file - transfers ride the WinRM channel itself ([File Transfers](file-transfers.html)). The WinRM port - is the only one you need to open. + transfers and remote file access ride the WinRM channel itself + ([File Transfers](file-transfers.html), [Remote Files](files.html)). The WinRM port is the only + one you need to open. ## Is WinRM already enabled? @@ -271,6 +272,7 @@ separately: | **Remote commands** (`client.command(...)`) | Remote access to the listener, remote shell access on the host (`AllowRemoteShellAccess`, `True` by default), and whatever rights **the command itself** needs once it runs. | | **Transfer-and-run** (`upload(...)`) | Both of the above, plus write access to `\Temp\winrm-upload-`, and `certutil` and `forfiles` present on the host. See [File Transfers](file-transfers.html). | | **`uploadFile(...)`** to an explicit path | Remote shell access, write access to the destination directory, and `certutil` on the host (the same transfer engine, minus the transfer directory and its `forfiles` housekeeping). | +| **Remote file access** (`client.file(...)`: reads, `info()`, `list()`) | Remote shell access, **PowerShell 2.0 or later in `FullLanguage` mode** (not constrained by AppLocker or WDAC), and read access to the files and directories themselves. Nothing is written on the host. See [Remote Files](files.html). | So an account can perfectly well run WQL queries and fail to run commands, or the reverse. When diagnosing, test the two independently — as in [Checking from the client](#Checking_from_the_client) @@ -281,7 +283,8 @@ above. Non-administrative access is possible, and is the right choice for a monitoring account that only needs to read WMI. Step 1 below is always required. **Step 2 is only needed for WQL queries** (and therefore for transfer-and-run, which discovers the remote Windows directory with one): an account -that only runs commands or calls `uploadFile(...)` never reaches WMI, so grant it nothing there. +that only runs commands, calls `uploadFile(...)` or accesses remote files never reaches WMI, so +grant it nothing there. **1. Grant remote access to the WinRM listener.** The default listener security descriptor (`RootSDDL`) grants full access to `BUILTIN\Administrators` and read access to interactive users @@ -355,7 +358,8 @@ an administrative account when you run commands. Whichever you choose, verify it A remote command authenticates with a **network logon** whose credentials **cannot be delegated onward**. So a command that reaches a *second* remote resource — a UNC path, another server, a mapped drive — fails with access denied, even though the same command works when run locally on -the host. +the host. The same goes for a UNC path given to [`client.file(...)`](files.html), which runs as a +remote command too. Windows solves this with CredSSP or Kerberos constrained delegation. This client **does not support CredSSP** ([Authentication](authentication.html)), so the workaround is to avoid the second hop: @@ -401,5 +405,6 @@ The full exception surface, including how to read a WSMan fault code, is describ * [Authentication](authentication.html) — NTLM and Kerberos, and what each needs from the host * [TLS / HTTPS](tls.html) — trusting the listener's certificate * [File Transfers](file-transfers.html) — what transfers need on the host +* [Remote Files](files.html) — reading and listing remote files, and what that needs on the host * [Timeouts and Errors](timeouts-and-errors.html) — the exception surface and WSMan fault detail * [Command-Line Client](cli.html) — the quickest way to test a host's configuration diff --git a/src/site/markdown/timeouts-and-errors.md b/src/site/markdown/timeouts-and-errors.md index cdb810b..40a2792 100644 --- a/src/site/markdown/timeouts-and-errors.md +++ b/src/site/markdown/timeouts-and-errors.md @@ -35,8 +35,10 @@ socket timeouts follow each operation's own deadline. ### Streaming terminals: inactivity timeout -The streaming terminals — `stream()` on a WQL request and `start()` on a command (see -[WQL Queries](wql.html) and [Remote Commands](commands.html)) — interpret the same `timeout(...)` +The streaming terminals — `stream()` on a WQL request, `start()` on a command, +`openStream()`/`openReader()` on a remote file and `stream()` on a directory listing (see +[WQL Queries](wql.html), [Remote Commands](commands.html) and [Remote Files](files.html)) — +interpret the same `timeout(...)` value differently, because an overall deadline would make long-running streams impossible: there it is an **inactivity timeout**, the longest silence tolerated from the server between two responses. A query result can be consumed, or a command can keep streaming output, for arbitrarily diff --git a/src/test/java/org/metricshub/winrm/RemoteFileListingTest.java b/src/test/java/org/metricshub/winrm/RemoteFileListingTest.java new file mode 100644 index 0000000..04ae524 --- /dev/null +++ b/src/test/java/org/metricshub/winrm/RemoteFileListingTest.java @@ -0,0 +1,391 @@ +package org.metricshub.winrm; + +/*- + * ╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲ + * WinRM Java Client + * ჻჻჻჻჻჻ + * Copyright (C) 2023 - 2026 MetricsHub + * ჻჻჻჻჻჻ + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * ╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱╲╱ + */ +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.metricshub.winrm.light.FakeWsmanResponses.commandResponse; +import static org.metricshub.winrm.light.FakeWsmanResponses.done; +import static org.metricshub.winrm.light.FakeWsmanResponses.envelope; +import static org.metricshub.winrm.light.FakeWsmanResponses.receiveResponse; +import static org.metricshub.winrm.light.FakeWsmanResponses.resourceCreated; +import static org.metricshub.winrm.light.FakeWsmanResponses.signalResponse; +import static org.metricshub.winrm.light.FakeWsmanResponses.stream; + +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.time.Instant; +import java.util.ArrayList; +import java.util.Base64; +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; +import java.util.stream.Stream; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.metricshub.winrm.exceptions.WinRMClientException; +import org.metricshub.winrm.light.FakeWsmanServer; + +/** + * Tests of the metadata side of the remote file access: the record parser, and + * {@link RemoteFile#info()} and {@link RemoteDirectoryListing} against {@link FakeWsmanServer} + * with scripted records. The scripts themselves run in {@link RemoteFilesScriptTest}. + */ +class RemoteFileListingTest { + + private static final String DOMAIN = "FAKE"; + private static final String USER = "user"; + private static final String PASSWORD = "s3cret-Passw0rd"; + private static final String COMMAND_ID = "CMD-1"; + private static final String DIR = "C:\\inetpub\\logs"; + + /** 2026-01-02T03:04:05.6789012Z as a FileTime. */ + private static final long FILETIME = 134_117_966_456_789_012L; + private static final Instant INSTANT = Instant.parse("2026-01-02T03:04:05.6789012Z"); + + private FakeWsmanServer server; + + @BeforeEach + void startServer() throws Exception { + server = new FakeWsmanServer(DOMAIN, USER, PASSWORD); + } + + @AfterEach + void stopServer() { + server.close(); + } + + private WinRMClient client() { + return WinRMClient + .builder("127.0.0.1") + .port(server.port()) + .credentials(DOMAIN + "\\" + USER, PASSWORD.toCharArray()) + .timeout(Duration.ofSeconds(10)) + .build(); + } + + private static String b64(final String text) { + return Base64.getEncoder().encodeToString(text.getBytes(StandardCharsets.UTF_8)); + } + + private static String file(final String path, final long size) { + return "F 32 " + size + " " + FILETIME + " " + FILETIME + " " + FILETIME + " " + b64(path) + "\n"; + } + + private static String dir(final String path) { + return "F 16 0 " + FILETIME + " " + FILETIME + " " + FILETIME + " " + b64(path) + "\n"; + } + + private static String denied(final String path) { + return "! " + b64(path) + " " + b64("Access to the path '" + path + "' is denied.") + "\n"; + } + + /** Script a whole command: shell creation, command, the given stdout chunks (the last one completes), Signal. */ + private void enqueue(final int exitCode, final String... chunks) { + server.enqueue(200, envelope(resourceCreated("SHELL-1"))).enqueue(200, envelope(commandResponse(COMMAND_ID))); + for (int i = 0; i < chunks.length; i++) { + final boolean last = i == chunks.length - 1; + server.enqueue( + 200, + envelope( + receiveResponse( + stream("stdout", COMMAND_ID, chunks[i].getBytes(StandardCharsets.US_ASCII)), + last ? done(COMMAND_ID, exitCode) : null + ) + ) + ); + } + server.enqueue(200, envelope(signalResponse())); + } + + /** The PowerShell script the client sent, decoded from its -EncodedCommand. */ + private String sentScript() { + final String command = server + .decryptedRequests() + .stream() + .filter(r -> r.contains("-EncodedCommand")) + .findFirst() + .orElseThrow(() -> new AssertionError("no PowerShell command sent")); + final Matcher matcher = Pattern.compile("-EncodedCommand ([A-Za-z0-9+/=]+)").matcher(command); + assertTrue(matcher.find(), command); + return new String(Base64.getDecoder().decode(matcher.group(1)), StandardCharsets.UTF_16LE); + } + + @Test + void parseEntryReadsEveryField() { + final RemoteFileInfo info = RemoteFiles.parseEntry( + "F 1063 5368709120 " + FILETIME + " 116444736000000000 116444736000000001 " + b64("D:\\données\\漢字.bin") + ); + assertEquals("D:\\données\\漢字.bin", info.path()); + assertEquals("漢字.bin", info.name()); + // 0x427: read-only, hidden, system, archive, reparse point. + assertEquals(0x427, info.attributes()); + assertTrue(info.isReadOnly() && info.isHidden() && info.isSystem() && info.isArchive() && info.isReparsePoint()); + assertFalse(info.isDirectory()); + // Larger than 4 GiB. + assertEquals(5_368_709_120L, info.size()); + assertEquals(INSTANT, info.lastModified()); + assertEquals(Instant.EPOCH, info.created()); + assertEquals(Instant.ofEpochSecond(0, 100), info.lastAccessed()); + assertEquals( + info, + RemoteFiles.parseEntry( + "F 1063 5368709120 " + FILETIME + " 116444736000000000 116444736000000001 " + b64("D:\\données\\漢字.bin") + ) + ); + assertTrue(info.toString().endsWith("D:\\données\\漢字.bin"), info.toString()); + + final RemoteFileInfo junction = RemoteFiles + .parseEntry(dir("C:\\Documents and Settings").strip().replace("F 16 ", "F 1046 ")); + assertTrue(junction.isDirectory() && junction.isReparsePoint() && junction.isHidden() && junction.isSystem()); + assertEquals("Documents and Settings", junction.name()); + assertEquals("C:", RemoteFiles.parseEntry(dir("C:\\").strip()).name()); + assertEquals("share", RemoteFiles.parseEntry(dir("\\\\server\\share").strip()).name()); + } + + @Test + void fileTimesConvertBothWays() { + assertEquals(FILETIME, RemoteFiles.toFileTime(INSTANT)); + assertEquals(INSTANT, RemoteFiles.fromFileTime(FILETIME)); + assertEquals(Instant.parse("1601-01-01T00:00:00Z"), RemoteFiles.fromFileTime(0)); + } + + @Test + void parseInaccessibleReturnsThePath() { + assertEquals("C:\\Windows\\CSC", RemoteFiles.parseInaccessible(denied("C:\\Windows\\CSC").strip())); + } + + @Test + void malformedOrTruncatedRecordsFailClearly() { + final String good = file("C:\\a.log", 1).strip(); + for (final String bad : List.of( + good.substring(0, good.lastIndexOf(' ')), // truncated: no path + good + " extra", // the path field is not base64 + good.replace("F 32 1", "F 32 x"), // non-numeric size + good.replace("F 32", "X 32"), // unknown record type + good.substring(0, good.length() - 3) + "!!!", // corrupted base64 + "! " + b64("C:\\x"), // ! record without message + "garbage" + )) { + final WinRMClientException e = assertThrows( + WinRMClientException.class, + () -> { + if (bad.startsWith("!")) { + RemoteFiles.parseInaccessible(bad); + } else { + RemoteFiles.parseEntry(bad); + } + }, + bad + ); + assertTrue(e.getMessage().startsWith("Malformed remote file record: "), e.getMessage()); + } + } + + @Test + void executeCollectsEntriesAndInaccessibleDirectories() { + // Records cut across Receive chunks, keepalive blank lines in between. + final String records = file(DIR + "\\a.log", 10) + "\n" + dir(DIR + "\\W3SVC1") + denied(DIR + "\\W3SVC1\\private") + + + file(DIR + "\\W3SVC1\\u_ex260101.log", 5_000_000_000L) + "\n"; + enqueue(0, records.substring(0, 30), records.substring(30, 100), records.substring(100)); + + final List reported = new ArrayList<>(); + try (WinRMClient client = client()) { + final RemoteFileList list = client.file(DIR).list().recursive().onInaccessible(reported::add).execute(); + assertEquals( + List.of(DIR + "\\a.log", DIR + "\\W3SVC1", DIR + "\\W3SVC1\\u_ex260101.log"), + list.entries().stream().map(RemoteFileInfo::path).collect(Collectors.toList()) + ); + assertEquals(5_000_000_000L, list.entries().get(2).size()); + assertEquals(List.of(DIR + "\\W3SVC1\\private"), list.inaccessible()); + assertEquals(list.inaccessible(), reported); + assertEquals("3 entries, 1 inaccessible", list.toString()); + } + final String script = sentScript(); + assertTrue(script.contains(b64(DIR)), script); + // Recursive, unlimited, no filter. + assertTrue( + script.contains( + "FromBase64String(''));$k=0;$mn=0;$mx=" + Long.MAX_VALUE + ";$ta=-1;$tb=" + Long.MAX_VALUE + ";$md=" + + Integer.MAX_VALUE + ";" + ), + script + ); + } + + @Test + void anEmptyDirectoryIsAnEmptyList() { + enqueue(0, "\n"); + try (WinRMClient client = client()) { + final RemoteFileList list = client.file(DIR).list().execute(); + assertTrue(list.entries().isEmpty()); + assertTrue(list.inaccessible().isEmpty()); + } + // Not recursive: depth 1. + assertTrue(sentScript().contains(";$md=1;"), sentScript()); + } + + @Test + void filtersReachTheScript() { + enqueue(0, "\n"); + final Instant after = Instant.parse("2026-01-01T00:00:00Z"); + final Instant before = Instant.parse("2026-02-01T00:00:00Z"); + try (WinRMClient client = client()) { + client + .file(DIR) + .list() + .glob("u_ex*.log") + .maxDepth(3) + .directoriesOnly() + .filesOnly() + .minSize(1024) + .maxSize(1 << 20) + .modifiedAfter(after) + .modifiedBefore(before) + .execute(); + } + assertTrue( + sentScript() + .contains( + "FromBase64String('" + b64("^u_ex.*\\.log$") + "'));$k=1;$mn=1024;$mx=1048576;$ta=" + + RemoteFiles.toFileTime(after) + ";$tb=" + RemoteFiles.toFileTime(before) + ";$md=3;" + ), + sentScript() + ); + } + + @Test + void streamYieldsEntriesAsTheyArriveAndReportsInaccessibleDirectories() { + enqueue(0, dir(DIR + "\\a") + denied(DIR + "\\a\\b"), file(DIR + "\\a\\c.log", 1)); + final List reported = new ArrayList<>(); + try ( + WinRMClient client = client(); + Stream entries = client.file(DIR).list().recursive().onInaccessible(reported::add).stream()) { + assertEquals( + List.of("a", "c.log"), + entries.map(RemoteFileInfo::name).collect(Collectors.toList()) + ); + assertEquals(List.of(DIR + "\\a\\b"), reported); + } + } + + @Test + void aFailureIsThrownFromTheStreamAndByExecute() { + enqueue(RemoteFiles.EXIT_NOT_FOUND, ""); + try (WinRMClient client = client(); Stream entries = client.file(DIR).list().stream()) { + final WinRMClientException e = assertThrows(WinRMClientException.class, entries::count); + assertTrue(e.getMessage().contains("not found") && e.getMessage().contains(DIR), e.getMessage()); + } + enqueue(RemoteFiles.EXIT_NOT_DIRECTORY, ""); + try (WinRMClient client = client()) { + final RemoteDirectoryListing listing = client.file(DIR).list(); + final WinRMClientException e = assertThrows(WinRMClientException.class, listing::execute); + assertTrue(e.getMessage().contains("not a directory"), e.getMessage()); + } + } + + @Test + void infoReturnsThePropertiesOrEmptyForAMissingPath() throws Exception { + enqueue(0, file(DIR + "\\a.log", 10) + "\n"); + try (WinRMClient client = client()) { + final RemoteFileInfo info = client.file(DIR + "\\a.log").info().orElseThrow(); + assertEquals(10, info.size()); + assertEquals(INSTANT, info.lastModified()); + } + assertTrue(sentScript().contains(b64(DIR + "\\a.log")), sentScript()); + + server.close(); + server = new FakeWsmanServer(DOMAIN, USER, PASSWORD); + enqueue(RemoteFiles.EXIT_NOT_FOUND, ""); + try (WinRMClient client = client()) { + assertTrue(client.file(DIR + "\\missing.log").info().isEmpty()); + } + + server.close(); + server = new FakeWsmanServer(DOMAIN, USER, PASSWORD); + enqueue(RemoteFiles.EXIT_NOT_FOUND, ""); + try (WinRMClient client = client()) { + assertFalse(client.file(DIR + "\\missing.log").exists()); + } + } + + @Test + void progressRecordsAreLeftOutOfTheErrorMessage() { + server.enqueue(200, envelope(resourceCreated("SHELL-1"))).enqueue(200, envelope(commandResponse(COMMAND_ID))); + final String stderr = "#< CLIXML\r\nThe path is too long.\r\n\r\n"; + server.enqueue( + 200, + envelope( + receiveResponse(stream("stderr", COMMAND_ID, stderr.getBytes(StandardCharsets.UTF_8)), done(COMMAND_ID, 1)) + ) + ); + server.enqueue(200, envelope(signalResponse())); + try (WinRMClient client = client()) { + final RemoteFile file = client.file(DIR); + final WinRMClientException e = assertThrows(WinRMClientException.class, file::info); + assertTrue(e.getMessage().endsWith("(exit code 1): The path is too long."), e.getMessage()); + } + } + + @Test + void infoFailsOnAccessDenied() { + enqueue(RemoteFiles.EXIT_ACCESS_DENIED, ""); + try (WinRMClient client = client()) { + final RemoteFile file = client.file("C:\\System Volume Information\\x"); + final WinRMClientException e = assertThrows(WinRMClientException.class, file::info); + assertTrue(e.getMessage().contains("Access denied"), e.getMessage()); + } + } + + @Test + void globsBecomeAnchoredRegexesWithOnlyTwoWildcards() { + assertEquals("^u_ex.*\\.log$", RemoteFiles.globRegex("u_ex*.log")); + assertEquals("^a.\\ \\[1\\]\\(é\\)\\$\\^\\+漢$", RemoteFiles.globRegex("a? [1](é)$^+漢")); + } + + @Test + void theScriptsFitTheCommandLineWithLongPaths() { + final String path = "C:\\" + "a".repeat(400); + assertNotNull(CommandRequest.encodePowerShell(RemoteFiles.infoScript(path))); + assertNotNull( + CommandRequest + .encodePowerShell(RemoteFiles.listScript(path, "*.log", 1, 0, Long.MAX_VALUE, -1, Long.MAX_VALUE, 3)) + ); + } + + @Test + void invalidSettingsAreRejectedLocally() { + try (WinRMClient client = client()) { + final RemoteDirectoryListing listing = client.file(DIR).list(); + assertThrows(IllegalArgumentException.class, () -> listing.glob(" ")); + assertThrows(IllegalArgumentException.class, () -> listing.maxDepth(0)); + assertThrows(IllegalArgumentException.class, () -> listing.modifiedAfter(null)); + assertThrows(IllegalArgumentException.class, () -> listing.onInaccessible(null)); + assertThrows(IllegalArgumentException.class, () -> listing.timeout(Duration.ZERO)); + } + assertEquals(0, server.decryptedRequests().size()); + } +} diff --git a/src/test/java/org/metricshub/winrm/RemoteFilesScriptTest.java b/src/test/java/org/metricshub/winrm/RemoteFilesScriptTest.java index 7a60d28..244fa16 100644 --- a/src/test/java/org/metricshub/winrm/RemoteFilesScriptTest.java +++ b/src/test/java/org/metricshub/winrm/RemoteFilesScriptTest.java @@ -22,6 +22,8 @@ import static org.junit.jupiter.api.Assertions.assertArrayEquals; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; import java.io.ByteArrayOutputStream; import java.io.FileOutputStream; @@ -30,11 +32,17 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; +import java.nio.file.attribute.FileTime; import java.security.MessageDigest; +import java.time.Instant; import java.util.Arrays; import java.util.Base64; +import java.util.List; +import java.util.Map; import java.util.Random; +import java.util.Set; import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.condition.EnabledOnOs; @@ -84,6 +92,51 @@ private static final class Outcome { } private static Outcome run(final String script) throws Exception { + final Listing raw = exec(script); + final ByteArrayOutputStream bytes = new ByteArrayOutputStream(); + for (final String line : raw.lines) { + bytes.write(Base64.getDecoder().decode(line)); + } + return new Outcome(bytes.toByteArray(), raw.exitCode); + } + + /** The outcome of a metadata script: its non-blank stdout lines and the exit code. */ + private static final class Listing { + + final List lines; + final int exitCode; + + Listing(final List lines, final int exitCode) { + this.lines = lines; + this.exitCode = exitCode; + } + + /** The entries, parsed like the client does, by path relative to the test directory. */ + Map entries() { + return lines + .stream() + .filter(l -> l.startsWith("F ")) + .map(RemoteFiles::parseEntry) + .collect(Collectors.toMap(i -> relative(i.path()), i -> i)); + } + + List inaccessible() { + return lines + .stream() + .filter(l -> l.startsWith("! ")) + .map(RemoteFiles::parseInaccessible) + .map(RemoteFilesScriptTest::relative) + .collect(Collectors.toList()); + } + } + + private static String relative(final String path) { + final String root = directory.toString() + "\\"; + assertTrue(path.startsWith(root), path); + return path.substring(root.length()); + } + + private static Listing exec(final String script) throws Exception { final String encoded = Base64.getEncoder().encodeToString(script.getBytes(StandardCharsets.UTF_16LE)); final Process process = new ProcessBuilder( "powershell.exe", @@ -100,13 +153,12 @@ private static Outcome run(final String script) throws Exception { process.destroyForcibly(); throw new AssertionError("powershell.exe did not complete"); } - final ByteArrayOutputStream bytes = new ByteArrayOutputStream(); - for (final String line : stdout.split("\r?\n")) { - if (!line.isBlank()) { - bytes.write(Base64.getDecoder().decode(line.strip())); - } - } - return new Outcome(bytes.toByteArray(), process.exitValue()); + final List lines = Arrays + .stream(stdout.split("\r?\n")) + .filter(l -> !l.isBlank()) + .map(String::strip) + .collect(Collectors.toList()); + return new Listing(lines, process.exitValue()); } private static byte[] read(final long offset, final long length) throws Exception { @@ -193,4 +245,191 @@ void digestMatchesTheLocalDigest() throws Exception { assertEquals(0, outcome.exitCode); assertArrayEquals(MessageDigest.getInstance("SHA-256").digest(content), outcome.bytes); } + + private static final Instant OLD = Instant.parse("2020-01-01T00:00:00.1234567Z"); + + /** + * {@code tree\a.log} (10 bytes, 2020), {@code tree\b.txt} (2000 bytes), + * {@code tree\données-漢字 [x].log}, {@code tree\sub\c.log}, {@code tree\sub\deep\d.log}, + * {@code tree\sub\deep\deeper\e.log}. + */ + private static Path tree() throws Exception { + final Path tree = directory.resolve("tree"); + if (!Files.exists(tree)) { + Files.createDirectories(tree.resolve("sub\\deep\\deeper")); + Files.setLastModifiedTime(Files.write(tree.resolve("a.log"), new byte[10]), FileTime.from(OLD)); + Files.write(tree.resolve("b.txt"), new byte[2000]); + Files.write(tree.resolve("données-漢字 [x].log"), new byte[1]); + Files.write(tree.resolve("sub\\c.log"), new byte[5]); + Files.write(tree.resolve("sub\\deep\\d.log"), new byte[5]); + Files.write(tree.resolve("sub\\deep\\deeper\\e.log"), new byte[5]); + } + return tree; + } + + private static Listing list(final Path path, final String glob, final int type, final int maxDepth) throws Exception { + return exec(RemoteFiles.listScript(path.toString(), glob, type, 0, Long.MAX_VALUE, -1, Long.MAX_VALUE, maxDepth)); + } + + private static void cmd(final String... command) throws Exception { + final Process process = new ProcessBuilder(command).redirectErrorStream(true).start(); + final String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + assertEquals(0, process.waitFor(), output); + } + + @Test + void listingReportsEveryFieldAndOnlyTheDirectEntriesByDefault() throws Exception { + final Listing listing = list(tree(), null, 0, 1); + assertEquals(0, listing.exitCode); + final Map entries = listing.entries(); + assertEquals(Set.of("tree\\a.log", "tree\\b.txt", "tree\\données-漢字 [x].log", "tree\\sub"), entries.keySet()); + + final RemoteFileInfo a = entries.get("tree\\a.log"); + assertEquals(10, a.size()); + assertEquals("a.log", a.name()); + assertFalse(a.isDirectory()); + // FileTime precision: 100 ns, exactly what was set. + assertEquals(OLD, a.lastModified()); + assertTrue(a.created().isAfter(OLD)); + + final RemoteFileInfo sub = entries.get("tree\\sub"); + assertTrue(sub.isDirectory()); + assertEquals(0, sub.size()); + assertEquals("données-漢字 [x].log", entries.get("tree\\données-漢字 [x].log").name()); + assertTrue(listing.inaccessible().isEmpty()); + } + + @Test + void recursionHonorsTheMaximumDepth() throws Exception { + final Set depth2 = list(tree(), null, 0, 2).entries().keySet(); + assertTrue(depth2.contains("tree\\sub\\c.log") && depth2.contains("tree\\sub\\deep"), depth2.toString()); + assertFalse(depth2.contains("tree\\sub\\deep\\d.log"), depth2.toString()); + assertEquals(9, list(tree(), null, 0, Integer.MAX_VALUE).entries().size()); + } + + @Test + void filtersAreEvaluatedOnTheHost() throws Exception { + // Case-insensitive, whole-name glob; [ and ] are literal; directories are traversed anyway. + assertEquals( + Set.of( + "tree\\a.log", + "tree\\données-漢字 [x].log", + "tree\\sub\\c.log", + "tree\\sub\\deep\\d.log", + "tree\\sub\\deep\\deeper\\e.log" + ), + list(tree(), "*.LOG", 1, Integer.MAX_VALUE).entries().keySet() + ); + assertEquals(Set.of("tree\\données-漢字 [x].log"), list(tree(), "*[x].log", 0, 1).entries().keySet()); + assertEquals(Set.of("tree\\a.log"), list(tree(), "?.log", 0, 1).entries().keySet()); + assertTrue(list(tree(), "*.lo", 0, 1).entries().isEmpty()); + assertEquals( + Set.of("tree\\sub", "tree\\sub\\deep", "tree\\sub\\deep\\deeper"), + list(tree(), null, 2, Integer.MAX_VALUE).entries().keySet() + ); + + final String path = tree().toString(); + // Size bounds: files only, directories pass. + assertEquals( + Set.of("tree\\b.txt", "tree\\sub"), + exec(RemoteFiles.listScript(path, null, 0, 1000, Long.MAX_VALUE, -1, Long.MAX_VALUE, 1)).entries().keySet() + ); + assertEquals( + Set.of("tree\\données-漢字 [x].log", "tree\\sub"), + exec(RemoteFiles.listScript(path, null, 0, 0, 5, -1, Long.MAX_VALUE, 1)).entries().keySet() + ); + // Time bounds are exclusive, at the 100 ns FileTime precision. + final long old = RemoteFiles.toFileTime(OLD); + assertEquals( + Set.of("tree\\a.log"), + exec(RemoteFiles.listScript(path, null, 1, 0, Long.MAX_VALUE, old - 1, old + 1, 1)).entries().keySet() + ); + assertTrue( + exec(RemoteFiles.listScript(path, "a.log", 1, 0, Long.MAX_VALUE, old, Long.MAX_VALUE, 1)).entries().isEmpty() + ); + assertTrue(exec(RemoteFiles.listScript(path, "a.log", 1, 0, Long.MAX_VALUE, -1, old, 1)).entries().isEmpty()); + } + + @Test + void aJunctionLoopIsReportedButNotFollowed() throws Exception { + final Path loop = Files.createDirectories(directory.resolve("loop")); + Files.write(loop.resolve("f.txt"), new byte[1]); + final Path junction = loop.resolve("back"); + cmd("cmd.exe", "/c", "mklink", "/J", junction.toString(), loop.toString()); + try { + final Map entries = list(loop, null, 0, Integer.MAX_VALUE).entries(); + assertEquals(Set.of("loop\\f.txt", "loop\\back"), entries.keySet()); + assertTrue(entries.get("loop\\back").isReparsePoint()); + assertTrue(entries.get("loop\\back").isDirectory()); + // Listing the junction itself, explicitly, lists its target's entries. + assertEquals(2, list(junction, null, 0, Integer.MAX_VALUE).entries().size()); + } finally { + // rmdir removes the junction, not its target (JUnit would otherwise walk the loop). + cmd("cmd.exe", "/c", "rmdir", junction.toString()); + } + } + + @Test + void pathsLongerThanMaxPathAreNotTruncated() throws Exception { + final Path deep = directory.resolve("long").resolve("a".repeat(100)).resolve("b".repeat(100)); + final Path longFile = Files.write(Files.createDirectories(deep).resolve("c".repeat(80) + ".txt"), new byte[42]); + assertTrue(longFile.toString().length() > 300); + + final Listing listing = list(directory.resolve("long"), "c*", 1, Integer.MAX_VALUE); + assertEquals(0, listing.exitCode, listing.lines::toString); + final RemoteFileInfo entry = listing.entries().get(relative(longFile.toString())); + assertEquals(42, entry.size()); + assertEquals(longFile.toString(), entry.path()); + + final Listing info = exec(RemoteFiles.infoScript(longFile.toString())); + assertEquals(longFile.toString(), RemoteFiles.parseEntry(info.lines.get(0)).path()); + } + + @Test + void anInaccessibleSubdirectoryIsReportedAndTheWalkGoesOn() throws Exception { + final Path root = Files.createDirectories(directory.resolve("partial")); + final Path secret = Files.createDirectories(root.resolve("secret")); + Files.write(secret.resolve("hidden.txt"), new byte[1]); + Files.write(Files.createDirectories(root.resolve("zpublic")).resolve("open.txt"), new byte[1]); + // Deny "list folder" to Everyone. + cmd("icacls", secret.toString(), "/deny", "*S-1-1-0:(RD)"); + try { + final Listing listing = list(root, null, 0, Integer.MAX_VALUE); + assertEquals(0, listing.exitCode); + assertEquals( + Set.of("partial\\secret", "partial\\zpublic", "partial\\zpublic\\open.txt"), + listing.entries().keySet() + ); + assertEquals(List.of("partial\\secret"), listing.inaccessible()); + + // Listing the inaccessible directory itself fails. + assertEquals(RemoteFiles.EXIT_ACCESS_DENIED, list(secret, null, 0, 1).exitCode); + } finally { + cmd("icacls", secret.toString(), "/remove:d", "*S-1-1-0"); + } + } + + @Test + void infoReportsAFileOrADirectoryAndMissingPathsExitWithNotFound() throws Exception { + final Listing fileInfo = exec(RemoteFiles.infoScript(file.toString())); + assertEquals(0, fileInfo.exitCode); + assertEquals(1, fileInfo.lines.size()); + final RemoteFileInfo info = RemoteFiles.parseEntry(fileInfo.lines.get(0)); + assertEquals(file.toString(), info.path()); + assertEquals(content.length, info.size()); + + final RemoteFileInfo dir = RemoteFiles.parseEntry(exec(RemoteFiles.infoScript(directory.toString())).lines.get(0)); + assertTrue(dir.isDirectory()); + + final Listing missing = exec(RemoteFiles.infoScript(directory.resolve("missing.txt").toString())); + assertEquals(RemoteFiles.EXIT_NOT_FOUND, missing.exitCode); + assertTrue(missing.lines.isEmpty()); + assertEquals( + RemoteFiles.EXIT_NOT_FOUND, + exec(RemoteFiles.infoScript(directory.resolve("no\\such\\dir").toString())).exitCode + ); + + assertEquals(RemoteFiles.EXIT_NOT_DIRECTORY, list(file, null, 0, 1).exitCode); + assertEquals(RemoteFiles.EXIT_NOT_FOUND, list(directory.resolve("missing"), null, 0, 1).exitCode); + } } diff --git a/src/test/java/org/metricshub/winrm/WinRMLiveTest.java b/src/test/java/org/metricshub/winrm/WinRMLiveTest.java index 71273b0..d38a618 100644 --- a/src/test/java/org/metricshub/winrm/WinRMLiveTest.java +++ b/src/test/java/org/metricshub/winrm/WinRMLiveTest.java @@ -11,6 +11,8 @@ import java.security.MessageDigest; import java.time.Duration; import java.util.List; +import java.util.stream.Collectors; +import java.util.stream.Stream; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.condition.EnabledIfSystemProperty; @@ -192,4 +194,113 @@ void remoteFileRangeIsServedBySeekingAndAFileOpenForWritingCanBeRead() throws Ex client.command("del /q \"" + big + "\" \"" + log + "\"").execute(); } } + + @Test + void remoteDirectoryListing() throws Exception { + final String base = "C:\\Windows\\Temp\\winrm-java-live-list"; + final String nonAscii = base + "\\donn\u00e9es-\u6f22\u5b57.txt"; + // 3 x 100 characters below the base: over 330 characters in all. + final String longDir = base + "\\long\\" + "a".repeat(100) + "\\" + "b".repeat(100); + final String longFile = longDir + "\\" + "c".repeat(100) + ".txt"; + + try (WinRMClient client = client()) { + // .NET 4.6.2+ accepts \\?\ paths (long paths); PowerShell 2.0 on .NET 2.0 does not. + final boolean longPaths = client + .powerShell("try{[void][IO.Path]::GetFullPath('\\\\?\\C:\\x');'yes'}catch{'no'}") + .execute() + .stdout() + .contains("yes"); + client + .powerShell( + "$b='" + base + "';" + + "foreach($d in 'sub\\deep','secret'){[void][IO.Directory]::CreateDirectory(\"$b\\$d\")};" + + "foreach($f in 'a.log','sub\\b.log','sub\\deep\\c.log','secret\\hidden.txt'){" + + "[IO.File]::WriteAllText(\"$b\\$f\",'x')};" + + "[IO.File]::WriteAllText('" + nonAscii + "','x');" + + "cmd /c mklink /J \"$b\\loop\" \"$b\" | Out-Null;" + + "icacls \"$b\\secret\" /deny '*S-1-1-0:(RD)' | Out-Null;" + + (longPaths + ? "[void][IO.Directory]::CreateDirectory('\\\\?\\" + longDir + "');" + + "[IO.File]::WriteAllText('\\\\?\\" + longFile + "','0123456789')" + : "") + ) + .execute(); + try { + // A plain listing of C:\Windows\Temp. + assertTrue( + client + .file("C:\\Windows\\Temp") + .list() + .directoriesOnly() + .execute() + .entries() + .stream() + .anyMatch(e -> e.path().equalsIgnoreCase(base)) + ); + + // Depth-limited. + final List depth2 = paths(client.file(base).list().maxDepth(2).execute().entries()); + assertTrue(depth2.contains(base + "\\sub\\b.log"), depth2::toString); + assertTrue(depth2.contains(base + "\\sub\\deep"), depth2::toString); + assertFalse(depth2.contains(base + "\\sub\\deep\\c.log"), depth2::toString); + + // The whole tree: the junction loop terminates, the denied directory is reported. + final long start = System.nanoTime(); + final RemoteFileList all = client.file(base).list().recursive().execute(); + System.out.printf("Listed %s in %d ms%n", all, (System.nanoTime() - start) / 1_000_000L); + final List paths = paths(all.entries()); + assertTrue(paths.contains(nonAscii), paths::toString); + assertTrue(paths.contains(base + "\\secret"), paths::toString); + // An administrator's WinRM session has SeBackupPrivilege enabled, and directory + // enumeration uses backup semantics: the deny ACE then does not apply. + if (paths.contains(base + "\\secret\\hidden.txt")) { + assertTrue(all.inaccessible().isEmpty(), all.inaccessible()::toString); + System.out.println("The deny ACE was bypassed (backup privilege): inaccessible directories not tested"); + } else { + assertEquals(List.of(base + "\\secret"), all.inaccessible()); + } + final RemoteFileInfo loop = all + .entries() + .stream() + .filter(e -> e.path().equals(base + "\\loop")) + .findFirst() + .orElseThrow(); + assertTrue(loop.isReparsePoint() && loop.isDirectory()); + assertTrue(paths.stream().noneMatch(p -> p.startsWith(base + "\\loop\\")), paths::toString); + + // Streamed, with filters. + try (Stream logs = client.file(base).list().recursive().glob("*.LOG").filesOnly().stream()) { + assertEquals( + List.of("a.log", "b.log", "c.log"), + logs.map(RemoteFileInfo::name).sorted().collect(Collectors.toList()) + ); + } + + // Properties. + assertEquals(1, client.file(base + "\\a.log").info().orElseThrow().size()); + assertTrue(client.file(nonAscii).exists()); + assertFalse(client.file(base + "\\missing.log").exists()); + + if (longPaths) { + assertEquals(10, client.file(longFile).info().orElseThrow().size()); + assertTrue( + paths(client.file(base + "\\long").list().recursive().filesOnly().execute().entries()).contains(longFile) + ); + } else { + System.out.println("Long paths not supported by this host's .NET: skipped"); + } + } finally { + client + .powerShell( + "$b='" + base + "';icacls \"$b\\secret\" /remove:d '*S-1-1-0' | Out-Null;" + + "cmd /c rmdir \"$b\\loop\";cmd /c rmdir /s /q \"\\\\?\\$b\"" + ) + .execute(); + } + } + } + + private static List paths(final List entries) { + return entries.stream().map(RemoteFileInfo::path).collect(Collectors.toList()); + } } From cf07edeb9888a8d67c90e0516ec5084097e29b83 Mon Sep 17 00:00:00 2001 From: Bertrand Martin Date: Thu, 24 Sep 2026 20:34:31 +0200 Subject: [PATCH 2/2] Time the listing keepalive with a Stopwatch, not Environment.TickCount (Codex review) TickCount is a signed 32-bit millisecond counter, negative for about half of every 49.7-day cycle. With the clock starting at 0, "TickCount - $t" was negative on such hosts, so a selective walk that never filled the 32 KB buffer sent no keepalive and could trip stream()'s inactivity timeout. A wrap during a walk had the same effect, because PowerShell promotes the overflowing subtraction to a double instead of wrapping. The walker now times the silence since its last write with a System.Diagnostics.Stopwatch: monotonic, 64-bit, and available on .NET 2.0 (PowerShell 2.0). Verified locally: an 89-second walk of C:\Windows that matched nothing sent 86 keepalives. The live listing test passes on Windows Server 2008 R2. files.md: correct the command-line path limits (measured): about 1,150 characters for info(), 450 for list(); the docs had both at 450. Co-Authored-By: Claude Opus 5.5 --- src/main/java/org/metricshub/winrm/RemoteFiles.java | 10 ++++++---- src/site/markdown/files.md | 10 +++++----- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/src/main/java/org/metricshub/winrm/RemoteFiles.java b/src/main/java/org/metricshub/winrm/RemoteFiles.java index 5344e71..1f2c929 100644 --- a/src/main/java/org/metricshub/winrm/RemoteFiles.java +++ b/src/main/java/org/metricshub/winrm/RemoteFiles.java @@ -128,7 +128,8 @@ private RemoteFiles() {} *
  • {@code out} writes the buffered records and a newline to the raw stdout stream as one * write (see {@link #READ_RANGE} for why); with nothing buffered, the bare newline is a * keepalive the parser skips, so a long walk that matches nothing does not trip the inactivity - * timeout; + * timeout; {@code $v} times the silence since the last write — a {@code Stopwatch}, monotonic + * and 64-bit, where {@code Environment.TickCount} is negative for half of its 49.7-day cycle; *
  • the path is made absolute, then given the {@code \\?\} prefix ({@code \\?\UNC\} for a * UNC path) that lifts the 260-character {@code MAX_PATH} limit — where .NET accepts it (4.6.2 * and later: older versions reject {@code GetFullPath('\\?\...')}, and paths stay limited @@ -137,9 +138,10 @@ private RemoteFiles() {} * */ private static final String METADATA = PREAMBLE + - "$o=[Console]::OpenStandardOutput();$u=[Text.Encoding]::UTF8;$w=New-Object Text.StringBuilder;$t=0;" + + "$o=[Console]::OpenStandardOutput();$u=[Text.Encoding]::UTF8;$w=New-Object Text.StringBuilder;" + + "$v=[Diagnostics.Stopwatch]::StartNew();" + "function out{$y=$u.GetBytes(\"$w`n\");$o.Write($y,0,$y.Length);$o.Flush();$w.Length=0;" + - "$script:t=[Environment]::TickCount};" + + "$v.Reset();$v.Start()};" + "try{$q=[IO.Path]::GetFullPath($p);" + "try{$q=[IO.Path]::GetFullPath((($q-replace'^\\\\\\\\(?=[^\\\\?.])','\\\\?\\UNC\\')" + "-replace'^(?=[A-Za-z]:\\\\)','\\\\?\\'))}catch{};" + @@ -183,7 +185,7 @@ private RemoteFiles() {} "$f=$e.LastWriteTimeUtc.ToFileTimeUtc();" + "if(($k -eq 0 -or ($k -eq 2) -eq $i) -and ($i -or ($e.Length -ge $mn -and $e.Length -le $mx)) -and " + "$f -gt $ta -and $f -lt $tb -and $e.Name -match $g){" + RECORD + "};" + - "if($w.Length -gt 32000 -or [Environment]::TickCount-$t -gt 1000){out}}}" + + "if($w.Length -gt 32000 -or $v.ElapsedMilliseconds -gt 1000){out}}}" + "catch{if($n -eq 1){fail $_.Exception};$x=$_.Exception;if($x.InnerException){$x=$x.InnerException};" + "[void]$w.Append(\"! $([Convert]::ToBase64String($u.GetBytes($pa+$d.FullName.Substring($pn)))) " + "$([Convert]::ToBase64String($u.GetBytes($x.Message)))`n\")}};out"; diff --git a/src/site/markdown/files.md b/src/site/markdown/files.md index ec3e65f..43e9ca0 100644 --- a/src/site/markdown/files.md +++ b/src/site/markdown/files.md @@ -98,10 +98,10 @@ operation. See [Timeouts and Errors](timeouts-and-errors.html). * Non-ASCII paths and content are safe: the path travels base64-encoded (UTF-8) inside the script and the content comes back base64-encoded, so neither depends on the remote console code page. * Remote file access never writes anything on the host: its scripts travel on the command line, - which limits the path to about **1,450 characters** for a read (about 720 with accented letters, - 480 with CJK characters) and about **450 characters** for `info()` and `list()`, whose script - is larger — both well above the classic 260-character `MAX_PATH`. A longer path fails with a - `WinRMClientException` before anything is sent. + which limits the path to about **1,450 characters** for a read, **1,150** for `info()`, and + **450** for `list()`, whose walker script is the largest (roughly half as many with accented + letters, a third with CJK characters) — all above the classic 260-character `MAX_PATH`. A + longer path fails with a `WinRMClientException` before anything is sent. ## Read performance @@ -243,7 +243,7 @@ DMTF strings. versions with WMF 5.1 and an updated .NET): the scripts use `\\?\`-prefixed paths there, and report paths without the prefix. On older hosts (e.g. Windows Server 2008 R2 with PowerShell 2.0), a path over 260 characters fails with an explicit - "path too long" error. The path *given* to `info()` or `list()` is limited to about 450 + "path too long" error. The directory *given* to `list()` is limited to about 450 characters (see [the requirements](#errors-and-requirements)); the entries a listing reports can be of any length. * **UNC paths** (`\\server\share\...`) are a *second hop*: the host must authenticate to the file