winrm-light 1/6: Framework-free WSMan core (templated envelopes, JDK HTTP, NTLM over HTTPS) #103

Activity

1. bertysentry

   added

   enhancementNew feature or requestNew feature or request

   on Jul 22, 2026
2. bertysentry commented on Jul 22, 2026

   

   bertysentry

   on Jul 22, 2026

   ContributorAuthor

   More actions

   Transport-layer spike results (against a real Windows Server 2008 R2 host)

   Validated the pure-JDK transport with a WSMan Identify request — java.net.http.HttpClient + JDK default XML, zero third-party libraries:

   • ✅ JDK HTTP transport works — well-formed request/response round-trip, no dependencies.
   • The host answers Identify with HTTP 401 and WWW-Authenticate: Negotiate, Kerberos — no bare NTLM scheme offered.

   Implication for the auth design

   NTLM must be carried under the Negotiate (SPNEGO) header, not a bare Authorization: NTLM .... This matches what the current stack already does — the tell is NtlmMasqAsSpnegoSchemeFactory registered under AuthSchemes.SPNEGO in WinRMInvocationHandler. Two viable approaches for the light client:

   1. Raw NTLMSSP tokens under the Negotiate header (what pywinrm does): Windows accepts a bare NTLM type-1/type-3 token base64'd under Authorization: Negotiate .... Simplest; no ASN.1.
   2. Wrap NTLM in a minimal SPNEGO NegTokenInit/NegTokenResp (ASN.1 DER). More faithful but more code.

   Recommend starting with (1).

   Scope note discovered during the spike

   The test host exposes only the HTTP listener (5985) with AllowUnencrypted at its default (false). So a #103-only client (no message encryption) cannot complete a real WQL/command exchange against a default-configured host — the server rejects the unencrypted SOAP body. #103 can be validated to the auth-handshake boundary (successful 200 on an authenticated Identify), but full end-to-end WQL against a stock host needs the NTLM message encryption from #104.

   Good news: #104 is a port of already-working code — the existing org.metricshub.winrm.service.client.encryption classes authenticate and exchange encrypted payloads against this exact host today.

   🤖 Generated with Claude Code
3. bertysentry commented on Jul 22, 2026

   

   bertysentry

   on Jul 22, 2026

   ContributorAuthor

   More actions

   Prototype spike results — concept validated, one transport-level blocker isolated

   Built a complete dependency-free prototype (org.metricshub.winrm.light): raw-socket HTTP/1.1 transport with connection affinity, NTLM-masqueraded-as-Negotiate handshake, NTLM message encryption over HTTP (ported from the existing …client.encryption/…auth.ntlm classes, stripped of the CXF/apache coupling), templated WSMan envelopes, and JDK-default XML parsing. Tested against a real Windows Server 2008 R2 host over HTTP/5985 with NTLM (AllowUnencrypted=false).

   What is proven working

   • Transport + NTLM handshake: Type1→Type2→Type3 completes; the server accepts the auth. Confirmed decisively — sending Type3 with an unencrypted body returns HTTP 500 (a WSMan-layer error, i.e. the request passed http.sys and auth), whereas a rejected auth would be a 401 re-challenge.
   • Message-encryption crypto is byte-perfect: a differential harness seals an identical plaintext with identical injected keys through both the ported NtlmCrypto and the reference NtlmEncryptionUtils — byte-identical output. Key derivation (signing/sealing keys from the exported session key) is likewise byte-identical to the reference NtlmKeys formula.
   • Wire framing is byte-identical to the working client: captured the working CXF client's requests via a logging TCP relay and byte-diffed. The multipart/encrypted framing (boundaries, OriginalContent, 4-byte signature-length prefix, 16-byte signature layout, trailer) and all HTTP headers match exactly.
   • JAXP immunity: uses DocumentBuilderFactory.newDefaultInstance(), so the Oracle-xmlparserv2 class of bug (Isolate extensions in dedicated classloaders to prevent cross-extension classpath pollution metricshub-community#1271) cannot occur.

   The blocker (transport-level, not crypto)

   The encrypted operation request is rejected by http.sys with HTTP 400 (empty body, Server: Microsoft-HTTPAPI/2.0), where the CXF client gets 200. A hybrid test isolates it conclusively:

   transport	crypto	result
   Apache HttpClient (CXF)	reference	200
   my raw-socket transport	reference	400
   my raw-socket transport	ported (light)	400

   So the fault is in the transport/connection handling, not the crypto — the reference crypto that works under Apache HttpClient fails under my transport. Yet the requests are byte-identical (verified via relay capture) and the connection choreography matches (unauth probe on one connection → Type1→Type3 on a fresh kept-alive connection). Ruled out: header order/casing, User-Agent, TCP_NODELAY, the pre-auth bogus body, empty vs non-empty Type1 body, single vs split head/body writes.

   This is a subtle TCP/connection nuance http.sys is sensitive to that doesn't show in a byte diff of individual requests. Next step: packet-level (Wireshark) capture on the host to compare the two transports' actual TCP behavior, and/or trying java.net.http/Apache HttpClient purely for the connection layer while keeping the framework-free envelopes+crypto. The crypto and envelope work (the bulk of #103/#104) is done and proven reusable.

   🤖 Generated with Claude Code
4. bertysentry commented on Jul 22, 2026

   

   bertysentry

   on Jul 22, 2026

   ContributorAuthor

   More actions

   Follow-up: tried java.net.http — same 400, which sharpens the diagnosis

   Swapped the raw-socket transport for the JDK's java.net.http.HttpClient (production HTTP client), keeping the reference crypto. Result: also HTTP 400, identical failure. Verified via TCP relay that java.net.http correctly reused a single connection for the whole NTLM handshake (Type1→Type2→Type3 on one connection), and its Type3 request framing is byte-identical to CXF's.

   Also confirmed my reference-crypto driver (RefNtlm) replicates the real flow exactly: NTLMScheme.authenticate uses one Type3Message instance for both getResponse() (the wire token) and initEncryption→NtlmKeys.apply (key derivation) — which is precisely what my driver does. So the crypto is not only byte-correct but orchestrated identically to CXF.

   Sharpened conclusion

   transport	crypto	result
   Apache HttpClient (async, via CXF)	repo crypto	200
   java.net.http	repo crypto	400
   raw sockets	repo crypto	400

   Two independent, correct HTTP clients fail identically on byte-identical requests; only Apache's async client succeeds. So the HTTP-client library isn't the variable — there is a TCP/socket-level behavior of Apache's async HttpClient that http.sys on this host depends on (candidates: write chunking/segmentation of the NIO client, socket options, or connection-lifecycle timing) that neither a raw socket nor java.net.http reproduce. This is only visible with a packet capture, not a byte diff of the HTTP requests.

   Two ways forward

   1. Pragmatic / high-value now: keep Apache HttpClient as the transport (it is already a winrm-java dependency, pulled in by CXF) but drop the CXF/JAX-WS/JAXB/SAAJ/Woodstox/wsdl4j stack, using the hand-written WSMan envelopes + the existing encryption classes. This still removes ~9 MB of the ~10.6 MB shaded artifact and keeps the proven transport. Recommended for winrm-light 6/6: Switch default backend, deprecate and remove the CXF stack #108.
   2. Zero-dependency: crack the transport nuance with a Wireshark capture comparing Apache-async vs raw-socket TCP behavior, then replicate it. The envelopes + crypto are done and proven; only the socket write behavior remains.

   🤖 Generated with Claude Code
5. bertysentry commented on Jul 22, 2026

   

   bertysentry

   on Jul 22, 2026

   ContributorAuthor

   More actions

   Major correction: it is NOT the transport

   Extending the isolation matrix overturns the earlier 'transport' conclusion:

   path	result
   CXF (real NTLMScheme + AsyncHTTPConduit)	200
   direct httpasyncclient + manual NTLM	400
   sync Apache httpclient + manual NTLM	400
   java.net.http + manual NTLM	400
   raw sockets, Windows + manual NTLM	400
   raw sockets, Linux/WSL + manual NTLM	400

   The invariant across every failure is manual NTLM orchestration (my RefNtlm/RefSeal driving the repo's own Type3Message/NtlmKeys/NtlmEncryptionUtils). The only success is the full flow driven by Apache HttpClient's auth framework + CXF's conduit. Even direct use of the exact async client CXF uses fails. So the transport was never the variable.

   Verified identical between manual and real flows: credentials (WinRMEndpoint splits anaxagore\patrol → domain anaxagore, user patrol — same as manual), Type1 flags/token (byte-identical on the wire), Type3 construction (RefNtlm mirrors NTLMScheme.authenticate: one Type3Message feeds both getResponse() and NtlmKeys.apply), and the seal call (NtlmEncryptionUtils.of(creds).encryptAndSign). The wire framing is byte-identical too.

   So something the httpclient auth framework does while driving NTLMScheme — not the socket, not the crypto classes themselves — makes http.sys accept the encrypted request. Candidates: repeated authenticate/processChallenge calls mutating shared credential state, an extra handshake leg, or the entity being re-sealed across auth attempts in a way that changes the sequence number or keys.

   Decisive next step

   Instrument the real CXF flow to log its client signing/sealing keys + the exact sealed bytes for one request, and compare against RefSeal output for the same session. That will show whether the real flow's sealed request differs structurally from the manual one (and how). Until then, the 'drop CXF, keep a light transport' plan is blocked — the working request generation is entangled with CXF/httpclient's auth-framework integration, not just the socket.

   🤖 Generated with Claude Code
6. bertysentry commented on Jul 22, 2026

   

   bertysentry

   on Jul 22, 2026

   ContributorAuthor

   More actions

   RESOLVED — the dependency-free client works end-to-end 🎉

   Root cause found (with Codex's help pointing at a session-key/state pairing error the byte diffs couldn't see): my light client did not uppercase the NTLM domain before computing NTOWFv2.

   Apache's NTCredentials uppercases the domain in its constructor. NTOWFv2 = HMAC-MD5(MD4(password), UPPER(user) || domain) depends on the domain, and the Type 3 message's DomainName field goes on the wire uppercased (via unqualifiedDomain.toUpperCase() in Type3Message.buildMessage). So the server derives its NTLM session key using ANAXAGORE, while my client had computed the user session key (and thus the exported session key it sealed with) using lowercase anaxagore. Net effect: auth succeeds (Event 169), then message integrity fails (Event 140, error code 5) → HTTP 400 — exactly the observed signature.

   How it was isolated

   • Reference crypto over a raw socket → 200 (Codex's RawEngineProbe); my crypto over the same transport → 400 (LightRawProbe). So it was my crypto driver, not the transport, not the HTTP client, not the OS.
   • Proved byte-identical: CipherGen (seeded), key derivation, seal output, and Type 3 token structure/flags.
   • MixProbe (reference token+esk + MY derive/seal) → 200 pinpointed the divergence to token/esk generation in my Type3Message orchestration.
   • The WinRM Operational log (Event 169 + Event 140/error 5) confirmed auth-OK + integrity-fail.
   • Uppercasing the domain → HTTP 200.

   Result (pure-JDK client, zero third-party deps, against Windows Server 2008 R2 over HTTP/5985 with NTLM message encryption)

   Identify: Microsoft Corporation / OS: 6.1.7601 SP: 1.0 Stack: 2.0
   WQL SELECT Caption,Version,CSName FROM Win32_OperatingSystem ->
     Caption = Microsoft Windows Server 2008 R2 Enterprise
     CSName  = ANAXAGORE
     Version = 6.1.7601
   

   So #103 (framework-free core) and #104 (NTLM message encryption over HTTP) are both demonstrated working. Next: productionize into the repo package, feature parity (#106), and the recorded-exchange test rig (#107).

   🤖 Generated with Claude Code
7. bertysentry commented on Jul 23, 2026

   

   bertysentry

   on Jul 23, 2026

   ContributorAuthor

   More actions

   Delivered in #109 (branch `feature/winrm-light`). The framework-free core is in `org.metricshub.winrm.light`: `WsmanClient` (templated SOAP envelopes for Enumerate/Pull and the Create/Command/Receive/Signal/Delete shell lifecycle), `HttpTransport` (raw keep-alive socket — NTLM is connection-bound), and JDK-default XML factories (immune by construction to the Oracle xmlparserv2 / JAXP-poisoning bug, metricshub-community#1271). NTLM auth under Negotiate. Live-verified WQL + command execution against a real Windows host.

   The "over HTTPS" part of the title is being split out and pursued next, together with Kerberos (#105).

   🤖 Generated with Claude Code
8. bertysentry

   closed this as completedon Jul 23, 2026
9. bertysentry

   added a commit that references this issue on Jul 23, 2026

   Remove the CXF backend; the dependency-free client is the only one (2…

   ...

   70c2546