Validate crypto keygen responses against the received frame length

[yosuke-wolfssl]

Validate crypto keygen responses against the received frame length

Summary

The client key generation paths deserialize a server-declared payload length
out of the response buffer. Two of the four did not validate that the declared
length fit within the frame that actually arrived, so a short response could
cause the deserializer to consume stale bytes left in the comm buffer.

• Ed25519 checked against WOLFHSM_CFG_COMM_DATA_LEN, the buffer
  capacity, not the received length. It also read res->keyId before any
  check, and the check it had never ran on the cache path.
• Curve25519 had no bounds check at all, truncating the untrusted
  res->len and passing it straight to the deserializer.

ECC and RSA already had the correct check.

This does not need a hostile server. When a crypto handler returns non-zero the
server replies with the generic header only, and the client admits positive
wolfCrypt codes as success via if (ret >= 0) — so a header-only frame reaches
the unchecked reads through an ordinary error path. No out-of-bounds access is
possible since the comm buffer is always full size; the impact is stale bytes
deserialized as key material and a bogus key id returned to the caller.

Addressed by f_5461.

Changes

src/wh_client_crypto.c — add the frame-length check to _Ed25519MakeKey and
_Curve25519MakeKey, ahead of every read of the response struct:

const size_t hdr_sz =
    sizeof(whMessageCrypto_GenericResponseHeader) + sizeof(*res);

if (res_len < hdr_sz || res->outSz > (res_len - hdr_sz)) {
    return WH_ERROR_ABORTED;
}

The first clause short-circuits, so the subtraction cannot underflow. Valid
responses are unaffected: the server sets *outSize = sizeof(response) + ser_size and the dispatcher adds the header, so res_len == hdr_sz + declared_len holds exactly, on both the export and cache paths.

test-refactor/posix/wh_test_malformed_response.c (new) — negative-path
coverage for the rejection branches, which no existing test could reach since
the portable harness only emits well-formed frames. A fake server at the comm
layer sends bodies of arbitrary length, driving two shapes per algorithm: one
stopping after the generic header, one declaring a payload past the frame. Both
must yield WH_ERROR_ABORTED. Covers all four keygen paths.

Port-owned rather than in the common list because it supplies its own client and
fake server over a private transport pair, following the thread-safety stress
test precedent.

Testing

Each check was confirmed to fail before its fix — reverting Ed25519 fails its
rows with -2000/-140, reverting Curve25519 fails its rows with -140, with
other algorithms still passing, so every row is tied to its own path.

Clean rebuilds, no warnings under -std=c90 -Werror -Wall -Wextra: default
(38 passed), NOCRYPTO=1 (11), DMA=1 (42), THREADSAFE=1 (39) — all with
0 failed, plus the legacy suite clean.

[Copilot]

Pull request overview

This PR hardens the client-side crypto key-generation response handling by validating server-declared payload lengths against the actual received frame length, preventing stale bytes in the comm buffer from being deserialized as key material (notably on short/generic-header-only responses).

Changes:

• Add received-frame-length bounds checks to Curve25519 and Ed25519 keygen response parsing in src/wh_client_crypto.c.
• Introduce a POSIX-only “malformed response” regression test using a fake comm-layer server that sends truncated/oversized-declared frames.
• Register the new POSIX-port-owned test in the POSIX test runner so it runs before the real server thread starts.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
src/wh_client_crypto.c	Adds defensive checks ensuring res->len/outSz fits within the received response frame before any response-body reads/deserialization.
test-refactor/posix/wh_test_malformed_response.c	New negative-path test that injects malformed response lengths via a fake comm server and asserts WH_ERROR_ABORTED across keygen algorithms.
test-refactor/posix/wh_test_posix_main.c	Registers and runs the new malformed-response test in the pre-server “misc” slot to avoid overlapping the real server.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #465

Scan targets checked: wolfhsm-crypto-bugs, wolfhsm-src

No new issues found in the changed files. ✅