internal: send a disconnect when key exchange fails

[yosuke-wolfssl]

Problem

Every key exchange failure path aborted the handshake without transmitting
SSH_MSG_DISCONNECT. The peer saw only a dropped transport with no reason
code. DoKexInit() already answers a KEXINIT negotiation failure with
WOLFSSH_DISCONNECT_KEY_EXCHANGE_FAILED, so the convention existed but was
not applied once key agreement began.

Fix (src/internal.c)

Two guards at the KEX message boundaries, mirroring the DoKexInit() pattern:

if (ret == WS_CRYPTO_FAILED || ret == WS_PUBKEY_REJECTED_E) {
    (void)SendDisconnect(ssh, WOLFSSH_DISCONNECT_KEY_EXCHANGE_FAILED);
}

• DoKexDhReply() covers the client: all of KeyAgree_client().
• DoKexDhInit() covers the server: the eSz range check and all of
  SendKexDhReply().

Both handlers also serve the GEX message IDs, so two insertion points cover
every KEX algorithm. Both codes are terminal, so the guard cannot misfire on
a WS_WANT_WRITE from SendNewKeys(). The (void) cast is required: taking
the return would mask the KEX error with SendDisconnect()'s own status.

Closes f-8838.

Tests (tests/regress.c)

New mutator modes on the duplex harness — truncated f in KEXDH_REPLY,
truncated and zero-length e in KEXDH_INIT — plus disconnect recording on
both endpoints. Five cases now assert the reason code reaches the wire:

Case	Role	Code
Truncated f	client	WS_CRYPTO_FAILED
Truncated e	server	WS_CRYPTO_FAILED
Zero-length e	server	WS_PUBKEY_REJECTED_E
No pubkey-check callback	client	WS_PUBKEY_REJECTED_E
Callback rejects host key	client	WS_PUBKEY_REJECTED_E

A successful handshake asserts no disconnect is seen, so the tests cannot pass
on a detector that always fires. DH and ML-KEM are covered by construction, not
by test: a truncated DH mpint is still a valid group element.

Verification

• 4/4 suites pass (regress, unit, kex, api).
• Negative controls: removing either guard, narrowing it to WS_CRYPTO_FAILED,
  or forcing the detector to always fire each breaks the expected test.
• Clean under gcc-13 -Werror across all 6 preflight configs.
• ASan + UBSan clean, 0 reports.
• Config sweep: curve25519 off (falls back to ECDH), both off (compiles out),
  --enable-sshd only.

Not in this PR

Host key signature verification failures (WS_RSA_E, WS_ECC_E,
WS_ED25519_E, WS_MLDSA_E) still abort without a disconnect. That is server
authentication rather than key exchange, and OpenSSH is the precedent: its
client calls fatal("Host key verification failed.") and sends nothing. For
reference, OpenSSH defines SSH2_DISCONNECT_KEY_EXCHANGE_FAILED and
HOST_KEY_NOT_VERIFIABLE but sends neither anywhere — all its protocol errors
go out as PROTOCOL_ERROR. wolfSSH's more specific reason code follows its own
DoKexInit() precedent.

[Copilot]

Pull request overview

This PR ensures wolfSSH sends an explicit SSH_MSG_DISCONNECT with reason WOLFSSH_DISCONNECT_KEY_EXCHANGE_FAILED when key exchange fails during KEXDH_INIT / KEXDH_REPLY, instead of silently dropping the transport. It also extends the duplex regression harness to mutate KEX packets and assert the disconnect reason is transmitted on the wire.

Changes:

• Add disconnect-on-fatal-KEX-failure guards to DoKexDhInit() (server side) and DoKexDhReply() (client side).
• Extend the regression duplex mutator to truncate f (in KEXDH_REPLY) and truncate/empty e (in KEXDH_INIT), plus record outbound disconnect reason codes.
• Add new regression cases asserting the disconnect reason is observed, and assert successful handshakes do not produce disconnects.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/internal.c	Sends SSH_MSG_DISCONNECT with KEY_EXCHANGE_FAILED when KEX fails mid-exchange in both client/server KEX handlers.
tests/regress.c	Adds new KEX packet mutation modes and disconnect sniffing/assertions for KEX failure paths.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #1171

Scan targets checked: wolfssh-bugs, wolfssh-src

Findings: 4
4 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #1171

Scan targets checked: wolfssh-bugs, wolfssh-src

Findings: 4
4 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #1171

Scan targets checked: none
Failed targets: wolfssh-bugs, wolfssh-src

⚠️ Review incomplete — one or more scan targets failed before findings could be produced. See the Fenrir PR review detail page for logs.