Fixes for AF-ALG code paths and add CI test by JacobBarthelmeh · Pull Request #11162 · wolfSSL/wolfssl

JacobBarthelmeh · 2026-08-12T22:24:09Z

No description provided.

Copilot

Pull request overview

This pull request tightens correctness of the Linux AF_ALG AES-GCM implementation by ensuring the AF_ALG operation (encrypt vs decrypt) is set on every call when reusing an Aes context, and adds CI coverage so these kernel-offload paths are exercised continuously.

Changes:

Add an AES-GCM regression test that reuses a single Aes context across encrypt → decrypt → encrypt without re-setting the key.
Fix AF_ALG AES-GCM to always set the AF_ALG operation control message on every encrypt/decrypt call.
Add a dedicated GitHub Actions workflow to build/test AF_ALG configurations and preflight kernel algorithm availability.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
wolfcrypt/test/test.c	Adds an AES-GCM “reuse context across directions” regression test and runs it from `aesgcm_default_test()`.
wolfcrypt/src/port/af_alg/afalg_aes.c	Ensures AF_ALG operation selection is refreshed per AES-GCM call when reusing the same socket/context; updates related comments.
doc/dox_comments/header_files/aes.h	Documents the Xilinx AF_ALG combined ciphertext+tag buffer behavior and alignment expectations for GCM encrypt/decrypt.
.github/workflows/afalg.yml	Introduces CI that preflights AF_ALG kernel support and runs parallel out-of-tree builds/tests with `--enable-afalg`.

Suppressed comments (1)

wolfcrypt/src/port/af_alg/afalg_aes.c:809

Same as above: the warning comment says sz | 16 bytes but this should be sz + 16 bytes to describe the required buffer sizing for the combined ciphertext+tag path.

 *          and out buffers are large enough to hold cipher text and tag. That is
 *          sz | 16 bytes. The in buffer has tag appended even though it is
 *          const for this wolfSSL API, and sz | 16 bytes are read back into the
 *          out buffer. The input and output buffer is expected to be 64 bit
 *          aligned.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-08-13T01:08:49Z

MemBrowse Memory Report

gcc-arm-cortex-m0plus

FLASH: .text +92 B (+0.1%, 64,975 B / 262,144 B, total: 25% used)

gcc-arm-cortex-m3

FLASH: .text +128 B (+0.1%, 123,707 B / 262,144 B, total: 47% used)

gcc-arm-cortex-m4

FLASH: .text +128 B (+0.1%, 201,967 B / 262,144 B, total: 77% used)

gcc-arm-cortex-m4-baremetal

FLASH: .text +128 B (+0.2%, 67,555 B / 262,144 B, total: 26% used)

gcc-arm-cortex-m4-crypto-only

FLASH: .text +128 B (+0.1%, 176,106 B / 262,144 B, total: 67% used)

gcc-arm-cortex-m4-dtls13

FLASH: .text +640 B (+0.3%, 183,548 B / 1,048,576 B, total: 18% used)

gcc-arm-cortex-m4-min-ecc

FLASH: .text +128 B (+0.2%, 62,533 B / 262,144 B, total: 24% used)

gcc-arm-cortex-m4-openssl-compat

FLASH: .text +256 B (+0.0%, 775,804 B / 1,048,576 B, total: 74% used)

gcc-arm-cortex-m4-pkcs7

FLASH: .text +128 B (+0.1%, 214,958 B / 262,144 B, total: 82% used)

gcc-arm-cortex-m4-pq

FLASH: .text +128 B (+0.0%, 297,756 B / 1,048,576 B, total: 28% used)

gcc-arm-cortex-m4-rsa-only

FLASH: .text +128 B (+0.0%, 328,240 B / 1,048,576 B, total: 31% used)

gcc-arm-cortex-m4-sp-math

FLASH: .text +128 B (+0.2%, 62,533 B / 262,144 B, total: 24% used)

gcc-arm-cortex-m4-tls12

FLASH: .text +128 B (+0.1%, 124,467 B / 262,144 B, total: 47% used)

gcc-arm-cortex-m4-tls13

FLASH: .text +128 B (+0.1%, 238,497 B / 262,144 B, total: 91% used)

gcc-arm-cortex-m7

FLASH: .text +128 B (+0.1%, 201,967 B / 262,144 B, total: 77% used)

gcc-arm-cortex-m7-pq

FLASH: .text +128 B (+0.0%, 298,652 B / 1,048,576 B, total: 28% used)

gcc-arm-cortex-m7-tls13

FLASH: .text +128 B (+0.1%, 238,561 B / 262,144 B, total: 91% used)

linuxkm-pie

Data: __patchable_function_entries +24 B (+0.1%, 26,592 B)

linuxkm-standard

Data: __patchable_function_entries +24 B (+0.0%, 49,400 B)

stm32-sim-stm32h753

FLASH: .text +192 B (+0.1%, 186,352 B / 2,097,152 B, total: 9% used)

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #11162

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port-bugs, wolfcrypt-rs-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src

Findings: 10
10 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

+        0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+        0xba, 0x63, 0x7b, 0x39
+    };
+    WOLFSSL_SMALL_STACK_STATIC const byte a[] =


🔵 [Low] aesgcm_default_test now runs a 20-byte-AAD vector, breaking its documented restriction · API contract violations

aesgcm_default_test is documented as "tests that only use 12 byte IV and 16 or less byte AAD" and is the reduced-capability GCM test run on ports where aesgcm_test is skipped. The sub-test it now calls uses a 20-byte AAD a[], breaking that guarantee. The comment stating the restriction also now sits above aesgcm_reuse_ctx_test rather than the function it describes.

Fix: Trim the AAD to 16 bytes (adjusting the expected tag) and move the restriction comment back directly above aesgcm_default_test.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

-    if (aes && (aes->dir != AES_ENCRYPTION)) {
+    /* argument sanity checks come before the key usage check so that bad
+     * arguments always report BAD_FUNC_ARG, matching the software version */
+    if (aes == NULL || out == NULL || in == NULL) {


🔵 [Low] AF_ALG ECB sanity checks omit the block-size length check the software path enforces · API contract violations

The new check block is documented as matching the software version, but sz % WC_AES_BLOCK_SIZE is not rejected. aes.c:16403 returns BAD_LENGTH_E for a partial block, while the AF_ALG path forwards it to the kernel and surfaces WC_AFALG_SOCK_E. wc_AesEcbDecrypt (line 1037) has the same gap.

Related known finding #7443 (similar but distinct): Both concern AF_ALG AES API-contract behavior, but this is ECB partial-block input validation in wc_AesEcbEncrypt/Decrypt, whereas #7443 is AES-GCM authentication-tag-size caching in different operations. Their root causes and required patches are distinct.

Fix: Return BAD_LENGTH_E when sz % WC_AES_BLOCK_SIZE != 0 in both wc_AesEcbEncrypt and wc_AesEcbDecrypt.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

+        ERROR_OUT(WC_TEST_RET_ENC_NC, out);
+
+    /* decrypt with the same structure that was just used to encrypt */
+    ret = wc_AesGcmDecrypt(enc, resultP, resultC, sizeof(p), iv1, sizeof(iv1),


⚪ [Info] GCM reuse test never exercises the authentication-failure return of wc_AesGcmDecrypt · Missing edge-case coverage on a function the PR also changed

wc_AesGcmDecrypt is modified by this PR, but its AES_GCM_AUTH_E branch stays uncovered for AF_ALG and devcrypto: aesgcm_test() (test.c:2920) and test_wc_AesGcmEncryptDecrypt() (tests/api/test_aes.c:3339) are both compiled out for those ports, and the new test only asserts success.

Fix: Add a decrypt call with a flipped tag byte to aesgcm_reuse_ctx_test and assert AES_GCM_AUTH_E.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z


 #elif defined(WOLFSSL_AFALG)
    /* implemented in wolfcrypt/src/port/af_alg/afalg_aes.c */
+    #define _AesEcbEncrypt(aes, out, in, sz) wc_AesEcbEncrypt(aes, out, in, sz)


🟠 [Medium] AES-XTS bulk ECB now routed through AF_ALG helper that ignores short socket transfers · Cryptographic correctness

The new macros make _AesXtsHelper (aes.c:17773/17776) run the multi-block XTS ECB step through AF_ALG's wc_Afalg_AesDirect, which checks only < 0 on sendmsg/read. A short transfer returns 0 while out still holds plaintext XOR tweak, which the caller then un-XORs back to cleartext.

Related known finding #3557 (similar but distinct): Both concern AES-XTS behavior in aes.c, but the candidate faults the AF_ALG-backed bulk ECB operation in _AesXtsHelper due to unchecked short socket I/O, whereas #3557 faults missing XTS data-unit length enforcement in wc_AesXtsEncrypt. The root causes and required patches are separate.

Fix: Require the AF_ALG sendmsg/read byte counts to equal sz, or split the XTS ECB call into socket-sized chunks.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

+        ERROR_OUT(WC_TEST_RET_ENC_NC, out);
+
+    /* decrypt with the same structure that was just used to encrypt */
+    ret = wc_AesGcmDecrypt(enc, resultP, resultC, sizeof(p), iv1, sizeof(iv1),


🔵 [Low] AES-GCM context-reuse test omits decrypt-first ordering and tag-rejection assertion · Missing edge-case coverage on a function the PR also changed

The test only covers encrypt->decrypt->encrypt on a shared Aes. Decrypt-first-then-encrypt and the tag-mismatch rejection path are never exercised, so an encrypt still running in the kernel's decrypt mode, or a decrypt that never checks the tag, passes the suite.

Fix: Add a decrypt-first-then-encrypt sequence and a corrupted-tag decrypt asserted to return AES_GCM_AUTH_E.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

+        ERROR_OUT(WC_TEST_RET_ENC_NC, out);
+
+    /* decrypt with the same structure that was just used to encrypt */
+    ret = wc_AesGcmDecrypt(enc, resultP, resultC, sizeof(p), iv1, sizeof(iv1),


🔵 [Low] New AES-GCM reuse test never exercises the authentication-failure path · Missing edge-case coverage on a function the PR also changed

The test only covers successful decrypt with a valid tag. The defect class it targets is an AF_ALG socket left in the wrong operation direction, where no authentication is performed at all, so a corrupted-tag case returning AES_GCM_AUTH_E is the branch that distinguishes real verification from none.

Fix: Add a decrypt call with one tag byte flipped and assert it returns AES_GCM_AUTH_E.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

@@ -19271,8 +19271,142 @@ static wc_test_ret_t aesgcm_default_test_helper(byte* key, int keySz, byte* iv,
 /* tests that only use 12 byte IV and 16 or less byte AAD


⚪ [Info] NIST/12-byte-IV comment now documents the wrong function · Copy-paste errors

The comment "tests that only use 12 byte IV and 16 or less byte AAD / test vectors are from NIST SP 800-38D" belonged to aesgcm_default_test, but aesgcm_reuse_ctx_test was inserted between them, so it now reads as describing a test that uses a 20-byte AAD.

Fix: Move the NIST/IV/AAD comment back down so it sits directly above aesgcm_default_test.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

-    if (aes && (aes->dir != AES_ENCRYPTION)) {
+    /* argument sanity checks come before the key usage check so that bad
+     * arguments always report BAD_FUNC_ARG, matching the software version */
+    if (aes == NULL || out == NULL || in == NULL) {


⚪ [Info] New AF-ALG argument sanity checks have no test coverage · Missing edge-case coverage on a function the PR also changed

The PR adds BAD_FUNC_ARG guards to wc_AesEcbEncrypt, wc_AesEcbDecrypt, wc_AesEncryptDirect and wc_AesDecryptDirect, and reorders them ahead of the KEYUSAGE_E check, but the only test added covers AES-GCM reuse; no test passes NULL or a wrong-direction key to these entry points.

Related known finding #7442 (similar but distinct): Both concern AF-ALG AES behavior, but the candidate is missing tests for newly added argument/key-direction guards in ECB/direct APIs; issue 7442 is a stale socket operation-direction bug in GCM decrypt. The operations, root causes, functions, and required patches differ.

Fix: Add assertions that NULL aes/out/in return BAD_FUNC_ARG and a wrong-direction key returns KEYUSAGE_E.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

-    if (aes && (aes->dir != AES_ENCRYPTION)) {
+    /* argument sanity checks come before the key usage check so that bad
+     * arguments always report BAD_FUNC_ARG, matching the software version */
+    if (aes == NULL || out == NULL || in == NULL) {


🔵 [Low] AF_ALG ECB entry points omit the block-size length check enforced by the portable implementation · Buffer overflows

The reworked argument validation in wc_AesEcbEncrypt/wc_AesEcbDecrypt adds NULL guards but not the sz % WC_AES_BLOCK_SIZE rejection that the portable wc_AesEcbEncrypt (wolfcrypt/src/aes.c:16403) enforces. wc_Afalg_AesDirect then ignores the read() byte count, so a non-block-multiple sz returns 0 while the trailing sz % 16 bytes of out retain stale caller memory.

Related known finding #7442 (similar but distinct): Both affect AF_ALG AES code in afalg_aes.c, but this finding concerns ECB encrypt/decrypt accepting non-block-aligned lengths because the entry points omit validation and the direct helper ignores short reads. Issue 7442 concerns GCM decrypt using a stale encryption direction after socket reuse. The faulting operations, root causes, functions, and required patches differ.

Fix: Return BAD_LENGTH_E when sz % WC_AES_BLOCK_SIZE != 0 in both AF_ALG ECB entry points, matching wolfcrypt/src/aes.c:16403.

wolfSSL-Fenrir-bot · 2026-08-14T23:27:02Z

+        ERROR_OUT(WC_TEST_RET_ENC_NC, out);
+
+    /* decrypt with the same structure that was just used to encrypt */
+    ret = wc_AesGcmDecrypt(enc, resultP, resultC, sizeof(p), iv1, sizeof(iv1),


🔵 [Low] New AES-GCM context-reuse test has no negative authentication assertion · Missing edge-case coverage on a function the PR also changed

aesgcm_reuse_ctx_test exercises only successful encrypt/decrypt/encrypt on a reused context. wc_AesGcmDecrypt is modified by this PR, but no case asserts that a tampered resultC or resultT yields AES_GCM_AUTH_E, so a build where the reused AF_ALG socket authenticates against the wrong tag length or skips verification still passes.

Fix: Add a decrypt call with a flipped ciphertext byte and one with a flipped tag byte, asserting both return AES_GCM_AUTH_E.

JacobBarthelmeh added 2 commits August 12, 2026 16:06

Fix for reuse of AES-GCM struct with encrypt and decrypt f-7442

801063d

add documentation for f-8238 report

6c8a44a

JacobBarthelmeh self-assigned this Aug 12, 2026

Copilot AI lite review requested due to automatic review settings August 12, 2026 22:24

Copilot started reviewing on behalf of JacobBarthelmeh August 12, 2026 22:26 View session

Copilot AI reviewed Aug 12, 2026

View reviewed changes

Comment thread wolfcrypt/src/port/af_alg/afalg_aes.c

JacobBarthelmeh force-pushed the afalg branch 2 times, most recently from 32d937b to d6924df Compare August 13, 2026 16:14

add github actions CI test for AF-ALG

626950b

JacobBarthelmeh force-pushed the afalg branch from d6924df to 626950b Compare August 13, 2026 20:04

JacobBarthelmeh added 2 commits August 13, 2026 16:00

AF-ALG redirect for AES-ECB function

9304abe

add better sanity checks on input arguments

4fb7883

JacobBarthelmeh requested a review from wolfSSL-Fenrir-bot August 14, 2026 22:25

wolfSSL-Fenrir-bot reviewed Aug 14, 2026

View reviewed changes

		@@ -19271,8 +19271,142 @@ static wc_test_ret_t aesgcm_default_test_helper(byte* key, int keySz, byte* iv,
		/* tests that only use 12 byte IV and 16 or less byte AAD

Uh oh!

Conversation

JacobBarthelmeh commented Aug 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

github-actions Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #11162

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Aug 13, 2026 •

edited

Loading