Fixes for static memory builds with WOLFSSL_NO_MALLOC - #11432
Conversation
wolfSSL-Fenrir-bot
left a comment
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #11432
Scan targets checked: none
Failed targets: wolfcrypt-src, wolfcrypt-bugs
wolfSSL-Fenrir-bot
left a comment
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #11432
Error: CalledProcessError
6c11cd5 to
5a03400
Compare
wolfSSL-Fenrir-bot
left a comment
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #11432
Scan targets checked: wolfcrypt-src, wolfcrypt-bugs
Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
5a03400 to
667312f
Compare
wolfSSL-Fenrir-bot
left a comment
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #11432
Scan targets checked: wolfcrypt-src, wolfcrypt-bugs
Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
667312f to
8b7f6a8
Compare
wolfSSL-Fenrir-bot
left a comment
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #11432
Scan targets checked: wolfcrypt-src, wolfcrypt-bugs
Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
|
Jenkins retest this please |
ParseCert() guarded its RSA public-key copy with !WOLFSSL_NO_MALLOC, while StoreKey() guards the equivalent copy for every non-RSA key with !WC_ASN_NO_HEAP. Those are not the same condition: WC_ASN_NO_HEAP is auto-defined only when WOLFSSL_NO_MALLOC and NO_WOLFSSL_MEMORY are set without XMALLOC_USER or WOLFSSL_STATIC_MEMORY, so a static-memory build defines WOLFSSL_NO_MALLOC yet still has a working allocator. In such a build the copy was skipped, cert->pubKeyStored stayed 0, and FillSigner() therefore never populated signer->publicKey/pubKeySize. ParseCertRelative() then passed a NULL key and a zero key size to ConfirmSignature(), which rejects them with BAD_FUNC_ARG before its WOLFSSL_ENTER. The effect was that no certificate issued by an RSA CA could be verified against it - wolfSSL_CertManagerVerifyBuffer() and TLS peer validation alike - while ECC, Ed25519, Ed448 and ML-DSA CAs worked, because those keys travel through StoreKey(). Use WC_ASN_NO_HEAP in all three guards, including the one on the ptr declaration. FreeDecodedCert() and FreeSigner() already key off pubKeyStored, so ownership and freeing are unchanged.
…uilds With WOLFSSL_NO_MALLOC there is no allocator behind a NULL-heap XMALLOC, so an allocation made outside any CTX or SSL object can only be served from the global heap hint. Loading a static memory pool never set that hint, and nothing else does either, so the hint stayed NULL for an application that had in fact handed wolfSSL a pool. wolfSSL_Init() makes such an allocation: under OPENSSL_EXTRA it seeds the compatibility-layer RNG, whose _InitRng() call allocates with a NULL heap. That returned NULL, wolfSSL_Init() reported WC_INIT_E, and every wolfSSL_CTX_new_ex() that triggered the lazy init failed - so no TLS connection could be established at all in a static-memory build, even though the pool was large enough. The example client and server both hit this, from their two different entry points: the server through wolfSSL_CTX_load_static_memory() and the client through a direct wc_LoadStaticMemory() call. Set the hint in wc_LoadStaticMemory_ex(), which both paths funnel through, and only when no hint has been set yet, so an application that manages the global hint itself keeps control. Adopt a pool only when it holds at least one general bucket. wc_partition_static_memory() places every chunk of a WOLFMEM_IO_POOL or WOLFMEM_IO_POOL_FIXED pool on the heap's io list and leaves the general buckets empty, and it also reports success for a buffer with room for the tracking structures but not for a single bucket. Adopting either one produces a global allocator that fails every general request and then blocks a usable pool loaded afterwards from taking the hint. Testing the bucket lists rather than the flag covers both cases, and is also right under WOLFSSL_STATIC_MEMORY_LEAN, where the IO flag is ignored and such a pool does get general buckets. An earlier version of this commit claimed an IO pool could not displace the general one; that only held when the general pool happened to load first. Clear the hint in wc_UnloadStaticMemory() when it refers to the heap being unloaded. The hint lives inside the caller's pool buffer, which the caller may reuse once the pool is gone, and the mutex it reaches through has just been destroyed. This part is not restricted to WOLFSSL_NO_MALLOC: an application setting the global hint by hand has always been able to leave it dangling the same way. scripts/resume.test and scripts/tls13.test go from failing to passing with --enable-staticmemory -DWOLFSSL_NO_MALLOC. The remaining failures in that configuration are unrelated: they are test programs that read certs from files without ever loading a pool.
With WOLFSSL_NO_MALLOC there is no system heap to fall back on, so an allocation the compatibility layer makes with a NULL heap is served from the static pool the test loads rather than by malloc. gTestMemory was sized for the wolfCrypt tests alone, so those extra allocations exhausted it and openssl_test() failed - first in wolfSSL_CRYPTO_malloc(), then, as the pool was enlarged, further along in wolfSSL_X509_load_certificate_file(). Give that combination its own 1 MB pool. Every other configuration keeps the size it had. testsuite/testsuite.test now passes with --enable-staticmemory -DWOLFSSL_NO_MALLOC. scripts/unit.test still fails there for an unrelated reason: it asks for a single 16384 byte buffer while LARGEST_MEM_BUCKET is 16128 in this configuration, which no pool size can satisfy.
8b7f6a8 to
e364887
Compare
|
Jenkins retest this please |
|
|
||
| for (i = 0; i < WOLFMEM_MAX_BUCKETS; i++) { | ||
| if (heap->ava[i] != NULL) { | ||
| (void)wolfSSL_SetGlobalHeapHint(hint); |
There was a problem hiding this comment.
I'm not a fan of the automatic setting and unsetting of the global heap hint. The setting of the global heap hint itself is not mutex protected so there could be obscure edge race cases here on which heap hint gets set. It's meant for an application to manage setting the global heap hint if wanted otherwise the build should fail on malloc when not set and no heap hint has been used.
There was a problem hiding this comment.
I do like the WC_ASN_NO_HEAP fix in asn.c in this PR though!
4 participants
WOLFSSL_NO_MALLOCremoves only the stdio malloc fall-back, so a build that also has--enable-staticmemorystill has a working allocator. Two places treat that configuration as having no allocator at all: one makes every RSA CA useless for chain verification, the other stops any TLS connection being established. They are independent and sit in different files. A third commit sizes the wolfCrypt test pool somake checkexercises the configuration.Found while running wolfCert's CI matrix, which builds it.
1. RSA public key never reaches the CA Signer
ParseCert()guarded its RSA public-key copy with!defined(WOLFSSL_NO_MALLOC), whileStoreKey()guards the equivalent copy for every non-RSA key with!defined(WC_ASN_NO_HEAP). Only the latter means "genuinely no allocator", which is what the copy depends on, so a static memory build skipped it despite having somewhere to copy to.cert->pubKeyStoredtherefore stayed 0,FillSigner()never populatedsigner->publicKey/pubKeySize, andConfirmSignature()was handed a NULL key and zero size, which it rejects withBAD_FUNC_ARG. No certificate issued by an RSA CA could be verified against it, throughwolfSSL_CertManagerVerifyBuffer()or TLS peer validation alike. ECC, Ed25519, Ed448 and ML-DSA CAs were unaffected, because those keys travel throughStoreKey().The fix uses
WC_ASN_NO_HEAPin all three guards inParseCert(), including the one on thechar* ptrdeclaration.FreeDecodedCert()andFreeSigner()already key offpubKeyStored, so ownership is unchanged.2. Loading a static pool never set the global heap hint
An allocation made outside any
WOLFSSL_CTXorWOLFSSLobject can only be served from the global heap hint, and loading a pool never set it.wolfSSL_Init()makes exactly such an allocation: underOPENSSL_EXTRAit seeds the compatibility layer RNG with a NULL heap. That returned NULL,wolfSSL_Init()reportedWC_INIT_E, and everywolfSSL_CTX_new_ex()that triggered the lazy init failed, so no TLS connection could be established at all regardless of pool size.The two bundled examples reach this from different entry points, the server through
wolfSSL_CTX_load_static_memory()and the client through a directwc_LoadStaticMemory(), so the fix goes inwc_LoadStaticMemory_ex(), which both funnel through. It applies only when no hint has been set yet, so an application managing the hint itself keeps control.Pools loaded with
WOLFMEM_IO_POOLorWOLFMEM_IO_POOL_FIXEDare skipped.wc_partition_static_memory()puts every chunk of such a pool on the heap's io list and leaves the general buckets empty, so adopting one would give a global allocator that fails every general request, and would then stop a general pool loaded afterwards from taking the hint. Load order does not matter.wc_UnloadStaticMemory()now clears the hint when it refers to the heap being unloaded. The hint lives inside the caller's pool buffer, which the caller may reuse once the pool is gone, and the mutex it reaches through has just been destroyed. That part is not gated onWOLFSSL_NO_MALLOC: an application setting the global hint by hand has always been able to leave it dangling the same way.Reviewers may recall 98a19f9, which removed commented-out
wolfSSL_SetGlobalHeapHint()calls from both examples because the pool "does not handle all memory used on default build". That concern does not apply here: the change is confined toWOLFSSL_NO_MALLOC, where the alternative to serving those allocations from the pool is failing them outright. Builds with a system heap are unaffected.3. wolfCrypt test pool sized for the configuration
Once the pool serves the compatibility layer's NULL-heap allocations,
gTestMemorywas too small andopenssl_test()failed on exhaustion. TheWOLFSSL_NO_MALLOCandOPENSSL_EXTRAcombination now gets 1 MB; every other configuration keeps the size it had.Testing
With
--enable-staticmemory -DWOLFSSL_NO_MALLOC,make checkgoes from 0 of 5 passing to 3 of 5:resume.test,tls13.testandtestsuiteall move from fail to pass. A standalonewolfSSL_CertManagerVerifyBuffer()on an RSA CA in that build goes from -173 toWOLFSSL_SUCCESS, and wolfCert's integration suite passes against it with its local workaround for defect 1 removed.--enable-staticmemorywithoutWOLFSSL_NO_MALLOC- the configuration the unload change newly touches - passesmake check5 of 5. The default configuration passesmake checkandtestwolfcrypt, and a true no-allocator build (-DWOLFSSL_NO_MALLOC -DNO_WOLFSSL_MEMORY) compiles with behaviour unchanged by design. Both predicates in 1 and 2 are unchanged for those two cases, so onlyWOLFSSL_NO_MALLOCbuilds that do have an allocator change behaviour.Two failures deliberately left in place
unit.testreaches a structural limit rather than a sizing one:test_wc_ChaCha20Poly1305_Encrypt_Decrypt_Big()asks for a single 16384 byte buffer whileLARGEST_MEM_BUCKETis 16128 for this feature set. A static pool is partitioned into fixed size buckets, so no pool size satisfies that request, and raising the ceiling changes the layout for every static memory user.pem.testfails becauseexamples/pem/pem.cnever loads a pool at all, so itsXMALLOC(..., NULL, ...)has no source in a build with no system heap.Neither is a regression.