fix: use a scratch buffer in simp3btoken #731
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #251 crashes because "fill" overtakes "s" and produces malformed tokens, leading to an infinite loop. Avoid this possibility by writing the output tokens into a scratch buffer, and copy the result back at the end.
Additionally, re-allocate the buffers if fill is "close to" the end of the scratch buffer. Here "close to" is a guess at how much space might be required per iteration.
Here is a proposed fix of #251. The "magic"
+20is no longer required, but instead there is a magic "how much space is required at the end of the token buffer for each iteration of the while loop?" At least in the future if someone reports thatfillgoes out of bounds oftmptokensit is clear what is going on.Some
TEMPTYspace is still required at the start ofAC.tokensbefore the call ofsimp3btoken:sandtpointers end up in this space even if initially they are moved beyond it.I don't measure any performance penalty due to the allocations (they are not all that large) but in principle one could introduce a permanent
AC.tmptokensto serve as scratch space without the allocations.Any comments?