Anchor prepared patterns on the rarest literal run#8
Open
killerra wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anchor-selection follow-up, stacked on #7. The static heuristic prefers the longest literal window, ranked by a fixed byte-rarity table — it cannot know how common a run actually is in the binary being scanned.
? ? 48 8b 0d ? ? ? ? 15 7canchors on48 8b 0d(639 occurrences in the PE fixture) even though15 7coccurs only 32 times, so the executor verifies ~20× more candidates than necessary.Changes
bench(scanner): cover the prepared-pattern scan path
matches_preparedbenches next to the existingmatches_codeones, plus aprepare_patterngroup timing preparation itself. Committed first so the criterion baseline covers the new bench points on unchanged behaviour.perf(scanner): anchor prepared patterns on the rarest literal run
Scanner::prepare_pattern/prepare_pattern_strnow extract one candidate window per maximal literal run of the pattern and count each candidate's actual occurrences in the view's code withmemchr::memmem(early-exit once a candidate exceeds the current best). The prepared pattern anchors on the run with the fewest occurrences; ties prefer the longer window, then the static score.48 8b 0dis a common instruction encoding rather than three coincidental bytes.matches_code,finds_code) and the view-lessPreparedPattern::from_atomsare untouched.Benchmarks (
scan_anchor, criterion A/B against the bench-only commit)PE64 fixture:
matches_prepared/leading_wildcardsmatches_prepared/late_rare_runmatches_prepared/weak_lead_two_runsmatches_prepared/strong_lead_controlmatches_code/*(all four)prepare_pattern(≥2 literal runs)prepare_pattern(single run)ELF64 fixture: all scan benches flat (±3%). On that fixture no candidate run is materially rarer than the static pick, so counting confirms the static choice — the refinement pays off exactly where the binary makes a difference and never costs anything at scan time.
Verification
? ? 48 8b 0d ? ? ? ? 15 7c→ anchors on15 7c).cargo test --workspace --all-targets --all-features+cargo test --workspace --docgreen.cargo clippy --workspace --all-targets --all-features -- -D warnings -D clippy::allclean;cargo fmt --all -- --checkclean.