Skip to content

Fix quadratic escaped string parsing in escaped_byte_buf - #610

Merged
juntyr merged 3 commits into
ron-rs:masterfrom
enomado:fix/609-quadratic-escaped-string
Jul 16, 2026
Merged

Fix quadratic escaped string parsing in escaped_byte_buf#610
juntyr merged 3 commits into
ron-rs:masterfrom
enomado:fix/609-quadratic-escaped-string

Conversation

@enomado

@enomado enomado commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #609.

escaped_byte_buf calls find('"') once per escape. That call scans from the cursor to
the closing quote, while the cursor only advances past one escape per iteration —
so a string with N escapes rescans the tail N times, making string parsing O(N²) in the
number of escapes.

Unlike #607, this is not limited to ron::Value / deserialize_any: it affects any
deserialization of a string containing escapes, including the typed path.

Root cause

The quadratic scan came in with #534, which replaced a single scan for the first of
" / \\:

let (new_i, end_or_escape) = self
    .find_char_index(|c| matches!(c, '\\' | '"'))
    .ok_or(Error::ExpectedStringEnd)?;

with "find the closing quote, then look for a backslash before it". That is genuinely
faster for the common no-escape case, so this PR keeps it as the fast path outside
the loop. Inside the loop, though, only the first of " / \\ is ever needed — scanning
just to the nearest delimiter is both sufficient and linear.

Affected releases (git tag --contains 08c691d0): v0.9.0 … v0.12.2.

Result

["\n\n\n…"], release build, at d0e99bc:

input before after ratio before ratio after
25 KB 6.64 ms 0.10 ms
50 KB 26.24 ms 0.19 ms x3.95 x1.94
100 KB 103.38 ms 0.38 ms x3.94 x1.97
200 KB 414.43 ms 0.76 ms x4.01 x2.00
400 KB 1652.96 ms 1.54 ms x3.99 x2.02

1073x at 400 KB, and the per-doubling ratio goes from ~4 (quadratic) back to ~2
(linear). Throughput: 0.2 MB/s → 260 MB/s.

Both branches already did the same extend_from_slice, so the fix also makes the loop
slightly shorter.

Tests

The regression test asserts on scaling, not absolute time, so it stays meaningful on
slow or noisy CI: the ratio is ~2 when linear and ~4 when quadratic, and the threshold
(3.0) sits far from both. It fails on the pre-fix parser (measured ratio 3.96) and passes
after. Same shape as the test in #608.

The existing suite (453 tests) stays green, plus the 2 new ones.

Note on error reporting for invalid input

For an unterminated string whose invalid escape follows an escaped quote — e.g.
"a\"b\c — this now reports InvalidEscape instead of ExpectedStringEnd. That is what
the parser did before #534 (verified by running the pre-#534 tree), and it is arguably the
more accurate diagnostic, since there genuinely is an invalid escape.

The outer find('"') is deliberately left untouched, so the common case — an unterminated
string with no escapes, e.g. the (4, "Hello) case pinned in de::tests::forgot_apostrophes
— still reports ExpectedStringEnd exactly as today. No existing test changes behaviour.

Happy to preserve the current error exactly if you'd prefer; it costs a little extra state
to track the last found quote across iterations.

enomado and others added 3 commits July 15, 2026 18:47
escaped_byte_buf called find('"') once per escape. That call scans from the
cursor to the closing quote, while the cursor only advances past one escape per
iteration, so a string with N escapes rescanned the tail N times, making string
parsing O(N^2) in the number of escapes.

Unlike ron-rs#607 this is not limited to ron::Value / deserialize_any: it affects any
deserialization of a string containing escapes, including the typed path.

The quadratic scan was introduced in ron-rs#534, which replaced a single scan for the
first of '"' / '\\' with "find the closing quote, then look for a backslash
before it". That is genuinely faster for the common no-escape case, so keep it
as the fast path outside the loop; inside the loop only the first of '"' / '\\'
is needed, and scanning just to the nearest delimiter is enough.

Restores linear scaling: a 400KB string of escapes goes from 1652.96ms to
1.54ms (1073x), and the ratio per doubling goes from 3.99 back to 2.02.

Add a regression test asserting on scaling rather than absolute time, so it
stays meaningful on slow or noisy machines: the ratio is ~2 when linear and ~4
when quadratic, and the test fails on the pre-fix parser.

Note on error reporting for invalid input: for an unterminated string whose
invalid escape follows an escaped quote (e.g. "a\"b\c), this reports
InvalidEscape rather than ExpectedStringEnd, which is what the parser did
before ron-rs#534. The outer find('"') is left untouched, so unterminated strings
without escapes still report ExpectedStringEnd as they do today.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
std::hint::black_box is only stable since Rust 1.66 (E0658 on the 1.64 MSRV job); drop it — the .unwrap()ed cross-crate from_str::<Value> can't be elided anyway. Also split the find(['"', '\\']) call across lines to satisfy rustfmt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@juntyr
juntyr merged commit 31529b8 into ron-rs:master Jul 16, 2026
10 checks passed
@enomado

enomado commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the CI: dropped the black_box hint (only stable since Rust 1.66, so it broke the 1.64 MSRV job with E0658) and split the long find(['"', '\\']) line to satisfy rustfmt. All green now.

enomado added a commit to enomado/ron that referenced this pull request Jul 16, 2026
ron has ~450 correctness tests but none that guard the asymptotic cost of parsing. That gap let ron-rs#534 regress escaped_byte_buf from O(n) to O(n^2) across v0.9.0..=v0.12.2 — four releases — with every test green, and a second O(n^2) entered next_bytes_is_float in v0.12.2 (ron-rs#602).

tests/complexity_scaling.rs asserts that parse time grows at most linearly: it doubles the input and checks the time ratio (~2 linear vs ~4 quadratic; threshold 3.0), using min-of-N + median to stay robust to timing noise. The tests are #[ignore]d and run in a dedicated release CI job, so they never slow down or flake the normal test job.

Two forms guard the regressions fixed in ron-rs#608 (next_bytes_is_float) and ron-rs#610 (escaped_byte_buf); two linear controls keep the harness honest. On the pre-fix tree (d0e99bc) those two forms report ratio ~4.0 and fail the 3.0 threshold; on current master they are ~2.0 and pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parsing strings with many escapes is quadratic (regressed in #534)

2 participants