Extended robustness test suite (proptest / fuzz / miri / kani) — offered as-is - #612
Open
enomado wants to merge 5 commits into
Open
Extended robustness test suite (proptest / fuzz / miri / kani) — offered as-is#612enomado wants to merge 5 commits into
enomado wants to merge 5 commits into
Conversation
An external, independent robustness audit of the parser+serializer. Adds reproducible tests exercising angles not currently in CI. Nothing here changes crate behaviour; the kani code is `#[cfg(kani)]`-gated and inert in normal builds. What's added: - tests/zz_audit_proptest.rs — proptest harness over a recursive `Value`: parser never panics on arbitrary input; the serializer never emits RON its own parser rejects; ser->parse is a stable fixpoint on canonical values; pretty and compact output parse to the same Value. ~14k cases green. - fuzz/fuzz_targets/roundtrip.rs — fixpoint round-trip target (parse -> serialize -> parse must agree). fuzz/fuzz_targets/raw_value.rs — drives the crate's only `unsafe` (RawValue transmutes + trim_boxed drain) under AddressSanitizer. - src/kani_verification.rs + `#[cfg(kani)] mod kani_parse_proofs` in parse.rs — bounded proofs. `decode_hex` and `is_int_char` are proven total/panic-free over the entire `char` domain. (The full `from_str` path exceeds CBMC's budget.) Findings (all reproduced as tests; correctness/fidelity, NOT soundness — no memory-safety issue was found by miri or 11.7M ASAN fuzz executions): - zz_finding_number_narrowing.rs: `Value` round-trip is not type-preserving. The default serializer emits no type suffix, so `1.5f64` -> "1.5" -> f32 and `5u16` -> "5" -> u8. Value survives, numeric type narrows. (fuzzer-found, minimised to `NaNf64`.) - zz_finding_implicit_some.rs: under IMPLICIT_SOME, map-key serialization is not injective — keys `Some(())` and `()` both print as `()`, yielding a duplicate key that silently drops an entry on re-parse. (proptest-found; same root cause as the documented IMPLICIT_SOME ambiguity class.) Offered as-is — take whatever is useful. No CI changes proposed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `roundtrip` fuzz target, on a longer run, minimised `924444480f64..922.` to a genuine value-drift: an f32-typed `Value` does not survive serialize->parse. 924444480 is f32-exact, so `Value` infers F32. Rust prints that f32's shortest round-tripping decimal as "924444500.0" (a different decimal mapping to the same f32), which ron serializes. Re-parsing "924444500.0" no longer fits f32, so ron falls back to F64(924444500.0) — the value drifted 924444480 -> 924444500. This is the same missing-suffix root cause as the number-narrowing note, but here the numeric VALUE changes, not just its type. Fidelity, not soundness. Pinned by tests/zz_finding_float_precision.rs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Follow-up: a longer fuzz run of the Minimised from
Same missing-suffix root cause as the number-narrowing note, but here the |
Adds a second case showing the drift is disambiguable: `to_string_pretty` with `number_suffixes(true)` writes `...f32`, so the parser recovers the exact f32 Value. Two residual gaps worth a maintainer's eye: suffixes are off by default, and compact `to_string` ignores the flag entirely (number_suffixes() only reads a pretty config), so a lossless *compact* f32 Value round-trip isn't reachable via the public API today. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- заменить let...else (1.65+) на match в тестах — MSRV ron = 1.64.0 - применить rustfmt к zz_* тестам (перенос длинных assert_eq!/prop_oneof!) - roundtrip.rs: doc-коммент без list-элементов (чинит clippy doc_lazy_continuation) + skip float-значений (drift ron-rs#613) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi! 👋 This is an independent, external robustness audit of the parser +
serializer, offered purely as an FYI — take whatever is useful and drop the
rest. No pressure, no CI changes proposed, and nothing here alters crate
behaviour (the
kanicode is#[cfg(kani)]-gated and inert in normal builds).Headline
No memory-safety issue was found.
ronheld up across every angle:RawValueunsafe— 0 UBfrom_str, a new fixpointroundtrip, and a newraw_valuetarget — 11.7M execs, 0 crashesdecode_hex+is_int_charproven total / panic-free over the entirechardomainWhat's added
tests/zz_audit_proptest.rs— property harness over a recursiveValue.fuzz/fuzz_targets/roundtrip.rs— fixpoint round-trip (parse → ser → parsemust agree).fuzz/fuzz_targets/raw_value.rs— drives the crate's onlyunsafe(RawValue transmutes +trim_boxeddrain) under AddressSanitizer.src/kani_verification.rs+#[cfg(kani)] mod kani_parse_proofsinparse.rs— bounded proofs. (Note: the fullfrom_strpath overwhelms CBMC's budget — expected for string-scanning parsers; the pure helpers verify instantly.)Two correctness/fidelity notes (NOT soundness), each pinned by a test
Valueround-trip is not type-preserving (zz_finding_number_narrowing.rs). The default serializer emits no type suffix, so1.5f64→"1.5"→f32and5u16→"5"→u8. The value survives; the numeric type narrows. Theroundtripfuzzer found and minimised this toNaNf64.IMPLICIT_SOMEmap serialization is not injective (zz_finding_implicit_some.rs). KeysSome(())and()both print as(), producing a duplicate-key map that silently drops an entry on re-parse. Same root cause as the documentedIMPLICIT_SOMEambiguity class in502_known_bugs.rs; proptest rediscovered it automatically.Both are surprising-but-arguably-known consequences of existing design trade-offs, not new bugs — sharing them in case the concrete reproductions are useful.
Thanks for
ron! 🦀🤖 Generated with Claude Code