persistence: fail-closed malformed index loader contract #151

Summary

Define and test the fail-closed malformed-input contract for persisted ordvec index loaders.

ordgrep can verify store-level hashes, config, row counts, dimensions, and manifest membership, but it should not duplicate every low-level .ordvec parser invariant. The primitive crate should own rejection of malformed/truncated/foreign binary index files before callers rely on loaded indexes.

Scope

• Audit RankQuant, SignBitmap, and other persisted index loader/prober paths for malformed input handling.
• Ensure loaders reject unsupported bits, invalid headers, truncated payloads, row-count mismatches, dimension mismatches, overflowing len * dim or packed-byte calculations, invalid packing lengths, and unexpected trailing bytes where the format requires exact length.
• Keep allocation-resistant metadata probing aligned with persistence: stable segment metadata and file-format compatibility policy #118, but test full loader failure modes too.
• Return structured or at least stable error context suitable for downstream CLIs such as ordgrep.

Acceptance Criteria

• Unit or fuzz tests cover malformed headers, truncated files, forged dimensions/lengths, unsupported bits, and oversized arithmetic cases.
• Full loaders fail closed before unsafe/SIMD search paths can observe malformed state.
• probe_index_metadata() and full loader behavior are consistent and documented where they intentionally differ.
• Downstream callers can treat loader success as the primitive-level binary safety boundary, while keeping manifest/provenance checks outside this issue.

Non-goals

• No signing or provenance schema.
• No database transaction or crash-consistency model.
• No ordgrep-specific store manifest logic.