persistence: stable segment metadata and file-format compatibility policy #118

Summary

Document and test a stable compatibility policy for persisted ordvec index files and IndexMetadata.

The goal is ecosystem compatibility: downstream vector databases and search systems need to know when a .tvr* file can be shipped, cached, replicated, probed, upgraded, or rejected. ordvec can be a good neighbor by making its primitive artifact contract explicit while leaving database durability and provenance outside the crate.

Current state

probe_index_metadata() already validates fixed headers and declared byte shape without reading full payloads. That is the right building block. The missing piece is a documented compatibility promise and golden fixtures that downstream systems can rely on.

Proposed scope

• Treat IndexMetadata as the manifest-friendly segment descriptor.
• Document compatibility rules for .tvr, .tvrq, .tvbm, and .tvsb files.
• Add golden fixture files or generated fixture tests for each persisted format.
• State what requires a format version bump.
• State what ordvec intentionally does not provide: checksums, signatures, transaction boundaries, provenance, or trust policy.

Acceptance criteria

• Docs describe forward/backward compatibility expectations for persisted files.
• Tests prove current loaders/probers read committed fixtures.
• Breaking format changes require explicit versioning, migration, or documented rejection.
• probe_index_metadata() behavior is documented as the allocation-resistant preflight path for host stores.
• The docs include a short example of using metadata in an external segment manifest.

Non-goals

• No signing/key management.
• No manifest store.
• No database replication protocol.
• No transaction model.