Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
62f38fa to
6732495
Compare
6732495 to
625afeb
Compare
625afeb to
650d76b
Compare
Store Conformance Harness OverviewRelated Fix PRs (split out from the initial conformance run)
|
Follow-Up Roadmap: Stopping the Regression Whack-a-MoleAn audit of the last ~150 commits and recent issues shows that recurring regressions fall into five main clusters, largely because existing tests exercise individual components against mocks at their immediate boundaries or override production defaults (like This PR establishes Pillar 1 (Store Topology & Fault-Injection Conformance). Below is the follow-up plan to cover the remaining failure modes across the stack: |
b7r6
left a comment
There was a problem hiding this comment.
Thanks for adding this conformance harness. Exercising composed stores with the default filesystem block size is useful coverage. I would like to address two gaps before merging: the repair test can pass without actually evicting the fast-tier entry, and the invalid-upload suite does not exercise oversized uploads despite describing that contract. Details and suggested assertions are inline.
Review was based on the diff and relevant store implementations; I did not run the tests.
|
Thanks for the review! Both points are addressed in 7cbf174. |
What and why
Store wrappers and leaf backends have primarily been tested in isolation, often with non-default settings such as
block_size: 1onFilesystemSpec, allowing contract mismatches across composed store tiers to slip through. This change introducesstore_conformance_test, a fault-injecting topology conformance harness that exercises a matrix of single-tier and multi-tier store stacks with production defaults (includingFilesystemSpec's default 4 KiBblock_size).Architecture & Topology Matrix
It covers six shared contracts across the topology matrix:
has()content size: Exacthas()content length and full/partialget_partround-trips across sub-block (1 B,146 B), exact-block (4096 B), multi-block (4097 B), and multi-chunk LZ4 (68,869 B) payload boundaries.FilesystemStore,RefStore -> FilesystemStore,CompressionStore,ShardStore, andDedupStore).Unavailable,ResourceExhausted,Aborted) on mid-stream upload and download failures through wrapper stacks.StoreDriver::remove()): Truncated fast-tier reads triggerDataLossand evict the bad fast copy viaremove(), while keeping protected durable slow-tier copies intact so the next read refills cleanly.has()queries against an evicting 4 KiB-blockFilesystemStorefast tier.How was this verified?
Ran
bazel test //nativelink-store:integration_tests/store_conformance_test_testundernix develop(all 6 conformance tests pass, along with Clippy, rustfmt, and pre-commit hooks).Risk
Low. Test-only addition (
nativelink-store/tests/store_conformance_test.rsandnativelink-store/BUILD.bazel) with no production code changes.AI assistance
Built the conformance harness with Cursor Agent; reviewed and verified against the Bazel test suite.