Add core foundation modules

[JingsongLi]

Summary

• Bring over basic building blocks from poc branch: distance (L2/IP/Cosine + SIMD), blas (sgemm), kmeans (k-means++/hierarchical/streaming), and product quantizer (train/encode/decode/distance table)
• Project infrastructure: .gitignore, LICENSE, NOTICE, deny.toml, workspace Cargo.toml
• Skips higher-level modules (ivfpq, opq, fastscan, shuffler, io) and language bindings (jni, python)

Test plan

• cargo build passes
• cargo test passes (unit tests included in each module)

[leaves12138]

Thanks for putting the initial core modules together. I found a blocker: the current PR does not compile.

Local validation on this head (35259c9741b1a27ee31a2c922a48dc96267e654e):

cargo test --all --all-features

fails in core/src/distance.rs because the AVX2 gather index expressions mix usize and u32, for example around lines 383-390:

7 * ksub + codes[i + 7] as u32

Here ksub is usize, while codes[...] as u32 makes the RHS u32, so Rust cannot add them. The same issue repeats for the 8 gather indices.

Please keep the index arithmetic in one integer type before casting to the intrinsic index type, e.g. use codes[...] as usize and cast the final result to i32 (or otherwise explicitly convert both operands safely). After that, please run the full Rust validation again.

[leaves12138]

Thanks for the update. The previous compile blocker is fixed: cargo test --all --all-features now passes locally with 15 unit tests.

However, this PR still fails the formatter check that the CI workflow is going to enforce:

cargo fmt --all -- --check

It reports formatting diffs in core/src/blas.rs, core/src/distance.rs, core/src/kmeans.rs, and core/src/pq.rs.

Please run cargo fmt --all and push the formatted result. After that I can re-check and approve if nothing else shows up.

[leaves12138]

Thanks for the formatting update. The previous formatter blocker is fixed now:

cargo fmt --all -- --check passes.

cargo test --all --all-features also still passes locally with 15 tests.

There is still one CI blocker: the same clippy command used by PR #2 fails on this branch:

cargo clippy --all-targets --workspace -- -D warnings

Current failures include clippy::too_many_arguments in core/src/blas.rs, multiple clippy::needless_range_loop findings in distance.rs, kmeans.rs, and pq.rs, clippy::manual_clamp, clippy::identity_op / clippy::erasing_op in the AVX2 gather offsets, clippy::manual_is_multiple_of, and clippy::bad_bit_mask in the 4-bit PQ assertions.

Since the CI workflow enforces -D warnings, please either fix these lints or add narrow #[allow(...)] annotations where the current style is intentional/performance-sensitive. After clippy passes, I can re-check and approve.

[leaves12138]

Thanks for the update. I re-checked the latest head (4985aa0ac475800c1422f233b72b390a4874a5b2) after the clippy fixes.

Local validation now passes:

• cargo fmt --all -- --check
• cargo clippy --all-targets --workspace -- -D warnings
• cargo build --all-targets
• cargo test --all --all-features (15 passed)

The GitHub CI checks for this PR are also green (check and rust-test). No further blocker from my side.