[ENH] Add eventual consistency to the frontend

[HammadB]

Description of changes

Summarize the changes made by this PR.

• Improvements & Bug fixes
  • ...
• New functionality
  • Adds read level enum to the frontend APIs and propagates it through to query nodes
  • Note js client tests are failing as they require committing generated files, this will occur in the top of the stack where client changes are addressed.

Test plan

Will be handled by e2e tests at top of stack

• Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Migration plan

IndexAndWal will be the default everywhere

Observability plan

What is the plan to instrument and monitor this change?

Documentation Changes

Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs section?

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[HammadB]

• [ENH] Eventual consistency in clients #5947
• [ENH] Add eventual consistency to the frontend #5946 👈 (View in Graphite)
• [ENH] Eventual consistency in query nodes #5945
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[propel-code-bot]

Add Eventual Consistency Read Levels to Frontend and Clients

Introduces a read_level parameter across Python, JavaScript, and Rust clients and the Rust backend, allowing users to choose between strong consistency (INDEX_AND_WAL, default) and eventual consistency (INDEX_ONLY). When INDEX_ONLY is selected, search queries skip reading from the Write-Ahead Log (WAL) and only query compacted segments, potentially improving performance at the cost of missing recent uncommitted writes.

Key Changes

• Defined ReadLevel enum (INDEX_AND_WAL, INDEX_ONLY) in Python, JavaScript, and Rust API types.
• Updated Collection.search methods in all client libraries to accept the read_level parameter.
• Modified SearchRequest and SearchPayload structs in Rust to carry the read_level.
• Updated ServiceBasedFrontend and WorkerServer to propagate read_level to the query execution orchestration logic.
• Added integration tests in Python, JavaScript, and Rust to verify the new parameter is correctly passed and handled.

Affected Areas

• chromadb/api/types.py
• chromadb/api/models/Collection.py
• chromadb/api/async_api.py
• chromadb/api/fastapi.py
• clients/new-js/packages/chromadb/src/collection.ts
• clients/new-js/packages/chromadb/src/types.ts
• rust/frontend/src/server.rs
• rust/worker/src/server.rs
• rust/types/src/execution/plan.rs
• rust/chroma/src/collection.rs

This summary was automatically generated by @propel-code-bot

[sanketkedia]

thoughts on having this per SearchPayload v/s one level for the entire request?

[HammadB]

I did consider this. I kind of prefer it to be homogenous. Reason is that if you are making decisions off this data, you want a consistent read across all the queries.

[sanketkedia]

Some tests are failing, lgtm with one question

[HammadB]

• Note js client tests are failing as they require committing generated files, this will occur in the top of the stack where client changes are addressed.

@sanketkedia Please see the comment in the PR about the test failure - `

Note js client tests are failing as they require committing generated files, this will occur in the top of the stack where client changes are addressed.

[propel-code-bot]

[Performance] Using tracing::info! here will generate a log line for every search query executed with IndexOnly. In a high-throughput scenario (which is the primary use case for IndexOnly), this will flood the logs.

Consider using tracing::debug! instead.

Context for Agents

Using `tracing::info!` here will generate a log line for every search query executed with `IndexOnly`. In a high-throughput scenario (which is the primary use case for `IndexOnly`), this will flood the logs.

Consider using `tracing::debug!` instead.

File: rust/worker/src/execution/orchestration/knn_filter.rs
Line: 302

[propel-code-bot]

[Logic] The assertion len(results["ids"]) == 1 contradicts the comment below it: "Results may be empty if data hasn't been compacted yet".

Since INDEX_ONLY skips the WAL and you just performed an add() (which writes to the WAL), the data might not be in the compacted segments yet, leading to 0 results. Asserting exactly 1 result makes this test flaky or incorrect for eventual consistency testing.

Consider asserting len(results["ids"]) >= 0 or ensuring compaction runs before this assertion if you strictly require the data to be visible.

Context for Agents

The assertion `len(results["ids"]) == 1` contradicts the comment below it: "Results may be empty if data hasn't been compacted yet". 

Since `INDEX_ONLY` skips the WAL and you just performed an `add()` (which writes to the WAL), the data might not be in the compacted segments yet, leading to 0 results. Asserting exactly 1 result makes this test flaky or incorrect for eventual consistency testing.

Consider asserting `len(results["ids"]) >= 0` or ensuring compaction runs before this assertion if you strictly require the data to be visible.

File: chromadb/test/api/test_search_api.py
Line: 76