feat(milvus): add Milvus/Zilliz Cloud vector store backend#1273
Open
supmo668 wants to merge 7 commits intogetzep:mainfrom
Open
feat(milvus): add Milvus/Zilliz Cloud vector store backend#1273supmo668 wants to merge 7 commits intogetzep:mainfrom
supmo668 wants to merge 7 commits intogetzep:mainfrom
Conversation
Add a provider-agnostic base class for vector store backends with 10 domain-aware methods (save/delete for entity nodes, entity edges, episodic nodes, and community nodes plus bulk clear operations). - graphiti_core/vector_store/client.py: VectorStoreClient base class with non-abstract methods that raise NotImplementedError for backward compatibility - graphiti_core/vector_store/__init__.py: export only base classes (no optional provider imports per CONTRIBUTING.md) - graphiti_core/driver/driver.py: add vector_store attribute (Any type to avoid circular imports) - pyproject.toml: add milvus optional extra and pytest-asyncio dev dep Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Insert best-effort vector store sync points after graph DB writes so that a plugged-in VectorStoreClient stays in sync without requiring changes to the graph database layer. Hooks added in: - nodes.py: save/delete for entity, episodic, and community nodes - edges.py: save/delete for entity edges - bulk_utils.py: bulk entity node/edge save after batch writes - community_operations.py: delete community nodes on removal - graph_data_operations.py: clear_all on full data wipe - edge_operations.py / node_operations.py: formatting cleanup All hooks are guarded by `if driver.vector_store is not None` and wrapped in try/except to keep vector store failures non-fatal. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Complete the provider-agnostic integration layer: - graphiti_core/graphiti.py: accept vector_store param, attach to driver, call ensure_ready() on build_indices, close on shutdown - mcp_server: VectorStoreAppConfig schema, VectorStoreFactory skeleton (no provider implementations yet), server wiring for vector store lifecycle - tests/test_dual_write.py: 23 unit tests covering all dual-write hooks (save, delete, bulk, error resilience) - tests/test_graphiti_vector_store.py: 7 unit tests for constructor integration, lifecycle (close, build_indices), and generic vector store behavior Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement MilvusVectorStoreClient with 4 collections (entity_nodes, entity_edges, episodic_nodes, community_nodes), hybrid search via HNSW dense vectors + BM25 sparse vectors, and group_id partition keys for multi-tenancy. - milvus_client.py: MilvusVectorStoreClient extending VectorStoreClient with all 10 domain-aware methods, lazy pymilvus import, COSINE HNSW - milvus_utils.py: collection schemas, serialization helpers, constants (INT64 epoch ms for datetimes, 0 = null sentinel) - milvus_search_interface.py: SearchInterface impl for hybrid retrieval - milvus_graph_operations.py: GraphOperationsInterface impl for CRUD Requires pymilvus>=2.5.3 (installed via graphiti-core[milvus] extra). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- graphiti.py: auto-attach MilvusSearchInterface when a MilvusVectorStoreClient is passed (lazy import, try/except guard) - vector_store_sync.py: backfill utility to sync existing graph DB data into Milvus collections for brownfield deployments - search.py: add query text to debug log for observability Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- schema.py: MilvusProviderConfig, VectorStoreProvidersConfig - factories.py: VectorStoreFactory milvus case with env var overrides - graphiti_mcp_server.py: MILVUS_URI auto-detect, MilvusSearchInterface auto-attach after Graphiti client creation - Dockerfile.standalone: INSTALL_MILVUS build arg for optional pymilvus - Docker Compose stacks for Neo4j+Milvus and FalkorDB+Milvus - YAML configs for both deployment variants Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_milvus_vector_store_client.py: 46 tests for MilvusVectorStoreClient (save, delete, ensure_ready, error handling, serialization) - test_milvus_utils.py: schema and utility function tests - test_milvus_search_interface.py: SearchInterface contract tests - test_milvus_graph_operations.py: GraphOperationsInterface tests - test_milvus_search_int.py: 10 integration tests against Zilliz Cloud (hybrid search, BM25, date filtering, edge cases) - test_vector_store_sync.py: backfill utility tests - test_graphiti_vector_store.py: Milvus auto-attach and override tests 153 unit tests pass, 10 integration tests pass against Zilliz Cloud. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This was referenced Feb 26, 2026
Contributor
Author
|
All CI checks pass (ruff, pyright, unit-tests, database-integration-tests, CodeQL). Rebased on latest This is Part 2 of the vector store integration (RFC #1263), building on the plugin interface in #1264. The Milvus implementation is fully self-contained — 153 unit tests + 10 integration tests against Zilliz Cloud all pass. Happy to walk through any of the implementation choices or address feedback. @danielchalef @prasmussen15 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Milvus/Zilliz Cloud implementation for the pluggable VectorStoreClient interface introduced in #1264. Provides hybrid vector + BM25 search as an acceleration layer alongside any graph database (Neo4j or FalkorDB).
Depends on: #1264 (Part 1 — provider-agnostic plugin interface)
RFC: #1263
Type of Change
Objective
Enable Graphiti users to leverage Milvus/Zilliz Cloud for high-performance hybrid retrieval (dense vector + BM25 sparse) while keeping their existing graph database for entity resolution, episode storage, and graph traversal. This is an overlay, not a replacement — the graph DB remains the source of truth.
Architecture
flowchart LR subgraph "Write Path" A[Graphiti Core] -->|primary| B[Graph DB<br/>Neo4j / FalkorDB] A -->|dual-write| C[Milvus / Zilliz] end subgraph "Read Path" D[Search Query] --> E{SearchInterface} E -->|hybrid search| C E -->|graph traversal| B endKey Features
group_idas partition keyMilvusSearchInterfaceauto-wired whenMilvusVectorStoreClientis passedNew Files
vector_store/milvus_client.pyMilvusVectorStoreClient— 10 domain-aware methodsvector_store/milvus_utils.pyvector_store/milvus_search_interface.pySearchInterfaceimpl for hybrid retrievalvector_store/milvus_graph_operations.pyGraphOperationsInterfaceimpl for CRUDutils/vector_store_sync.pyTesting
test_milvus_vector_store_client.pytest_milvus_utils.pytest_milvus_search_interface.pytest_milvus_graph_operations.pytest_milvus_search_int.pytest_vector_store_sync.pytest_graphiti_vector_store.pyBreaking Changes
No breaking changes. Milvus is an optional extra (
pip install "graphiti-core[milvus]"). All existing behavior is unchanged.Checklist
make lintpasses — 0 errors)pyrightpasses (0 errors, 0 warnings)Related Issues
Part 2 of RFC #1263
Depends on #1264 (Part 1)