Proposal: pluggable semantic index backend for larger vaults

[zc277584121]

Context

Semantic search currently stores float32 vectors in SQLite and scans them in Python in src/hyperresearch/core/embed.py. That is a good zero-configuration default and fits the existing design where Markdown is canonical and SQLite is a rebuildable cache.

Would the project welcome a small semantic-index boundary so larger vaults can opt into an external vector index without changing the default behavior?

Possible shape

• Keep the current SQLite BLOB plus brute-force cosine implementation as the default.
• Define a narrow backend contract for vector sync, upsert, delete, and search.
• Keep embedding provider selection, note metadata, FTS, and reciprocal-rank fusion in the existing project layers.
• Load external backends only through optional dependencies and explicit configuration.
• A first optional backend could be Milvus, using local Milvus Lite where supported or a configurable remote URI and token.

Questions

1. Does a pluggable semantic-index boundary fit the intended architecture?
2. Should external indexes remain derived and fully rebuildable from the Markdown vault and SQLite metadata?
3. If this direction is welcome, would you prefer the backend contract to live in core or in a separately installable integration package?

This issue is only to confirm the architecture and contribution route before any implementation.