feat(core): add semantic embedding text prefixes

Signed-off-by: Dmitry Golubev <lastguru@gmail.com>

Author: lastguru-net

docs/litellm-provider.md

@@ -48,6 +48,8 @@ All options can be set in config or as environment variables.
 | `semantic_embedding_forward_dimensions` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_FORWARD_DIMENSIONS` | Auto | Sends `dimensions` to LiteLLM only when supported. Auto is enabled for `text-embedding-3` model strings. |
 | `semantic_embedding_document_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE` | Auto | LiteLLM `input_type` for indexed notes/passages. |
 | `semantic_embedding_query_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE` | Auto | LiteLLM `input_type` for search queries. |
+| `semantic_embedding_document_prefix` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_PREFIX` | Unset | Literal text prefix prepended to indexed document chunks before embedding. |
+| `semantic_embedding_query_prefix` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_PREFIX` | Unset | Literal text prefix prepended to search queries before embedding. |
 | `semantic_embedding_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_BATCH_SIZE` | `2` | Number of text chunks per provider request. |
 | `semantic_embedding_request_concurrency` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_REQUEST_CONCURRENCY` | `4` | Maximum concurrent LiteLLM embedding requests. |
 | `semantic_embedding_sync_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_SYNC_BATCH_SIZE` | `2` | Number of prepared vector jobs flushed through the sync pipeline together. |
@@ -101,9 +103,17 @@ export BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE=passage
 export BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE=query
 ```
 
-Changing provider, model, dimensions, dimension-forwarding, or document/query
-roles changes the meaning of stored vectors. Rebuild embeddings after any of
-those changes:
+`input_type` is an API parameter. For models that require role text in the
+actual input string, configure literal prefixes instead or in addition:
+
+```bash
+export BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_PREFIX="title: none | text: "
+export BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_PREFIX="task: search result | query: "
+```
+
+Changing provider, model, dimensions, dimension-forwarding, document/query roles,
+or prefixes changes Basic Memory's stored vector identity. Rebuild embeddings
+after any of those changes:
 
 ```bash
 bm reindex --embeddings

docs/semantic-search.md

@@ -106,6 +106,8 @@ All settings are fields on `BasicMemoryConfig` and can be set via environment va
 | `semantic_embedding_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_BATCH_SIZE` | `2` | Number of texts to embed per batch. |
 | `semantic_embedding_document_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE` | Auto for known LiteLLM models | Optional LiteLLM `input_type` for indexed document/passages. |
 | `semantic_embedding_query_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE` | Auto for known LiteLLM models | Optional LiteLLM `input_type` for search queries. |
+| `semantic_embedding_document_prefix` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_PREFIX` | Unset | Optional literal text prefix prepended to indexed document chunks before embedding. |
+| `semantic_embedding_query_prefix` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_PREFIX` | Unset | Optional literal text prefix prepended to search queries before embedding. |
 | `semantic_vector_k` | `BASIC_MEMORY_SEMANTIC_VECTOR_K` | `100` | Candidate count for vector nearest-neighbour retrieval. Higher values improve recall at the cost of latency. |
 
 ## Embedding Providers
@@ -174,6 +176,18 @@ export BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE=passage
 export BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE=query
 ```
 
+Some asymmetric models require literal role text in the input string rather
+than, or in addition to, an API `input_type` parameter:
+
+```bash
+export BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_PREFIX="title: none | text: "
+export BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_PREFIX="task: search result | query: "
+```
+
+The document prefix is prepended to indexed chunks during sync/reindex. The
+query prefix is prepended to search text for vector and hybrid retrieval.
+Prefixes work with `fastembed`, `openai`, and `litellm` providers.
+
 #### Live LiteLLM Validation
 
 Provider APIs differ in subtle ways: some accept `dimensions`, some require separate
@@ -317,6 +331,7 @@ bm reindex -p my-project
 - **Model change**: After changing `semantic_embedding_model`
 - **Dimension change**: After changing `semantic_embedding_dimensions`
 - **LiteLLM role change**: After changing `semantic_embedding_document_input_type` or `semantic_embedding_query_input_type`
+- **Literal prefix change**: After changing `semantic_embedding_document_prefix` or `semantic_embedding_query_prefix`
 
 The reindex command shows progress with embedded/skipped/error counts:
 

src/basic_memory/config.py

@@ -293,6 +293,20 @@ def __init__(self, **data: Any) -> None: ...
             "Use with asymmetric embedding models such as Cohere or NVIDIA retrieval models."
         ),
     )
+    semantic_embedding_document_prefix: str | None = Field(
+        default=None,
+        description=(
+            "Optional literal text prefix prepended to indexed document chunks before "
+            "embedding. Use with prefix-sensitive asymmetric embedding models."
+        ),
+    )
+    semantic_embedding_query_prefix: str | None = Field(
+        default=None,
+        description=(
+            "Optional literal text prefix prepended to search queries before embedding. "
+            "Use with prefix-sensitive asymmetric embedding models."
+        ),
+    )
     semantic_embedding_sync_batch_size: int = Field(
         default=2,
         description="Batch size for vector sync orchestration flushes.",

src/basic_memory/mcp/server.py

@@ -81,7 +81,9 @@ async def lifespan(app: FastMCP):
                 f"Semantic search: provider={config.semantic_embedding_provider}, "
                 f"model={config.semantic_embedding_model}, "
                 f"dimensions={config.semantic_embedding_dimensions or 'auto'}, "
-                f"batch_size={config.semantic_embedding_batch_size}"
+                f"batch_size={config.semantic_embedding_batch_size}, "
+                f"document_prefix_set={bool(config.semantic_embedding_document_prefix)}, "
+                f"query_prefix_set={bool(config.semantic_embedding_query_prefix)}"
             )
 
         # Log configured projects with their routing mode

src/basic_memory/repository/embedding_provider_factory.py

@@ -7,12 +7,16 @@
 
 from basic_memory.config import BasicMemoryConfig, default_fastembed_cache_dir
 from basic_memory.repository.embedding_provider import EmbeddingProvider
+from basic_memory.repository.prefixing_provider import (
+    PrefixingEmbeddingProvider,
+    normalize_embedding_prefix,
+)
 
 # Cache key fields are limited to values that change the *identity* of the loaded
-# model (provider, model_name, dimensions, LiteLLM role/input-type/forward-dimension
-# settings, batch/request knobs that affect the LiteLLM identity, and the resolved
-# cache dir). Thread/parallel knobs are deliberately excluded — they change ONNX
-# *execution* only, not the loaded weights. Including them caused #872: in a
+# provider instance (provider, model_name, dimensions, semantic role/input-type/prefix
+# settings, batch/request knobs, and the resolved cache dir). Thread/parallel knobs
+# are deliberately excluded - they change ONNX *execution* only, not the loaded
+# weights. Including them caused #872: in a
 # container/cgroup the CPU-derived thread count can drift between calls, producing
 # a fresh cache key and reloading the ~2.3GB model into a CPU arena that never
 # returns memory to the OS.
@@ -25,6 +29,8 @@
     int,
     str | None,
     str | None,
+    str | None,
+    str | None,
     str,
 ]
 
@@ -105,6 +111,8 @@ def _provider_cache_key(app_config: BasicMemoryConfig) -> ProviderCacheKey:
         app_config.semantic_embedding_request_concurrency,
         app_config.semantic_embedding_document_input_type,
         app_config.semantic_embedding_query_input_type,
+        normalize_embedding_prefix(app_config.semantic_embedding_document_prefix),
+        normalize_embedding_prefix(app_config.semantic_embedding_query_prefix),
         _resolve_cache_dir(app_config),
     )
 
@@ -204,6 +212,15 @@ def create_embedding_provider(app_config: BasicMemoryConfig) -> EmbeddingProvide
     else:
         raise ValueError(f"Unsupported semantic embedding provider: {provider_name}")
 
+    document_prefix = normalize_embedding_prefix(app_config.semantic_embedding_document_prefix)
+    query_prefix = normalize_embedding_prefix(app_config.semantic_embedding_query_prefix)
+    if document_prefix is not None or query_prefix is not None:
+        provider = PrefixingEmbeddingProvider(
+            provider,
+            document_prefix=document_prefix,
+            query_prefix=query_prefix,
+        )
+
     with _EMBEDDING_PROVIDER_CACHE_LOCK:
         if cached_provider := _EMBEDDING_PROVIDER_CACHE.get(cache_key):
             return cached_provider

src/basic_memory/repository/prefixing_provider.py

@@ -0,0 +1,78 @@
+"""Embedding provider wrapper for role-specific literal text prefixes."""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+from basic_memory.repository.embedding_provider import EmbeddingProvider
+
+
+def normalize_embedding_prefix(value: str | None) -> str | None:
+    """Treat unset and empty prefixes as disabled while preserving meaningful spaces."""
+    if value == "":
+        return None
+    return value
+
+
+class PrefixingEmbeddingProvider(EmbeddingProvider):
+    """Apply document/query text prefixes before delegating to an embedding provider."""
+
+    def __init__(
+        self,
+        provider: EmbeddingProvider,
+        *,
+        document_prefix: str | None = None,
+        query_prefix: str | None = None,
+    ) -> None:
+        self.provider = provider
+        self.document_prefix = normalize_embedding_prefix(document_prefix)
+        self.query_prefix = normalize_embedding_prefix(query_prefix)
+
+    @property
+    def model_name(self) -> str:
+        return self.provider.model_name
+
+    @property
+    def dimensions(self) -> int:
+        return self.provider.dimensions
+
+    async def embed_query(self, text: str) -> list[float]:
+        if self.query_prefix is not None:
+            text = f"{self.query_prefix}{text}"
+        return await self.provider.embed_query(text)
+
+    async def embed_documents(self, texts: list[str]) -> list[list[float]]:
+        if self.document_prefix is not None:
+            texts = [f"{self.document_prefix}{text}" for text in texts]
+        return await self.provider.embed_documents(texts)
+
+    def runtime_log_attrs(self) -> dict[str, Any]:
+        attrs = self.provider.runtime_log_attrs()
+        attrs.update(
+            {
+                "document_prefix_set": self.document_prefix is not None,
+                "query_prefix_set": self.query_prefix is not None,
+            }
+        )
+        if self.document_prefix is not None:
+            attrs["document_prefix_length"] = len(self.document_prefix)
+        if self.query_prefix is not None:
+            attrs["query_prefix_length"] = len(self.query_prefix)
+        return attrs
+
+    def identity_key(self) -> str:
+        """Return embedding semantics including literal text-prefix transforms."""
+        provider_identity_key = getattr(self.provider, "identity_key", None)
+        if callable(provider_identity_key):
+            provider_identity = provider_identity_key()
+        else:
+            provider_identity = f"{self.provider.model_name}:{self.provider.dimensions}"
+
+        document_prefix = self.document_prefix or "-"
+        query_prefix = self.query_prefix or "-"
+        return (
+            f"{type(self.provider).__name__}:{provider_identity}:"
+            f"document_prefix={json.dumps(document_prefix, ensure_ascii=True)}:"
+            f"query_prefix={json.dumps(query_prefix, ensure_ascii=True)}"
+        )

src/basic_memory/repository/search_repository_base.py

@@ -641,15 +641,16 @@ def _embedding_model_key(self) -> str:
         assert self._embedding_provider is not None
         provider = self._embedding_provider
 
-        provider_identity = f"{provider.model_name}:{provider.dimensions}"
-        from basic_memory.repository.litellm_provider import LiteLLMEmbeddingProvider
-
-        if isinstance(provider, LiteLLMEmbeddingProvider):
-            # Trigger: LiteLLM can change request semantics without changing model/dimensions.
-            # Why: asymmetric providers use role-specific document/query params, and
-            # dimension forwarding changes provider-side output-size behavior.
+        provider_identity_key = getattr(provider, "identity_key", None)
+        if callable(provider_identity_key):
+            # Trigger: providers can change request/input semantics without changing
+            # model/dimensions.
+            # Why: asymmetric providers may use role-specific API params or literal
+            # text-prefix transforms that change stored vector meaning.
             # Outcome: reindex treats those semantic config changes as stale vectors.
-            provider_identity = provider.identity_key()
+            provider_identity = provider_identity_key()
+        else:
+            provider_identity = f"{provider.model_name}:{provider.dimensions}"
 
         return f"{type(provider).__name__}:{provider_identity}"
 

src/basic_memory/schemas/project_info.py

@@ -87,6 +87,8 @@ class EmbeddingStatus(BaseModel):
     embedding_provider: Optional[str] = None
     embedding_model: Optional[str] = None
     embedding_dimensions: Optional[int] = None
+    embedding_document_prefix_set: bool = False
+    embedding_query_prefix_set: bool = False
 
     # Counts
     total_indexed_entities: int = 0

src/basic_memory/services/project_service.py

@@ -989,6 +989,8 @@ async def get_embedding_status(self, project_id: int) -> EmbeddingStatus:
         provider = config.semantic_embedding_provider
         model = config.semantic_embedding_model
         dimensions = config.semantic_embedding_dimensions
+        document_prefix_set = bool(config.semantic_embedding_document_prefix)
+        query_prefix_set = bool(config.semantic_embedding_query_prefix)
 
         is_postgres = config.database_backend == DatabaseBackend.POSTGRES
 
@@ -1025,6 +1027,8 @@ async def get_embedding_status(self, project_id: int) -> EmbeddingStatus:
                 embedding_provider=provider,
                 embedding_model=model,
                 embedding_dimensions=dimensions,
+                embedding_document_prefix_set=document_prefix_set,
+                embedding_query_prefix_set=query_prefix_set,
                 total_indexed_entities=total_indexed_entities,
                 vector_tables_exist=False,
                 reindex_recommended=True,
@@ -1132,6 +1136,8 @@ async def _vec_scalar(vec_sql) -> int:
                 embedding_provider=provider,
                 embedding_model=model,
                 embedding_dimensions=dimensions,
+                embedding_document_prefix_set=document_prefix_set,
+                embedding_query_prefix_set=query_prefix_set,
                 total_indexed_entities=total_indexed_entities,
                 vector_tables_exist=False,
                 reindex_recommended=True,
@@ -1164,6 +1170,8 @@ async def _vec_scalar(vec_sql) -> int:
             embedding_provider=provider,
             embedding_model=model,
             embedding_dimensions=dimensions,
+            embedding_document_prefix_set=document_prefix_set,
+            embedding_query_prefix_set=query_prefix_set,
             total_indexed_entities=total_indexed_entities,
             total_entities_with_chunks=total_entities_with_chunks,
             total_chunks=total_chunks,

tests/repository/test_openai_provider.py

@@ -15,6 +15,7 @@
 )
 from basic_memory.repository.fastembed_provider import FastEmbedEmbeddingProvider
 from basic_memory.repository.openai_provider import OpenAIEmbeddingProvider
+from basic_memory.repository.prefixing_provider import PrefixingEmbeddingProvider
 from basic_memory.repository.semantic_errors import SemanticDependenciesMissingError
 
 
@@ -547,6 +548,69 @@ def test_embedding_provider_factory_creates_new_provider_for_different_cache_key
     assert provider_a is not provider_b
 
 
+def test_embedding_provider_factory_wraps_provider_when_prefixes_are_configured():
+    """Factory should apply literal prefixes independently of provider backend."""
+    config = BasicMemoryConfig(
+        env="test",
+        projects={"test-project": "/tmp/basic-memory-test"},
+        default_project="test-project",
+        semantic_search_enabled=True,
+        semantic_embedding_provider="fastembed",
+        semantic_embedding_document_prefix="title: none | text: ",
+        semantic_embedding_query_prefix="task: search result | query: ",
+    )
+
+    provider = create_embedding_provider(config)
+
+    assert isinstance(provider, PrefixingEmbeddingProvider)
+    assert isinstance(provider.provider, FastEmbedEmbeddingProvider)
+    assert provider.document_prefix == "title: none | text: "
+    assert provider.query_prefix == "task: search result | query: "
+
+
+def test_embedding_provider_factory_reuses_provider_for_same_prefixes():
+    """Prefix fields participate in the process-local provider cache key."""
+    config = BasicMemoryConfig(
+        env="test",
+        projects={"test-project": "/tmp/basic-memory-test"},
+        default_project="test-project",
+        semantic_search_enabled=True,
+        semantic_embedding_provider="fastembed",
+        semantic_embedding_document_prefix="doc: ",
+        semantic_embedding_query_prefix="query: ",
+    )
+
+    provider_a = create_embedding_provider(config)
+    provider_b = create_embedding_provider(config)
+
+    assert provider_a is provider_b
+
+
+def test_embedding_provider_factory_separates_cache_for_different_prefixes():
+    """Changing literal prefixes should not reuse a stale cached provider."""
+    shared_config = {
+        "env": "test",
+        "projects": {"test-project": "/tmp/basic-memory-test"},
+        "default_project": "test-project",
+        "semantic_search_enabled": True,
+        "semantic_embedding_provider": "fastembed",
+        "semantic_embedding_query_prefix": "query: ",
+    }
+    first_config = BasicMemoryConfig(
+        **shared_config,
+        semantic_embedding_document_prefix="doc: ",
+    )
+    second_config = BasicMemoryConfig(
+        **shared_config,
+        semantic_embedding_document_prefix="document: ",
+    )
+
+    first_provider = create_embedding_provider(first_config)
+    second_provider = create_embedding_provider(second_config)
+
+    assert first_provider is not second_provider
+
+
 def test_embedding_provider_factory_reuses_provider_when_only_thread_knobs_differ():
     """Thread/parallel knobs tune ONNX execution, not model identity (#872).