Skip to content

Commit 1e65197

Browse files
Maverick0351aMaverick0351a
committed
feat(namespaces): add multi-tenant namespace isolation, privacy & multitenancy docs, tests, error examples, docs integrity test
1 parent 19be4ff commit 1e65197

7 files changed

Lines changed: 405 additions & 65 deletions

File tree

‎MULTITENANCY.md‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Multi-Tenancy & Namespace Isolation
2+
3+
NeuralCache provides lightweight logical isolation between tenants ("namespaces") so
4+
feedback signals (narrative updates, pheromone exposure, and future adaptive state)
5+
do not bleed across customers or applications sharing a single API deployment.
6+
7+
## Overview
8+
9+
Each request may specify a namespace via the HTTP header:
10+
11+
```
12+
X-NeuralCache-Namespace: <name>
13+
```
14+
15+
If omitted, the `default` namespace is used. A per-namespace `Reranker` instance
16+
is created on first use and retained in-memory for subsequent requests.
17+
18+
| Aspect | Isolated Per Namespace | Notes |
19+
|---------------------|------------------------|-------|
20+
| Narrative state | ✅ | EMA vector & gating success accrual |
21+
| Pheromone tracking | ✅ | Exposure/decay history separate |
22+
| Feedback updates | ✅ | Selected doc reinforcement scoped |
23+
| CR Index (future) | ⏳ Planned | Currently shared; future path: per-namespace index selection |
24+
| Metrics aggregation | 🚧 Partial | Global metrics currently aggregate all namespaces |
25+
26+
## Namespace Semantics
27+
28+
Namespaces are labels (up to 64 chars) restricted by regex:
29+
30+
```
31+
^[a-zA-Z0-9_.-]{1,64}$
32+
```
33+
34+
Invalid names result in `400 BAD_REQUEST` with error code `BAD_REQUEST` and message
35+
`Invalid namespace`.
36+
37+
## Lifecycle & Memory Considerations
38+
39+
A namespace is instantiated lazily and kept indefinitely. There is no automatic
40+
LRU eviction today. Deployments with highly cardinal tenant identifiers should
41+
introduce an external lifecycle controller or run separate service instances.
42+
43+
### Potential Memory Growth
44+
45+
Each namespace maintains:
46+
- Narrative EMA vector (float32 length = `narrative_dim`)
47+
- Pheromone map (per document ID touched in that namespace)
48+
- Feedback cache references (shared global LRU mapping doc IDs to last rerank batch)
49+
50+
If you expect thousands of ephemeral namespaces, consider:
51+
- Fronting the service with a routing layer that shards namespaces across pods.
52+
- Periodically restarting pods (stateless if persistence disabled) and relying on retention.
53+
- Contributing an eviction strategy (e.g., size + inactivity timeout) upstream.
54+
55+
## Persistence & Retention
56+
57+
Retention sweeping currently scans all instantiated namespace rerankers and applies
58+
narrative/pheromone purging according to `NEURALCACHE_STORAGE_RETENTION_DAYS`.
59+
60+
Persistence paths (`narrative_store_path`, `pheromone_store_path`) remain *shared*.
61+
If persistence is enabled, serialization merges namespace state is NOT yet implemented.
62+
For multi-tenant deployments needing persisted isolation, run separate processes
63+
or disable persistence until namespaced persistence lands.
64+
65+
## Security & Isolation Guarantees
66+
67+
Logical isolation only—data is resident in the same process address space:
68+
- No cross-namespace feedback influence (narrative & pheromones).
69+
- No access control boundary: API tokens/IP ACLs still required if separation of customers matters.
70+
- A memory disclosure vulnerability could still reveal multi-tenant state; use distinct deployments for strong isolation.
71+
72+
## Roadmap
73+
74+
| Feature | Status | Planned Improvement |
75+
|---------|--------|--------------------|
76+
| Namespaced reranker registry | ✅ | Add LRU eviction / idle culling |
77+
| Namespaced persistence | ❌ | Persist per-namespace narrative & pheromones |
78+
| Namespaced metrics | 🚧 | Per-namespace metrics labels / filtering |
79+
| Namespaced CR index | ❌ | Optional per-namespace index load & selection |
80+
| Hard quotas (max namespaces) | ❌ | Enforce upper bound via config |
81+
82+
## Configuration Summary
83+
84+
| Setting | Description | Default |
85+
|---------|-------------|---------|
86+
| `NEURALCACHE_NAMESPACE_HEADER` | Header key to select namespace | `X-NeuralCache-Namespace` |
87+
| `NEURALCACHE_DEFAULT_NAMESPACE` | Namespace when header omitted | `default` |
88+
| `NEURALCACHE_NAMESPACE_PATTERN` | Regex for validation | `^[a-zA-Z0-9_.-]{1,64}$` |
89+
90+
## Examples
91+
92+
### Basic Request
93+
94+
```
95+
POST /rerank
96+
X-NeuralCache-Namespace: tenantA
97+
{
98+
"query": "improve latency",
99+
"documents": [{"id":"d1","text":"..."}]
100+
}
101+
```
102+
103+
### Error: Invalid Namespace
104+
```
105+
X-NeuralCache-Namespace: bad/tenant
106+
-> 400 {"error":{"code":"BAD_REQUEST","message":"Invalid namespace"}}
107+
```
108+
109+
## When to Use Separate Deployments Instead
110+
- Regulatory or contractual data isolation requirements.
111+
- Strong security boundaries (e.g., handling confidential embeddings per customer).
112+
- Extremely large or unpredictable namespace cardinality.
113+
114+
## Contributing
115+
Have a need for eviction strategies, metrics partitioning, or persisted isolation? Open an issue or PR with design context.

‎PRIVACY.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Privacy & Data Handling
2+
3+
This document describes how NeuralCache processes, stores, and retains data so
4+
operators can make informed decisions about deployment, compliance, and risk.
5+
6+
## Data Classes
7+
8+
| Data Type | Source | Stored? | Purpose | Notes |
9+
|-----------|--------|--------|---------|-------|
10+
| Query text | Client requests | No (transient) | Embedding & scoring | Only used in-memory; not persisted |
11+
| Document text | Client payloads | No (transient) | Scoring / feature extraction | Not written to disk unless embedding backend logs externally |
12+
| Embeddings | Generated (hash or model) | Narrative vector + pheromone features derived | Adaptive scoring | Embedding arrays reduced into aggregate state (EMA narrative) |
13+
| Feedback signals | /feedback endpoint | Narrative + pheromone updates | Improve personalization & success weighting | Stored indirectly in adaptive state, not raw payload |
14+
| API tokens | Environment variable | No | AuthZ | Not logged by default |
15+
| Metrics counters | In-process | Optional (Prometheus scrape) | Observability | No PII; aggregated counts & durations |
16+
17+
## What Is Persisted
18+
19+
If `storage_persistence_enabled = true` (default) the following artifacts may be
20+
written inside `storage_dir`:
21+
22+
| File | Contents | Privacy Considerations |
23+
|------|----------|------------------------|
24+
| `neuralcache.db` | SQLite metadata (if future features persist more) | Currently minimal; avoid storing PII here |
25+
| `narrative.json` | Narrative EMA vector(s) (currently global, roadmap: namespaced) | Numerical floats only; cannot reconstruct raw text |
26+
| `pheromones.json` | Document exposure/decay metadata (doc IDs & timestamps) | Ensure doc IDs are non-sensitive (hash if needed) |
27+
28+
Embeddings for documents are NOT persisted. Only aggregate statistical summaries
29+
and per-doc exposure timestamps are stored.
30+
31+
## Namespaces & Isolation
32+
33+
Namespaces (see `MULTITENANCY.md`) isolate adaptive state so feedback in one
34+
namespace does not influence another. Namespaces do not constitute a security
35+
boundary; run separate deployments for strict data isolation.
36+
37+
## Retention Controls
38+
39+
- `NEURALCACHE_STORAGE_RETENTION_DAYS`: When set (>0), periodic sweeping removes
40+
pheromone entries and purges stale narrative participation.
41+
- Setting it to `None` disables retention sweeping.
42+
- Set `storage_retention_sweep_interval_s` > 0 to enable continuous purging.
43+
44+
## Data Deletion Procedure (Operator Playbook)
45+
46+
1. Identify target namespace(s) requiring deletion.
47+
2. If persistence is enabled, stop the service to prevent new writes.
48+
3. Remove or archive `narrative.json` and `pheromones.json`.
49+
4. (If future namespaced persistence) remove only the namespace block / file.
50+
5. Restart the service; new state will be rebuilt lazily.
51+
52+
For immediate in-memory purge without restart (future feature), an admin endpoint
53+
could invalidate a namespace reranker—open an issue if needed.
54+
55+
## Minimizing Sensitive Exposure
56+
57+
- Use hashing or irreversible ID surrogates for document identifiers if they encode user info.
58+
- Pre-redact or classify text prior to sending; large language model hallucination
59+
risk is not increased since raw text isn't persisted.
60+
- Avoid embedding direct PII; upstream embedding providers may log inputs.
61+
- Consider pairwise differential privacy or noise injection if narrative vectors
62+
might leak per-user contribution (threat model dependent).
63+
64+
## Embedding Backends
65+
66+
| Backend | Privacy Characteristics | Guidance |
67+
|---------|------------------------|----------|
68+
| `hash` | Fully local, deterministic hashing | Safe for local dev; no external calls |
69+
| OpenAI / remote | Leaves host boundary | Ensure DPA with vendor; do not send prohibited data |
70+
| Sentence-transformers (local) | Local inference | Keep model updated for security patches |
71+
72+
## Logging & Observability
73+
74+
By default, application logs should not emit document text or queries. If you
75+
add custom logging, avoid raw payload dumps. Metrics endpoints expose only counts,
76+
latencies, and success/error labels.
77+
78+
## Threat Model Notes
79+
80+
Out-of-scope for current version:
81+
- Hard multi-tenant memory isolation (single process design)
82+
- Cryptographic deletion guarantees
83+
- Differential privacy enforcement
84+
85+
## Roadmap
86+
87+
| Area | Planned | Priority |
88+
|------|---------|----------|
89+
| Namespaced persistence | Yes | High |
90+
| Namespace eviction / purge endpoint | Yes | Medium |
91+
| Per-namespace metrics labeling | Yes | Medium |
92+
| Configurable doc ID hashing strategy | Investigate | Medium |
93+
| Differential privacy narrative updates | Investigate | Low |
94+
95+
## Operator Checklist Before Production
96+
97+
- [ ] Decide namespace strategy & expected cardinality
98+
- [ ] Set API tokens (`NEURALCACHE_API_TOKENS`)
99+
- [ ] Enable TLS termination at ingress / proxy
100+
- [ ] Review embedding provider data policies
101+
- [ ] Configure retention if required (`NEURALCACHE_STORAGE_RETENTION_DAYS`)
102+
- [ ] Validate no PII in document IDs
103+
- [ ] Monitor memory growth vs namespace count
104+
105+
## Questions / Contributions
106+
Open an issue for clarifications, or propose enhancements to improve privacy guarantees.

‎README.md‎

Lines changed: 38 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -115,66 +115,57 @@ Gating plugs in before narrative, pheromone, and MMR scoring—so downstream mem
115115

116116
---
117117

118-
## Integrations & interfaces
118+
## Multi-tenancy & namespaces
119119

120-
- **REST API** (`uvicorn neuralcache.api.server:app`) with `/rerank`, `/feedback`, `/metrics`, and `/healthz` endpoints.
121-
- **Plus API** (`uvicorn neuralcache.api.server_plus:app`) adds `/rerank/batch`, Prometheus `/metrics`, and mounts the legacy routes under `/v1`.
122-
- All responses include `X-NeuralCache-API-Version` (and temporary alias `X-API-Version`) so clients can log and assert expected contract versions. See `docs/VERSIONING.md` for the policy.
123-
- **CLI** (`neuralcache "<query>" docs.jsonl --top-k 5`) for quick experiments and scripting.
124-
- **LangChain adapter** (`pip install "neuralcache[adapters]"`): `from neuralcache.adapters import NeuralCacheLangChainReranker`
125-
- **LlamaIndex adapter** (`pip install "neuralcache[adapters]"`): `from neuralcache.adapters import NeuralCacheLlamaIndexReranker`
120+
NeuralCache now supports lightweight logical isolation using a namespace header:
126121

127-
See [`examples/quickstart.py`](examples/quickstart.py) for an end-to-end script.
128-
129-
---
122+
```
123+
X-NeuralCache-Namespace: tenantA
124+
```
130125

131-
## Feedback API: closing the loop
126+
If omitted, the `default` namespace is used. Narrative + pheromone feedback effects do not bleed across namespaces. See `MULTITENANCY.md` for details, limitations (no eviction yet, shared persistence), and roadmap.
132127

133-
Send successful reranks back to NeuralCache so the narrative EMA and pheromone
134-
signals keep learning:
128+
| Setting | Purpose | Default |
129+
|---------|---------|---------|
130+
| `NEURALCACHE_NAMESPACE_HEADER` | Header key to read namespace | `X-NeuralCache-Namespace` |
131+
| `NEURALCACHE_DEFAULT_NAMESPACE` | Fallback namespace when header missing | `default` |
132+
| `NEURALCACHE_NAMESPACE_PATTERN` | Validation regex (400 on mismatch) | `^[a-zA-Z0-9_.-]{1,64}$` |
135133

136-
```http
137-
POST /feedback
138-
Content-Type: application/json
134+
Invalid namespaces return a standardized error envelope:
139135

136+
```json
140137
{
141-
"query": "How do I rotate API keys?",
142-
"selected_ids": ["doc-42", "doc-71"],
143-
"success": 0.9,
144-
"best_doc_text": "Rotate keys via the dashboard > API tokens",
145-
"best_doc_embedding": [0.01, 0.32, -0.55, ...]
138+
"error": {
139+
"code": "BAD_REQUEST",
140+
"message": "Invalid namespace",
141+
"detail": null
142+
}
146143
}
147144
```
148145

149-
- `selected_ids` **must** match the `id` values returned by `/rerank`. The API
150-
rejects unknown IDs to prevent stale writes.
151-
- `success` scores the overall outcome (`1.0` for complete resolution, `0.0`
152-
for failure). Values below `settings.narrative_success_gate` are ignored for
153-
narrative updates but still count toward pheromone decay.
154-
- `best_doc_text`/`best_doc_embedding` are optional hints that let the reranker
155-
update the narrative vector even when a caller reformats the answer before
156-
returning it to the user.
146+
---
157147

158-
On success the endpoint responds with `{"status": "ok"}`.
148+
## Standardized error envelopes
159149

160-
Tip: throttle feedback submissions with a short debounce window (e.g., only send
161-
feedback after end-users click “helpful”) to avoid promoting documents for noisy
162-
sessions.
150+
All errors (including validation) resolve to a stable shape documented in `docs/ERROR_ENVELOPES.md`:
163151

164-
---
152+
```json
153+
{
154+
"error": {
155+
"code": "VALIDATION_ERROR",
156+
"message": "Validation failed",
157+
"detail": [ { "loc": ["body","query"], "msg": "Field required" } ]
158+
}
159+
}
160+
```
161+
162+
Common codes: `BAD_REQUEST`, `UNAUTHORIZED`, `NOT_FOUND`, `ENTITY_TOO_LARGE`, `VALIDATION_ERROR`, `RATE_LIMITED`, `INTERNAL_ERROR`.
165163

166-
## Privacy & retention tips
164+
---
167165

168-
- Set `NEURALCACHE_STORAGE_PERSISTENCE_ENABLED=false` to run fully in-memory. Narrative
169-
vectors and pheromones reset on process restart and never touch disk.
170-
- Configure `NEURALCACHE_STORAGE_RETENTION_DAYS` (e.g., `7`) to purge pheromones and
171-
narrative state older than the retention window on startup. SQLite purges directly
172-
via `metadata`/`pheromones`, and the JSON fallback trims files in place.
173-
- Rotate SQLite files regularly or place them on encrypted storage. Review
174-
[`SECURITY.md`](SECURITY.md) for reporting procedures and deployment guardrails.
166+
## Privacy & data handling
175167

176-
These controls let you scope how long user-derived signals persist while still
177-
benefiting from adaptive reranking.
168+
A concise operator playbook for data classification, retention, and namespace isolation is available in `PRIVACY.md`. Before production, review both `PRIVACY.md` and `SECURITY.md` and set appropriate retention and auth settings.
178169

179170
---
180171

@@ -201,6 +192,9 @@ benefiting from adaptive reranking.
201192
| `NEURALCACHE_DETERMINISTIC_SEED` | Seed used when deterministic mode is enabled | `1337` |
202193
| `NEURALCACHE_EPSILON` | Override ε-greedy exploration rate (0-1). Ignored when deterministic. | _unset_ |
203194
| `NEURALCACHE_MMR_LAMBDA_DEFAULT` | Default MMR lambda when request omits/nulls `mmr_lambda` | `0.5` |
195+
| `NEURALCACHE_NAMESPACE_HEADER` | Header key to read namespace | `X-NeuralCache-Namespace` |
196+
| `NEURALCACHE_DEFAULT_NAMESPACE` | Fallback namespace when header missing | `default` |
197+
| `NEURALCACHE_NAMESPACE_PATTERN` | Validation regex (400 on mismatch) | `^[a-zA-Z0-9_.-]{1,64}$` |
204198

205199
Adjust everything via `.env`, environment variables, or direct `Settings(...)` instantiation. `NEURALCACHE_EPSILON` (when set) takes precedence over `epsilon_greedy` setting unless deterministic mode is active. `NEURALCACHE_MMR_LAMBDA_DEFAULT` supplies fallback diversity weighting when omitted.
206200

0 commit comments

Comments
 (0)