Harden live KA route contracts

Author: kherrera6219

CHANGELOG.md

@@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Settings page crash — `Cannot read properties of undefined (reading 'size_bytes')`**: `DatabaseSettings.tsx` assumed every storage backend metric object was defined. When a backend (Neo4j/Chroma/object store) is not running in local SQLite desktop mode, the metric is `undefined` and the component crashed the whole Settings route. Hardened with `(metric ?? {})` and optional chaining on `data?.size_bytes` / `lastBackup?.size_bytes`; absent backends now render "0 B / Not created" instead of crashing.
 - **Settings API auth/validation drift**: `/api/v1/settings/ai` now uses the JSON/desktop-aware session decorator, accepts signed desktop requests without requiring a pre-existing Flask session, returns JSON unauthorized responses, canonicalizes provider input, restricts AI preference providers to `auto`/`openai`/`google`, and validates model preferences against current defaults.
 - **API route auth-boundary drift**: search, user-data, notification, operational admin, feature-flag, and LLM admin routes now use JSON/desktop-aware session authentication instead of page-style Flask-Login redirects; MCP admin routes no longer block ExternalAPIKey principals with an outer session-only wrapper, and MCP tool execution now builds connector-scope context from the resolved authenticated principal.
+- **KA API data-contract and workflow drift**: live KA routes now use the shared authenticated-principal helper for API-key/session/desktop callers, accept documented `data`/`context` execute and batch payloads while keeping `input` preferred, clamp algorithm pagination, validate batch request shapes, return id-based name fallbacks for sparse registry metadata, tolerate non-numeric layer names, use the real TruthCore API accessor, and bridge sync Flask routes to TruthCore async workflow methods.
 - **Google model selection drift**: the gateway overlay no longer offers the retired Google model before the current `gemini-3.1-pro-preview` default, and LLM-path comments/docstrings now match the live model constants.
 
 ### Removed

HANDOFF.md

@@ -19,7 +19,7 @@
 # DataLogicEngine — Session Handoff
 
 ## START HERE (next session) - updated 2026-07-04
-**Audit checkpoint: documentation slice and code audit slices 1-4 are complete.**
+**Audit checkpoint: documentation slice and code audit slices 1-5 are complete.**
 
 - **Documentation audit slice complete:** root maintained docs and the active `docs/` tree were read against live code. Active docs now align to desktop auth, current gateway/API surfaces, and the live Google default `gemini-3.1-pro-preview`. `docs/openapi.yaml` was replaced with a current partial contract. Legacy duplicate exports were moved from `docs/api/` to `docs/archive/api/`.
 - **Cleanup candidates identified:** root scratch-output files such as `.gitout.txt`, `audit_deep*.txt`, `audit_dup*.txt`, `core_backend_inversions*.txt`, `enc_*.txt`, `commit_msg.txt`, and `orphaned_modules.txt` are not tracked source docs. They remain pending explicit cleanup approval.
@@ -31,8 +31,10 @@
 - **Slice 3 regressions added:** tightened JSON unauthenticated assertions for user-data/search/notification/admin/feature-flag/LLM-admin routes, added a route-level MCP ExternalAPIKey regression for `/api/v1/mcp/clients`, and added unit coverage for `get_authenticated_principal()` plus MCP API-key tool context.
 - **Code audit slice 4 complete - dead KA route module and stale Flask page routes:** `backend/api/ka_management.py` was confirmed unregistered and duplicate of the live `backend/routes/ka_routes.py` API, then removed with its synthetic-only test coverage. The remaining app-level raw Flask-Login decorators were broken server-rendered `/chat` and `/knowledge-graph` routes pointing at missing Jinja templates, so they were removed; Electron/Next owns those UI surfaces.
 - **Slice 4 regressions added:** live KA route tests now prove `/api/v1/ka` and legacy `/api/ka` are registered through the active blueprint, unauthenticated KA list returns JSON 401, KA health remains public, and ExternalAPIKey access reaches the real KA list route. App route wiring now proves Flask does not register stale `/chat` or `/knowledge-graph` template pages.
-- **Validation completed for current checkpoint:** targeted backend ruff passed; frontend `ApiOverlayConfig.test.tsx` passed 8 tests; focused gateway/model pytest passed 53 tests with only a Neo4j driver teardown logging warning after successful exit; focused auth/settings pytest passed 16 tests; frontend API/auth/AI settings tests passed 34 tests; focused route/auth pytest passed 70 tests; live KA/app route pytest passed 10 tests; touched MCP phase route test passed; `scripts/generate_docs.py` refreshed inventory; `scripts/verify_docs_references.py` passes with 0 errors and 17 existing heading/style warnings.
-- **Next audit slice:** continue in the live KA API implementation (`backend/routes/ka_routes.py`) for behavior/data-contract correctness now that the dead duplicate blueprint is removed.
+- **Code audit slice 5 complete - live KA API behavior/data-contract correctness:** `backend/routes/ka_routes.py` now resolves API-key/session/desktop principals through the shared auth helper for execution/workflow logging, accepts the documented `data`/`context` execute payload while keeping `input` preferred, clamps algorithm pagination, falls back to KA id/name-safe metadata for sparse registry entries, validates batch request shapes, tolerates non-numeric layer names, and uses the real TruthCore accessor from `backend.truth_engine.api`.
+- **Slice 5 regressions added:** focused KA route tests now prove invalid pagination does not 500, sparse KA metadata returns a frontend-safe name, API-key execute and batch calls work with the documented payload, malformed execute/batch/workflow bodies return JSON 400s, high-stakes workflow awaits TruthCore async methods through the sync route bridge, trace routes use the real TruthCore accessor, layers accepts non-`L<number>` labels, and legacy `/api/ka` emits deprecation/successor headers.
+- **Validation completed for current checkpoint:** targeted backend ruff passed; frontend `ApiOverlayConfig.test.tsx` passed 8 tests; focused gateway/model pytest passed 53 tests with only a Neo4j driver teardown logging warning after successful exit; focused auth/settings pytest passed 16 tests; frontend API/auth/AI settings tests passed 34 tests; focused route/auth pytest passed 70 tests; live KA/app route pytest passed 10 tests; focused KA route pytest passed 15 tests; touched MCP phase route test passed; `scripts/generate_docs.py` refreshed inventory; `scripts/verify_docs_references.py` passes with 0 errors and 17 existing heading/style warnings.
+- **Next audit slice:** continue from the KA API into KA execution persistence/history correctness (`KAExecution`, history route, risk-tier/name mapping, and execution audit behavior).
 
 ---
 

TODO.md

@@ -18,7 +18,7 @@
 
 # DataLogicEngine TODO
 
-**Last updated:** 2026-07-04 (**Documentation audit + code audit slices 1-4** — root/docs tree reconciled to the live desktop-auth/API/model surface, stale duplicate API exports moved to archive, cleanup candidates identified, LLM provider/model configuration audited, `/api/v1/gateway/keys` hardened, `/api/v1/settings/ai` moved to the JSON/desktop-aware auth decorator, settings provider/model validation canonicalized, API route decorator/session/API-key boundaries hardened across search, user data, notifications, admin, feature flags, MCP, and LLM admin routes, and remaining raw Flask-Login route sites retired by removing a dead duplicate KA management blueprint plus broken server-rendered Flask page routes. Validation: docs reference check 0 errors / 17 existing style warnings, targeted ruff passed, API overlay test 8 passed, AI/API client tests 34 passed, focused auth/settings pytest 16 passed, focused gateway pytest 53 passed, focused route/auth pytest 70 passed, live KA/app route pytest 10 passed, and the touched MCP phase route test passed; pytest emitted a Neo4j teardown logging warning after successful exits. Prior, 2026-07-01: **LLM API & Database Initialization Fix, App Packaging** — database initialization, provider keys, model defaults, PyInstaller backend, Electron packaging, and 429 unit / 20 integration tests passed.)
+**Last updated:** 2026-07-04 (**Documentation audit + code audit slices 1-5** — root/docs tree reconciled to the live desktop-auth/API/model surface, stale duplicate API exports moved to archive, cleanup candidates identified, LLM provider/model configuration audited, `/api/v1/gateway/keys` hardened, `/api/v1/settings/ai` moved to the JSON/desktop-aware auth decorator, settings provider/model validation canonicalized, API route decorator/session/API-key boundaries hardened across search, user data, notifications, admin, feature flags, MCP, and LLM admin routes, remaining raw Flask-Login route sites retired by removing a dead duplicate KA management blueprint plus broken server-rendered Flask page routes, and live KA route data-contract/workflow defects fixed. Validation: docs reference check 0 errors / 17 existing style warnings, targeted ruff passed, API overlay test 8 passed, AI/API client tests 34 passed, focused auth/settings pytest 16 passed, focused gateway pytest 53 passed, focused route/auth pytest 70 passed, live KA/app route pytest 10 passed, focused KA route pytest 15 passed, and the touched MCP phase route test passed; pytest emitted a Neo4j teardown logging warning after successful exits. Prior, 2026-07-01: **LLM API & Database Initialization Fix, App Packaging** — database initialization, provider keys, model defaults, PyInstaller backend, Electron packaging, and 429 unit / 20 integration tests passed.)
 **Status:** Canonical planning source
 
 This is the canonical active TODO list for repository release readiness and operational work. `UKG_DataLogicEngine_Master_Completion_Plan_v1.txt` is the current phased execution plan for the broader UKG/DataLogicEngine completion roadmap; keep release go/no-go items mirrored here when they affect the current shipping branch.
@@ -133,7 +133,31 @@ Validation:
 1. `python -m ruff check app.py tests\integration_routes\test_app_route_wiring.py tests\integration_routes\test_api_routers.py tests\integration_routes\test_ka_route_auth_boundaries.py tests\integration\test_additional_coverage.py` — passed.
 2. `python -m pytest -q --no-cov tests\integration_routes\test_ka_route_auth_boundaries.py tests\integration_routes\test_app_route_wiring.py tests\integration_routes\test_api_routers.py tests\integration\test_app_routes.py` — 10 passed; pytest exited successfully with the shared SQLAlchemy `Query.get()` deprecation warning from `api_decorators.py`.
 
-Next code audit slice: continue in the live KA API implementation (`backend/routes/ka_routes.py`) for behavior/data-contract correctness now that the dead duplicate blueprint is removed.
+## Code audit slice 5 — Live KA API behavior/data-contract correctness — 2026-07-04
+
+Scope completed in this slice:
+
+1. Audited the live KA blueprint (`backend/routes/ka_routes.py`) against the frontend algorithm/history consumers, current `docs/API.md`, the KA master controller contract, and TruthCore async engine contract.
+2. Validated the legacy `/api/ka` alias remains wired to the current blueprint and emits successor/deprecation headers.
+3. Expanded focused live KA route coverage around API-key principals, documented payload shape, pagination bounds, batch input validation, non-numeric layer names, TruthCore workflow access, and trace access.
+
+Findings and fixes addressed before the next audit slice:
+
+| Finding | Resolution | Status |
+| --- | --- | --- |
+| KA execute/high-stakes workflow routes used Flask-Login `current_user` directly even though `api_login_required` also accepts ExternalAPIKey and signed desktop principals through `g.auth_user`. | Switched KA user-id resolution to the shared authenticated-principal helper so API-key principals do not 500 during execution/workflow logging. | Done |
+| `/api/v1/ka/workflow/high-stakes` and `/api/v1/ka/trace/<session_id>` imported the TruthCore accessor from the wrong module, and the high-stakes route called async TruthCore methods synchronously. | Moved both routes to `backend.truth_engine.api.get_truth_core_engine()` and added a sync async bridge for `create_session()` / `process()`. | Done |
+| `docs/API.md` documented an execute body with `data`/`context`, but the route only accepted `input`. | Added compatibility support for `data` plus optional `context`, kept `input` as the preferred payload, and updated the API doc. | Done |
+| `GET /api/v1/ka/algorithms?per_page=0` could divide by zero; registry metadata without `KA_Name` returned `name: null` despite frontend expecting a string. | Clamped `page`/`per_page` and made algorithm formatting fall back to the KA id for missing names/status. | Done |
+| KA batch and layer endpoints could mis-handle malformed request shapes or non-`L<number>` layer names. | Validated batch JSON/object/list input, reused the documented payload parser for batch execution, and made layer sorting tolerant of non-numeric layer labels. | Done |
+| The KA API reference omitted live endpoints beyond list/execute/workflow/trace. | Documented batch, search, categories, layers, dependencies, stats, and public health endpoints in `docs/API.md`. | Done |
+
+Validation:
+
+1. `python -m ruff check backend\routes\ka_routes.py tests\integration_routes\test_ka_route_auth_boundaries.py` — passed.
+2. `python -m pytest -q --no-cov tests\integration_routes\test_ka_route_auth_boundaries.py` — 15 passed; pytest exited successfully with the existing SQLAlchemy `Query.get()` warning and a teardown-only Neo4j logging warning.
+
+Next code audit slice: continue from the KA API into KA execution persistence/history correctness (`KAExecution`, history route, risk-tier/name mapping, and execution audit behavior).
 
 ## Unified Backlog