feat(sdk): wire semantic scanner into the SDK with on/off flag

[Pranjal0410]

First step on the scanner integration scope. Wires the semantic scanner from PR #58 into the SDK with an on/off config flag and a confidence threshold, so semantic signals flow into the existing signals field on the RiskContext alongside the sidecar's lexical signals.

What changed

sdk/python/acf/firewall.py

• New enable_semantic_scan constructor arg + ACF_SEMANTIC_SCAN env var (env wins; default off so base SDK stays zero-deps)
• New semantic_signal_threshold (default 0.85) — only hits at or above this similarity are forwarded as wire-format signals
• New _run_semantic_scanner and _extract_text helpers that handle each hook's payload shape (string for on_prompt and on_context, dict for on_tool_call and on_memory)
• Lazy import of the scanner package — missing [scanners] extra surfaces a clear FirewallError with the install command, not a cryptic ImportError at request time
• Scanner exceptions logged and swallowed — the sidecar's lexical scan is the safety net, a Python-side scanner failure must not break the request

sdk/python/tests/test_firewall_semantic_scan.py (new)

• 29 unit tests across flag resolution precedence, enabled/disabled paths, threshold filtering, per-hook payload shapes, _extract_text edge cases, scanner-failure handling, missing-extra error path
• Skipped cleanly when the [scanners] extra is not installed (CI without the extra still passes)

Why a threshold

TF-IDF scores both paraphrased attacks and benign surface-similar text in the 0.75–0.85 band — they don't separate cleanly with that backend. A higher SDK-side forwarding threshold keeps weak hits out of OPA's view until we move to sentence-transformers in the upgrade PR (cleaner separation, threshold can drop).

Latency

1000-call microbenchmark on _build_payload:

Config	Per call
Scanner off	0.002 ms
Scanner on (TF-IDF)	0.265 ms
Overhead	+0.262 ms

Well inside the 4–8 ms end-to-end budget. Sentence-transformer overhead will be measured separately in the upgrade PR.

Wire format

Sidecar's RiskContext.Signals already has json:"signals" and the scan stage uses append everywhere (sidecar/internal/pipeline/scan.go). No Go-side changes needed — the aggregate stage iterates rc.Signals regardless of source, so SDK signals and sidecar lexical signals both flow into OPA together.

Tests

29/29 in test_firewall_semantic_scan.py. Full SDK suite 109/109.

What's next

The detection rate measurement report — run the adversarial corpus + benign set twice (flag off vs on), report true positive rate, false positive rate, latency at p50/p95/p99, per-hook breakdown.

[tharindupr]

Good work overall — the on/off flag implementation is solid, the signal wire format matches the sidecar's Signal struct exactly (no Go-side changes needed), and the test coverage is thorough. Three things to fix before merge:

1. Backend is hardcoded to tfidf — sentence-transformer is unreachable

Firewall.init always calls SemanticScanner(backend="tfidf"). The docstring mentions "when users upgrade to the sentence-transformer backend" but there's no way to do that through the Firewall interface. Please add a backend: str = "tfidf" constructor argument (and optionally an ACF_SEMANTIC_SCAN_BACKEND env var) so users with sentence-transformers installed can opt in to the higher-accuracy backend.

2. on_tool_call is mapped to InputType.TOOL_OUTPUT

on_tool_call fires before the tool executes — it's checking the call parameters, not the output. Currently harmless because SemanticScanner.scan() never reads inp.input_type, but the field is described as "determines which policy rules apply" so this will cause problems if the scanner is ever updated to filter by type. Please add TOOL_CALL = "tool_call" to the InputType enum in scanners/models.py and update the hook_to_input_type mapping accordingly.

[Pranjal0410]

thanks for the catches @tharindupr, fixed them all

1. added semantic_backend constructor arg + ACF_SEMANTIC_SCAN_BACKEND env var (env wins, default tfidf)
2. added TOOL_CALL to InputType enum, updated the hook mapping. on_tool_call was incorrectly mapped to TOOL_OUTPUT - it's checking call params, not output

53/53 tests are passing

[tharindupr]

Perfect. Looks good now