Skip to content

Commit c9a0325

Browse files
v0.7 PR 2/6: CheckMetadata remediation policy fields (#25)
* PR 2/6 (v0.7 adoption): CheckMetadata remediation policy fields Adds three remediation policy fields to CheckMetadata, populates docs_url for all 45 entries, and adds 7 missing per-check sections to docs/checks.md so every check has a stable anchor. What changed: - core/models.py: CheckMetadata gains autofix_safe (bool, default False), requires_human_review (bool, default True), suggested_patch_kind (Literal, default "manual"). Defaults are the safe-closed shape so unannotated checks remain conservative. - checks/registry.py: introduces a `_meta()` helper that auto-derives docs_url from the check ID and applies per-check policy from a small `_REMEDIATION_OVERRIDES` table. The 45 existing `CheckMetadata(id=...)` calls are renamed to `_meta(id=...)`. v0.7 policy table: - Stale-manifest checks (3 IDs) → autofix_safe=True, requires_human_review=False, suggested_patch_kind=remove_pointer - SHIP-AUTH-SCOPE-COVERAGE-MISSING → suggested_patch_kind= append_pointer (autofix_safe stays False — adding scopes can encode policy choices and apply-patches --confidence high deliberately skips medium-confidence patches). - All other checks default to manual / human review required. - docs/checks.md: 7 new H3 per-check sections for the v0.4 atomic OpenAI API readiness checks (RETRY-POLICY, TIMEOUT, TEST-CASES, TOOL-OUTPUT-SCHEMA, RETRY-WITHOUT-IDEMPOTENCY, TRACE-APPROVAL, TRACE-CONFIRMATION). They were previously documented inline in a shared section table without per-check anchors, so docs_url resolution would 404. Trace-approval/confirmation entries explicitly document the "do not edit the trace" anti-pattern. - docs/checks.json: regenerated. Every entry now carries autofix_safe, requires_human_review, suggested_patch_kind, and docs_url. - tests/test_remediation_metadata.py (new, 12 tests): population test (every check has docs_url + policy fields), default safe-closed policy, parametrized stale-manifest assertions, scope-coverage medium-confidence guard, parametrized trace-finding manual permanence, anchor resolution against docs/checks.md headings, JSON dump carries new keys, regenerated checks.json reflects the changes. Verification: - 275 tests passing (was 263; +12 new). - python -m ruff check . — clean. - python scripts/generate_schemas.py leaves a clean tree (other than the intended docs/checks.json + docs/checks.md changes; report schema unchanged because no Finding/ReadinessReport fields were touched in this PR). NO schema bump in this PR. report_schema_version stays 0.6 until the Finding fields and derivation logic land in PR 3, where the bump is correct per STABILITY.md (additive change → minor bump). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * PR 2 review fix: catalog autofix_safe stays conservative for stale checks P2 review feedback on PR #25 addressed. The earlier override marked stale-manifest checks as `autofix_safe=True, requires_human_review=False` at the catalog level. But the existing patch generator (`checks/patches.py::_gen_stale_*`) deliberately falls back to `ManualPatch` when ≥ 2 manifest entries match the same evidence (ambiguous removal). That meant `list-checks --json` and `explain --json` would tell agents "this class of findings is safe to apply without review" while a real `scan --suggest-patches` could emit a ManualPatch — a contract violation that could lead agents to skip review on findings that need it. Fix: catalog-level `autofix_safe` and `requires_human_review` now describe the *worst-case* per-check outcome. The four checks with non-manual generators (3 stale-manifest + scope coverage) keep their `suggested_patch_kind` overrides (which are informational — documenting the kind the generator targets when conditions are clean) but inherit the safe-closed defaults for the safety bools. This matches the planned catalog-vs-Finding contract: catalog says "assume needs review unless you read the actual emitted patch"; per-Finding fields (PR 3) read `Finding.patches` and tell the truth for each instance. An agent acting on a specific finding must consult those, not the catalog. Also extended the registry header comment and the test docstring to make the contract explicit so future contributors don't re-introduce the over-promise. Updated the parametrized stale- manifest test name and assertions, plus the regenerated-checks.json test, to pin the new conservative values. `docs/checks.json` regenerated. 275 tests passing, ruff clean.
1 parent 5e73142 commit c9a0325

5 files changed

Lines changed: 602 additions & 135 deletions

File tree

0 commit comments

Comments
 (0)