Skip to content

feat(skills): land audit-rule-author skill (rule + agentic-audit authoring)#155

Open
anmolg1997 wants to merge 1 commit into
mainfrom
chore/land-audit-rule-author-skill
Open

feat(skills): land audit-rule-author skill (rule + agentic-audit authoring)#155
anmolg1997 wants to merge 1 commit into
mainfrom
chore/land-audit-rule-author-skill

Conversation

@anmolg1997

Copy link
Copy Markdown
Collaborator

What

Lands Akhilesh's audit-rule-author Claude Code skill (authored on the evidence-synthesis branch) into .claude/skills/. This is the rule-authoring skill — and the agentic-audit authoring guidance — we had been waiting on.

The skill helps author/maintain compliance audit rule config (backend/app/compliance/rules/):

  • SKILL.md — 3 paths (add/update rules, new framework, analyze results), evaluation-strategy decision tree, VC-* visual-check tags, merge-behavior reference, preflight checklist.
  • references/rule_authoring_playbook.md — authoring conventions & field semantics.
  • references/agentic_audit_guide.md — the agentic_audit strategy guide: target/context section design, context_sources, pass_criteria patterns (signature images, summary-vs-subrow tables, NOT_APPLICABLE thresholds, exclusion counter-examples), postpass validation.
  • scripts/peek_pages.py (OCR + section inspection), analyze_results.py (filter compliance_result.json). evals/ — 3 triggering/behavior evals.

Self-contained against main

All infra the skill references is already merged: validate_cli, registry, profiles, document_profiles.yaml, run_checklist_agentic_postpass.py, and the app/compliance/agentic/ stack. No dependency on the rest of evidence-synthesis (that branch's ~1,600-line rules/segmentation/report changes are a separate review and are intentionally not included here).

Review + main-compat fixes

An independent skill review (plugin-dev:skill-reviewer) found it high-quality and self-contained, with one real drift against main, which I fixed (the only changes to the original):

  • peek_pages.py read a per-page classification.yaml that doesn't exist on main (the pipeline writes segmentation.json, page-range DocumentSection). Rewrote the loader to read segmentation.json (legacy classification.yaml kept as fallback). Verified --list-sections renders real sections on a live doc.
  • Both scripts now honor AT_STORAGE__BASE_PATH (the pipeline's own storage var) and peek_pages.py gained --data-root, instead of only the hardcoded backend/data/documents cwd path.
  • SKILL.md data-location section updated to match.

Follow-ups for the author (non-blocking)

  • Description triggers are broad ("analyze results", "look at findings") — could over-trigger; worth scoping.
  • SKILL.md lists text_primary/llm_arbitrated under validate_cli, but validate_cli evaluates those text-only (doesn't run the escalate/arbitrate merge) — worth a caveat.

Testing

  • Both scripts py_compile clean; peek_pages.py --list-sections smoke-tested against a real doc → correct page-range sections. Skill auto-discovered by the harness (valid structure).

…oring)

Akhilesh's audit-rule-author Claude Code skill, brought over from the
evidence-synthesis branch. It guides authoring/maintaining compliance audit
rule config (backend/app/compliance/rules/), including the agentic_audit
strategy guide (target/context design, pass_criteria patterns, postpass
validation) — the agentic-audit authoring guidance we had been waiting on.

All infra it references (validate_cli, registry, profiles, document_profiles,
run_checklist_agentic_postpass, the agentic stack) is already on main, so the
skill is self-contained.

Main-compat adaptations (the only changes to the original):
- peek_pages.py: read segmentation.json (page-range DocumentSection) instead of
  the nonexistent per-page classification.yaml; legacy classification.yaml kept
  as a fallback. Verified --list-sections renders real sections on main.
- both scripts: honor AT_STORAGE__BASE_PATH (the pipeline's own storage var) and
  add --data-root to peek_pages, instead of only the hardcoded cwd-relative path.
- SKILL.md data-location section updated to match (segmentation.json + storage root).

Follow-ups for the author (non-blocking, documented in the PR): tighten the
description's broad triggers; caveat that validate_cli evaluates
text_primary/llm_arbitrated as text-only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant