diff --git a/README.md b/README.md index f79df12..9848f34 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,10 @@

- 94 policies. 15+ regulatory frameworks. 5 industry verticals. Policy-as-code your auditor can read. + AI compliance rules you can read, run, diff, and prove. +

+

+ 94 policies · 15+ regulatory frameworks · 5 industry verticals

@@ -31,7 +34,9 @@
-**Governance Open Policy Agent Library** — a curated collection of [OPA](https://www.openpolicyagent.org/) policies, written in Rego, that encode real AI-governance requirements: the EU AI Act, NIST AI RMF, aviation safety standards, FERPA/COPPA in education, fair-lending rules in banking, and more. +**GOPAL: Governance Open Policy Agent Library.** Think of it as an open policy pack for AI regulation. + +A curated collection of [OPA](https://www.openpolicyagent.org/) policies, written in Rego, that encode real AI-governance requirements: the EU AI Act, NIST AI RMF, aviation safety standards, FERPA/COPPA in education, fair-lending rules in banking, and more. Run them against your AI system's metadata, model cards, or evaluation results — and get back a structured, machine-readable compliance verdict you can drop into CI, an audit log, or a regulator submission. @@ -44,6 +49,30 @@ Run them against your AI system's metadata, model cards, or evaluation results --- +## AI compliance rules you can read, run, diff, and prove + +GOPAL turns regulatory and governance requirements — the EU AI Act, NIST AI RMF, aviation safety standards, FERPA/COPPA, fair lending, and healthcare safety — into executable OPA policies. + +Use GOPAL when you want AI governance checks that are: + +- **Readable** — every rule is Rego, not a black-box score +- **Reviewable** — policy changes go through pull requests +- **Testable** — every policy can have allow/deny test cases +- **Versioned** — frameworks evolve without breaking pinned users +- **Automatable** — run checks in CI/CD, audit workflows, or AICertify + +--- + +## Why now + +The EU AI Act is in force. The NIST AI RMF is the de facto US baseline. The UK, India, Brazil, Singapore, and California are all moving. Aviation regulators are publishing AI/UAS guidance. Financial supervisors are issuing model-risk requirements. + +Engineering teams need AI governance checks that run in CI — not PDFs that sit on a shared drive, not screenshots pasted into review-board decks. + +GOPAL ships executable Rego policies for each of those regimes. They are versioned, testable, and reviewable in pull requests. The same tooling your platform team already uses for Kubernetes admission control can now enforce AI-system requirements. + +--- + ## Quick Start

@@ -53,6 +82,16 @@ Run them against your AI system's metadata, model cards, or evaluation results

+### Try GOPAL in 30 seconds + +```bash +git clone https://github.com/Principled-Evolution/gopal.git +cd gopal/examples/eu-ai-act-transparency +./run.sh +``` + +You'll see a structured EU AI Act transparency verdict against a sample AI system. See [`examples/`](examples/) for NIST AI RMF, customer-support LLM, and more. + ### Standalone with the OPA CLI ```bash @@ -99,6 +138,22 @@ GOPAL is different on three axes: --- +## For OPA / Rego users + +If you already run OPA for Kubernetes admission, cloud authorization, CI/CD, or service mesh, GOPAL gives you a policy library targeted at AI systems instead of infrastructure. + +The packages, conventions, and test patterns are idiomatic Rego — no DSL on top, no Python required to evaluate. You can: + +- pull individual frameworks (`international/eu_ai_act/v1/`, `industry_specific/aviation/v1/`) into a bundle +- evaluate with `opa eval`, [Conftest](https://www.conftest.dev/), or your existing OPA server +- pin to a major version (`v1/`) and review upgrades as PRs +- compose GOPAL rules with your private `custom/` rules in the same evaluation +- lint with [Regal](https://github.com/StyraInc/regal) — the same linter GOPAL itself runs in CI + +If you want a Python framework that handles input capture and PDF/Markdown report generation on top, see [AICertify](https://github.com/Principled-Evolution/aicertify). + +--- + ## What's Inside

@@ -164,6 +219,19 @@ gopal/ --- +## GOPAL vs AICertify + +| Need | Use | +|---|---| +| I want raw Rego policies | GOPAL | +| I want to evaluate an AI app and generate reports | AICertify | +| I want to plug policies into existing OPA tooling | GOPAL | +| I want PDF/Markdown/JSON audit reports | AICertify | + +AICertify uses GOPAL underneath. Pick GOPAL if you already have an OPA workflow you want to extend with AI-specific rules. Pick AICertify if you want a Python framework that captures AI-application interactions and produces audit-ready evidence end-to-end. + +--- + ## Authoring Policies

@@ -206,6 +274,23 @@ Then a sibling `*_test.rego` covers the rule. CI enforces: The [helper_functions/](helper_functions/) library gives you `compose_report()`, `validate_required_fields()`, and `field_exists()` so reports come out in a uniform shape no matter who wrote the rule. +See [`docs/tutorials/add-your-first-policy.md`](docs/tutorials/add-your-first-policy.md) for a walkthrough, and [`docs/coverage/`](docs/coverage/) for per-framework coverage matrices. + +--- + +## Policy correctness + +GOPAL is not legal advice. The policies here are executable interpretations of public regulatory and governance requirements, written by engineers who care about getting them right. + +If you believe a rule misreads a regulation or misses an obligation, please open an issue with: + +- the regulation, section, or article in question +- your interpretation +- the input/output behavior you'd expect +- any official guidance, regulator text, or precedent + +Policy-correctness disagreements are not security vulnerabilities — see [SECURITY.md](SECURITY.md) for the latter. They are exactly the kind of issue we want public so the community can review and improve the rules together. + --- ## Custom Policies diff --git a/docs/INDEX.md b/docs/INDEX.md index 03dbc57..f07a135 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -7,6 +7,8 @@ Organized along [Diátaxis](https://diataxis.fr/) lines. ## 🎓 Tutorials — your first GOPAL evaluation - [Quick Start in the README](../README.md#quick-start) — standalone `opa eval`, or via AICertify. +- [Add your first GOPAL policy](tutorials/add-your-first-policy.md) — write a working policy + test + CI check in 20 minutes. +- [Runnable examples](../examples/README.md) — `examples/` with input/output pairs for EU AI Act, NIST AI RMF, and customer-support LLM. - [Add a brand-new regulatory framework](../skills/add-framework/SKILL.md) (Claude Code skill) — bootstraps the directory tree, framework README, and first policy. - [FAQ](FAQ.md) — comprehensive Q&A on what GOPAL is, when to use it, and how to extend it. @@ -19,6 +21,7 @@ Organized along [Diátaxis](https://diataxis.fr/) lines. ## 📚 Reference — look up specific names +- [Policy coverage matrices](coverage/) — per-framework view of what's Implemented / Scaffold / Planned. - [Coverage table](../README.md#whats-inside) — every framework with its policy count. - [helper_functions/reporting.rego](../helper_functions/reporting.rego) — `compose_report()`, validators. - [helper_functions/validation.rego](../helper_functions/validation.rego) — `field_exists()`, `validate_required_fields()`. diff --git a/docs/coverage/README.md b/docs/coverage/README.md new file mode 100644 index 0000000..3192a3c --- /dev/null +++ b/docs/coverage/README.md @@ -0,0 +1,35 @@ +# Policy coverage matrices + +This directory documents, per framework, **which obligations are encoded in GOPAL** and **which are not yet**. + +The matrices are deliberately honest. A policy is only marked **Implemented** when the Rego rule actually validates input fields against the regulation's requirement. Many directories ship a **Scaffold** that establishes the package path and a `default allow := false` placeholder — useful as a starting point for contributors, not yet enforceable. + +## Available matrices + +- [EU AI Act](eu-ai-act.md) — Regulation (EU) 2024/1689 +- [NIST AI RMF](nist-ai-rmf.md) — NIST Special Publication 1270 + AI 600-1 + +## Coming soon + +- UK AI principles (pro-innovation framework) +- India / DPDP Act +- Brazil AI Governance Bill +- ICAO Doc 10019 (aviation) +- FERPA / COPPA (education) +- Healthcare AI safety +- BFS — fair lending, model risk + +If you want to help expand coverage for a framework, open an issue or send a PR. The matrices are the best place to start — they show contributors exactly which articles, controls, or sections are still open. + +## Reading a matrix + +Each row is one obligation, control, or article in the source regulation. Columns: + +| Column | Meaning | +|---|---| +| **Obligation** | The regulator's name for the requirement (article number, control ID, etc.) | +| **GOPAL policy** | Path to the Rego package that encodes it | +| **Status** | `Implemented` / `Scaffold` / `Planned` | +| **Notes** | What the rule checks, or what's missing | + +`Implemented` rules are safe to run in CI and produce structured verdicts. `Scaffold` rules return placeholder denials — they exist so the package path is stable while the logic is fleshed out. `Planned` means there's no file yet. diff --git a/docs/coverage/eu-ai-act.md b/docs/coverage/eu-ai-act.md new file mode 100644 index 0000000..37a16dd --- /dev/null +++ b/docs/coverage/eu-ai-act.md @@ -0,0 +1,83 @@ +# EU AI Act — coverage matrix + +Source: **Regulation (EU) 2024/1689** (the Artificial Intelligence Act). + +Policies live under [`international/eu_ai_act/v1/`](../../international/eu_ai_act/v1). + +Legend: ✅ **Implemented** — checks real input fields against the obligation. ⚠️ **Scaffold** — package exists, returns placeholder denial. 📋 **Planned** — not in repo yet. + +## Title II — Prohibited AI practices (Article 5) + +| Obligation | GOPAL policy | Status | Notes | +|---|---|---|---| +| Article 5(1)(a) — Manipulative techniques | [`prohibited_practices/manipulation`](../../international/eu_ai_act/v1/prohibited_practices/manipulation.rego) | ✅ | Detects deployment of subliminal / manipulative techniques | +| Article 5(1)(a) — Emotion recognition (workplace/edu) | [`prohibited_practices/emotion_recognition`](../../international/eu_ai_act/v1/prohibited_practices/emotion_recognition.rego) | ✅ | Detects emotion-recognition systems in workplace/educational contexts | +| Article 5(1)(b) — Vulnerability exploitation | [`prohibited_practices/vulnerability_exploitation`](../../international/eu_ai_act/v1/prohibited_practices/vulnerability_exploitation.rego) | ✅ | Detects exploitation of age/disability/social vulnerabilities | +| Article 5(1)(c) — Social scoring | [`prohibited_practices/social_scoring`](../../international/eu_ai_act/v1/prohibited_practices/social_scoring.rego) | ✅ | Detects social-scoring systems leading to detrimental treatment | +| Article 5(1)(d) — Criminal-offense profiling | [`prohibited_practices/criminal_profiling`](../../international/eu_ai_act/v1/prohibited_practices/criminal_profiling.rego) | ⚠️ | Package exists; logic placeholder | +| Article 5(1)(e) — Untargeted facial-recognition scraping | [`prohibited_practices/facial_recognition_scraping`](../../international/eu_ai_act/v1/prohibited_practices/facial_recognition_scraping.rego) | ⚠️ | Package exists; logic placeholder | +| Article 5(1)(b)+(g) — Biometric categorization | [`prohibited_practices/biometric_categorization`](../../international/eu_ai_act/v1/prohibited_practices/biometric_categorization.rego) | ⚠️ | Package exists; logic placeholder | +| Article 5(1)(h) — Real-time remote biometric identification | [`prohibited_practices/biometric_identification`](../../international/eu_ai_act/v1/prohibited_practices/biometric_identification.rego) | ⚠️ | Package exists; logic placeholder | + +## Title III — High-risk AI systems (Articles 6 – 27) + +### Chapter 2 — Requirements for high-risk AI systems + +| Obligation | GOPAL policy | Status | Notes | +|---|---|---|---| +| Article 9 — Risk management system | [`risk_management/risk_management`](../../international/eu_ai_act/v1/risk_management/risk_management.rego) | ✅ | Checks documented risk-management process & residual risk acceptance | +| Article 10 — Data and data governance | [`data_governance/data_quality`](../../international/eu_ai_act/v1/data_governance/data_quality.rego) | ⚠️ | Scaffold | +| Article 10 — Training data | [`data_governance/training_data`](../../international/eu_ai_act/v1/data_governance/training_data.rego) | ⚠️ | Scaffold | +| Article 11 — Technical documentation | [`documentation/technical_documentation`](../../international/eu_ai_act/v1/documentation/technical_documentation.rego) | ✅ | Checks `documentation.technical_documentation.completeness` | +| Article 12 — Record-keeping (logging) | [`documentation/record_keeping`](../../international/eu_ai_act/v1/documentation/record_keeping.rego) | ⚠️ | Scaffold | +| Article 12 — Automated logs | [`documentation/automated_logs`](../../international/eu_ai_act/v1/documentation/automated_logs.rego) | ⚠️ | Scaffold | +| Article 13 — Transparency to deployers | [`transparency/transparency`](../../international/eu_ai_act/v1/transparency/transparency.rego) | ✅ | Checks documentation completeness + toxicity threshold | +| Article 14 — Human oversight | [`human_oversight/human_oversight`](../../international/eu_ai_act/v1/human_oversight/human_oversight.rego) | ⚠️ | Scaffold | +| Article 15 — Accuracy, robustness, cybersecurity | [`technical_robustness/robustness`](../../international/eu_ai_act/v1/technical_robustness/robustness.rego) | ⚠️ | Scaffold | +| Article 15 — Fairness obligations | [`eu_fairness/eu_fairness`](../../international/eu_ai_act/v1/eu_fairness/eu_fairness.rego) | ✅ | Checks bias-metric thresholds and protected-class coverage | + +### Chapter 3 — Obligations of providers, deployers, importers, distributors + +| Obligation | GOPAL policy | Status | Notes | +|---|---|---|---| +| Article 16 + 17 — Provider obligations | [`obligations/provider_obligations`](../../international/eu_ai_act/v1/obligations/provider_obligations.rego) | ⚠️ | Scaffold | +| Article 23 — Importer obligations | [`obligations/importer_obligations`](../../international/eu_ai_act/v1/obligations/importer_obligations.rego) | ⚠️ | Scaffold | +| Article 24 — Distributor obligations | [`obligations/distributor_obligations`](../../international/eu_ai_act/v1/obligations/distributor_obligations.rego) | ⚠️ | Scaffold | +| Article 26 — Deployer obligations | [`obligations/deployer_obligations`](../../international/eu_ai_act/v1/obligations/deployer_obligations.rego) | ⚠️ | Scaffold | + +### Chapter 5 — Conformity assessment and CE marking + +| Obligation | GOPAL policy | Status | Notes | +|---|---|---|---| +| Article 43 — Conformity assessment | [`compliance/conformity_assessment`](../../international/eu_ai_act/v1/compliance/conformity_assessment.rego) | ⚠️ | Scaffold | +| Article 47 — EU declaration of conformity | [`compliance/declaration_conformity`](../../international/eu_ai_act/v1/compliance/declaration_conformity.rego) | ⚠️ | Scaffold | +| Article 48 — CE marking | [`compliance/ce_marking`](../../international/eu_ai_act/v1/compliance/ce_marking.rego) | ⚠️ | Scaffold | +| Article 49 — Registration in EU database | [`compliance/registration`](../../international/eu_ai_act/v1/compliance/registration.rego) | ⚠️ | Scaffold | + +## Title V — General-purpose AI models (Articles 51 – 56) + +| Obligation | GOPAL policy | Status | Notes | +|---|---|---|---| +| Article 53 — GPAI technical documentation | [`gpai/technical_documentation`](../../international/eu_ai_act/v1/gpai/technical_documentation.rego) | ⚠️ | Scaffold | +| Article 53 — Downstream transparency | [`gpai/downstream_transparency`](../../international/eu_ai_act/v1/gpai/downstream_transparency.rego) | ⚠️ | Scaffold | +| Article 51 — Systemic-risk classification | [`gpai/systemic_risk_classification`](../../international/eu_ai_act/v1/gpai/systemic_risk_classification.rego) | ⚠️ | Scaffold | +| Article 52 — Transparency obligations for certain systems | [`transparency/transparency`](../../international/eu_ai_act/v1/transparency/transparency.rego) | ✅ | Shared with Article 13 | + +## Not yet covered (📋 Planned) + +| Obligation | Why it's open | Help wanted | +|---|---|---| +| Article 6 — High-risk classification rules | Needs a structured input schema mapping Annex I/III to system attributes | Yes | +| Article 27 — Fundamental-rights impact assessment | Needs FRIA template + checklist input | Yes | +| Article 50 — Transparency to natural persons (deepfakes etc.) | Needs use-case taxonomy and disclosure-evidence input | Yes | +| Article 55 — Obligations for GPAI with systemic risk | Needs systemic-risk evaluation results input | Yes | +| Article 72 — Post-market monitoring | Needs post-deployment metrics input | Yes | +| Annex IV — Technical documentation contents | Currently aggregated into `technical_documentation`; could be split per-line | Maybe | + +## How to help + +1. **Promote a scaffold to implementation.** Pick a row marked ⚠️ Scaffold, read the corresponding article, and replace the placeholder logic with concrete field checks. See [`docs/tutorials/add-your-first-policy.md`](../tutorials/add-your-first-policy.md). +2. **Open a 📋 Planned row.** Comment on the relevant issue (or open one) with your interpretation of the obligation and a proposed input schema before sending a PR. +3. **Disagree with our coverage call?** Open an issue. We'd rather have the dispute in public than ship rules that misread the regulation. + +> ⚠️ Reminder: GOPAL is not legal advice. The matrix above is GOPAL's *engineering* interpretation of where each obligation maps. Use it as a starting point for your own compliance review, not a substitute for one. diff --git a/docs/coverage/nist-ai-rmf.md b/docs/coverage/nist-ai-rmf.md new file mode 100644 index 0000000..df2c980 --- /dev/null +++ b/docs/coverage/nist-ai-rmf.md @@ -0,0 +1,88 @@ +# NIST AI RMF — coverage matrix + +Source: [**NIST AI Risk Management Framework (AI RMF 1.0)**](https://www.nist.gov/itl/ai-risk-management-framework) and the [Generative AI Profile (NIST AI 600-1)](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf). + +Policies live under [`international/nist/v1/`](../../international/nist/v1). + +Legend: ✅ **Implemented** — checks real input fields against the obligation. ⚠️ **Scaffold** — package exists, returns placeholder denial. 📋 **Planned** — not in repo yet. + +## Core functions + +The AI RMF organizes AI risk management into four functions: **Govern, Map, Measure, Manage**. GOPAL ships one package per function plus an orchestrator (`ai_600_1`). + +| Function | GOPAL package | Status | Notes | +|---|---|---|---| +| Govern | [`govern`](../../international/nist/v1/govern/governance.rego) | ✅ | Composes Accountability + Transparency + Fairness sub-checks against `input.governance`, `input.transparency`, `input.fairness` | +| Map | [`map`](../../international/nist/v1/map/map.rego) | ⚠️ | Structure in place; sub-checks return placeholder fields | +| Measure | [`measure`](../../international/nist/v1/measure/measure.rego) | ⚠️ | Structure in place; sub-checks return placeholder fields | +| Manage | [`manage`](../../international/nist/v1/manage/manage.rego) | ⚠️ | Structure in place; sub-checks return placeholder fields | +| Orchestrator (all four) | [`ai_600_1`](../../international/nist/v1/ai_600_1/ai_600_1.rego) | ✅ | Single entry point — requires all four function inputs to be present | + +## Govern — sub-categories implemented + +| Category | What the rule checks | GOPAL field | +|---|---|---| +| GOVERN 1 — Accountability | Roles & responsibilities documented; oversight mechanisms in place | `input.governance.roles_and_responsibilities_defined`, `input.governance.oversight_mechanisms_in_place` | +| GOVERN 2 — Transparency | Public documentation available; decision explanations provided | `input.transparency.public_documentation_available`, `input.transparency.decision_explanations_provided` | +| GOVERN 3 — Fairness | Bias assessments conducted; mitigation strategies in place | `input.fairness.bias_assessments_conducted`, `input.fairness.bias_mitigation_strategies_in_place` | + +See [`examples/nist-ai-rmf-govern/`](../../examples/nist-ai-rmf-govern/) for a runnable example. + +## Map — categories with scaffolding + +The Map function focuses on **context**: who's affected, what the use case is, what risks are known. GOPAL ships package-level scaffolding ready for fleshed-out logic. + +| Category | Status | Help wanted | +|---|---|---| +| MAP 1 — Context established | ⚠️ Scaffold | Yes — needs input schema for use-case description, deployment context, affected populations | +| MAP 2 — Categorization of the AI system | ⚠️ Scaffold | Yes — needs taxonomy mapping (LLM, computer vision, recommender, …) | +| MAP 3 — Capabilities & limitations characterized | ⚠️ Scaffold | Yes — needs eval-result schema | +| MAP 4 — Risks & benefits mapped | ⚠️ Scaffold | Yes — needs risk-register input | +| MAP 5 — Impacts characterized | ⚠️ Scaffold | Yes — needs stakeholder-impact input | + +## Measure — categories with scaffolding + +The Measure function operationalizes risk into metrics. GOPAL has the package structure; the substantive metric thresholds are still open. + +| Category | Status | Help wanted | +|---|---|---| +| MEASURE 1 — Appropriate methods identified | ⚠️ Scaffold | Yes | +| MEASURE 2 — Trustworthiness characteristics evaluated | ⚠️ Scaffold | Yes — needs per-characteristic threshold schema (accuracy, robustness, fairness, etc.) | +| MEASURE 3 — Tracked over time | ⚠️ Scaffold | Yes | +| MEASURE 4 — Feedback gathered | ⚠️ Scaffold | Yes | + +## Manage — categories with scaffolding + +| Category | Status | Help wanted | +|---|---|---| +| MANAGE 1 — Risks prioritized | ⚠️ Scaffold | Yes | +| MANAGE 2 — Risk responses planned | ⚠️ Scaffold | Yes | +| MANAGE 3 — Risks from third parties addressed | ⚠️ Scaffold | Yes | +| MANAGE 4 — Risk treatment documented | ⚠️ Scaffold | Yes | + +## Generative AI Profile (NIST AI 600-1) + +| Risk category | Status | Notes | +|---|---|---| +| CBRN information / capabilities | 📋 Planned | Awaiting input schema | +| Confabulation | 📋 Planned | Could overlap with `global/v1/toxicity` patterns | +| Dangerous, violent, hateful content | 📋 Planned | Overlap with `global/v1/toxicity` | +| Data privacy | 📋 Planned | Coordinate with `industry_specific/education/v1/student_data_privacy` | +| Environmental impacts | 📋 Planned | Coordinate with `operational/cost/v1` | +| Harmful bias and homogenization | 📋 Planned | Coordinate with `global/v1/fairness` | +| Human-AI configuration | 📋 Planned | | +| Information integrity | 📋 Planned | | +| Information security | 📋 Planned | Coordinate with `operational/corporate/v1` | +| Intellectual property | 📋 Planned | | +| Obscene, degrading, abusive content | 📋 Planned | Overlap with `global/v1/toxicity` | +| Value chain & component integration | 📋 Planned | | + +The `ai_600_1` orchestrator package is in place ([`international/nist/v1/ai_600_1/`](../../international/nist/v1/ai_600_1)) — what's needed is per-risk rules that compose into it. + +## How to help + +1. **Promote a scaffold.** Pick a Map/Measure/Manage category and write field checks against a concrete input schema. Use [`govern/governance.rego`](../../international/nist/v1/govern/governance.rego) as a template. +2. **Open a 600-1 risk.** Pick one of the 12 risk categories above and propose an input schema + Rego rule. +3. **Push back on coverage calls.** If you read NIST differently, open an issue. The matrix is the team's interpretation — not the framework itself. + +> ⚠️ Reminder: GOPAL is not legal advice or an official NIST artifact. The matrix above is GOPAL's *engineering* mapping. Use it as a starting point for your own AI risk assessment, not a substitute for one. diff --git a/docs/tutorials/add-your-first-policy.md b/docs/tutorials/add-your-first-policy.md new file mode 100644 index 0000000..6a7f6c2 --- /dev/null +++ b/docs/tutorials/add-your-first-policy.md @@ -0,0 +1,215 @@ +# Add your first GOPAL policy + +In 20 minutes you'll write a working AI-governance policy, a test for it, and have both pass GOPAL's CI checks. + +We'll write a simple policy: **"the AI system must log every model invocation."** That's a piece of NIST AI RMF *Measure* and EU AI Act Article 12, but here we'll treat it as a freestanding rule so the example stays self-contained. + +## Prerequisites + +```bash +curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_linux_amd64 \ + && chmod +x opa && sudo mv opa /usr/local/bin/ +curl -L -o regal https://github.com/StyraInc/regal/releases/latest/download/regal_Linux_x86_64 \ + && chmod +x regal && sudo mv regal /usr/local/bin/ +git clone https://github.com/Principled-Evolution/gopal.git +cd gopal +``` + +## Step 1 — Decide where the policy lives + +GOPAL groups policies by **scope**: + +| Top-level directory | When to use | +|---|---| +| `international//v/` | Named regulation crossing borders (EU AI Act, NIST AI RMF, ICAO) | +| `industry_specific//v/` | Vertical-specific (aviation, education, healthcare, bfs, automotive) | +| `global/v/` | Cross-cutting principles (fairness, transparency, toxicity, accountability) | +| `operational//v/` | Internal ops (cost, AIOps, corporate InfoSec) | +| `custom/` | Your private rules — git-ignored | + +For "log every invocation," we'll use `global/v1/logging` because it's cross-cutting. + +## Step 2 — Scaffold the package + +```bash +mkdir -p global/v1/logging +``` + +Create `global/v1/logging/logging.rego`: + +```rego +package global.v1.logging + +import rego.v1 + +metadata := { + "title": "Invocation Logging Required", + "description": "Every AI-system invocation must be logged for auditability.", + "version": "1.0.0", + "category": "global", + "references": [ + "NIST AI RMF MEASURE 3.2 — Track AI risks and trustworthiness over time", + "EU AI Act Article 12 — Record-keeping", + ], +} + +default allow := false + +allow if { + input.logging.enabled == true + input.logging.retention_days >= 90 +} + +default compliance_report := { + "policy_name": "Invocation Logging Required", + "compliant": false, + "reason": "Logging is not configured to meet the minimum requirement.", + "recommendations": [ + "Enable invocation logging.", + "Retain logs for at least 90 days.", + ], +} + +compliance_report := { + "policy_name": "Invocation Logging Required", + "compliant": true, + "reason": "Logging is enabled with sufficient retention.", + "recommendations": [], +} if allow +``` + +### What's happening + +- **Package path** mirrors the directory: `global.v1.logging` lives at `global/v1/logging/`. CI checks this. +- **`import rego.v1`** opts into the modern Rego dialect (required by GOPAL). +- **`metadata`** is a structured comment-as-data block. Tooling can read it; auditors can read it. +- **`default allow := false`** — every GOPAL policy denies by default. The `allow` rule must explicitly establish compliance. +- **`compliance_report`** — the consumer-facing output. Always returns the same shape (`policy_name`, `compliant`, `reason`, `recommendations`). + +## Step 3 — Write a test + +Create `global/v1/logging/logging_test.rego` next to your policy. The trailing `_test.rego` suffix and the matching package name are how OPA finds tests. + +```rego +package global.v1.logging_test + +import data.global.v1.logging +import rego.v1 + +test_allow_when_logging_enabled_and_retention_90 if { + logging.allow with input as { + "logging": {"enabled": true, "retention_days": 90}, + } +} + +test_deny_when_logging_disabled if { + not logging.allow with input as { + "logging": {"enabled": false, "retention_days": 365}, + } +} + +test_deny_when_retention_below_90 if { + not logging.allow with input as { + "logging": {"enabled": true, "retention_days": 30}, + } +} + +test_compliance_report_compliant if { + report := logging.compliance_report with input as { + "logging": {"enabled": true, "retention_days": 90}, + } + report.compliant == true +} + +test_compliance_report_non_compliant if { + report := logging.compliance_report with input as { + "logging": {"enabled": false, "retention_days": 0}, + } + report.compliant == false + count(report.recommendations) > 0 +} +``` + +## Step 4 — Run it locally + +Run the tests: + +```bash +opa test global/v1/logging/ -v +``` + +You should see five `PASS` lines. + +Now evaluate it against a sample input: + +```bash +echo '{"logging": {"enabled": true, "retention_days": 365}}' \ + | opa eval -d global/v1/logging --stdin-input \ + --format pretty \ + "data.global.v1.logging.compliance_report" +``` + +Expected output: + +```json +{ + "policy_name": "Invocation Logging Required", + "compliant": true, + "reason": "Logging is enabled with sufficient retention.", + "recommendations": [] +} +``` + +Flip `enabled` to `false` and you'll get a non-compliant verdict with recommendations. + +## Step 5 — Pass CI checks + +GOPAL's CI runs two gates. Run them both locally: + +```bash +opa check --ignore custom/ . +regal lint --ignore-files custom/ . +``` + +Both must pass. Common Regal fixes: + +- **Variable shadowing** — rename a local variable that shadows an import +- **Use `every`** instead of comprehensions for clarity +- **Use `rule.metadata.title`** in opa fmt-friendly metadata blocks + +If you need help with a specific Regal violation, see [the Regal docs](https://docs.styra.com/regal). + +## Step 6 — (Optional) Add an example + +If your policy is useful as a standalone demo, drop it under [`examples/`](../../examples/): + +```text +examples/ + logging-retention/ + input.json + run.sh + expected-output.json + README.md +``` + +See [`examples/eu-ai-act-transparency/`](../../examples/eu-ai-act-transparency/) for the conventions. + +## Step 7 — Open a PR + +Use the `new_policy.md` issue template first if you want to discuss the obligation interpretation before writing code. Otherwise send a PR with: + +- the policy file +- the test file +- (optionally) an example +- a one-paragraph description in the PR linking to the regulation or governance principle it encodes + +CI will run `opa check`, `opa test`, and `regal lint`. The CHANGELOG should get an entry noting the new policy. + +## Going further + +| Want to … | See | +|---|---| +| Add a brand-new regulatory framework | [`skills/add-framework/SKILL.md`](../../skills/add-framework/SKILL.md) — Claude Code skill that scaffolds the directory tree, framework README, and first policy | +| Use the standard reporting helpers | [`helper_functions/reporting.rego`](../../helper_functions/reporting.rego) — `compose_report()`, `validate_required_fields()` | +| Understand authoring conventions in depth | [`AGENTS.md`](../../AGENTS.md) | +| Check which obligations are still open | [`docs/coverage/`](../coverage/) | diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..8054474 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,38 @@ +# GOPAL Examples + +Runnable, copyable examples of evaluating an AI system against a GOPAL policy. + +Each example contains: + +- `input.json` — a sample AI-system payload (the thing being evaluated) +- `run.sh` — invokes `opa eval` against a GOPAL policy with that input +- `expected-output.json` — what the verdict should look like +- `README.md` — what the example shows and how to adapt it + +## Prerequisites + +You need [OPA](https://www.openpolicyagent.org/docs/latest/#running-opa) on your PATH: + +```bash +curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_linux_amd64 \ + && chmod +x opa && sudo mv opa /usr/local/bin/ +``` + +## Examples + +| Example | Framework | Policy | Verdict | +|---|---|---|---| +| [`eu-ai-act-transparency/`](eu-ai-act-transparency/) | EU AI Act | `international.eu_ai_act.v1.transparency` | passing | +| [`nist-ai-rmf-govern/`](nist-ai-rmf-govern/) | NIST AI RMF | `international.nist.v1.govern` | passing | +| [`customer-support-llm/`](customer-support-llm/) | Global content-safety | `global.v1.toxicity` | passing | +| [`education-proctoring/`](education-proctoring/) | Education (FERPA-aligned) | `industry_specific.education.v1.assessment_and_evaluation.responsible_ai_proctoring` | passing | + +Each `run.sh` is a one-liner you can copy into CI. To see a non-compliant verdict, edit `input.json` (e.g., lower a completeness score below `0.7`, flip a governance boolean to `false`) and run again. + +## Want a new example? + +Open an issue with the framework and use case, or send a PR following the layout above. Examples should be: + +- self-contained (no external services) +- runnable with `opa eval` only +- minimal — one policy, one input, one verdict diff --git a/examples/customer-support-llm/README.md b/examples/customer-support-llm/README.md new file mode 100644 index 0000000..f96721b --- /dev/null +++ b/examples/customer-support-llm/README.md @@ -0,0 +1,75 @@ +# Example — Customer-support LLM (content safety) + +Evaluates a customer-support chatbot's output toxicity against the [`global/v1/toxicity`](../../global/v1/toxicity) policy. + +This is the kind of check you'd run on **every LLM application** as a sanity floor — independent of which jurisdiction's framework you're certifying against. EU AI Act, NIST AI RMF, FTC consumer-protection enforcement, and most enterprise responsible-AI policies all assume your application is not emitting toxic output to customers. + +## Run + +```bash +./run.sh +``` + +Expected verdict: + +```json +{ + "policy": "Global Toxicity Policy", + "version": "1.0.0", + "overall_result": true, + "details": { + "toxicity_score": 0.04, + "threshold": 0.1, + "compliant": true + }, + "recommendations": [] +} +``` + +## What the policy checks + +| Field | Required | +|---|---| +| `evaluation.toxicity_score` | `< params.toxicity_threshold` | +| `params.toxicity_threshold` | default `0.1` if not provided | + +The policy is **parameterized** — your team picks the threshold based on use case. Internal-only tools may tolerate a higher threshold than customer-facing surfaces. + +## See it fail + +Raise `evaluation.toxicity_score` to `0.15` in [`input.json`](input.json) and re-run: + +```json +{ + "overall_result": false, + "details": { + "toxicity_score": 0.15, + "threshold": 0.1, + "compliant": false + }, + "recommendations": [ + "Reduce toxicity in AI responses by implementing additional content filtering" + ] +} +``` + +## Plug it into your eval pipeline + +```bash +# After every offline eval run, gate the deploy on the toxicity verdict: +opa eval -d gopal/global/v1/toxicity \ + --input eval-results.json \ + --fail-defined \ + "data.global.v1.toxicity.deny" +``` + +The build fails the moment `toxicity_score >= toxicity_threshold`. + +## Where this fits + +| Layer | Tool | +|---|---| +| Generate the score | Perspective API, OpenAI moderation, internal classifiers | +| Aggregate per release | Your eval harness — write `evaluation.toxicity_score` to `input.json` | +| Decide pass/fail | `global.v1.toxicity` (this policy) | +| Report to stakeholders | [AICertify](https://github.com/Principled-Evolution/aicertify) renders a PDF report on top | diff --git a/examples/customer-support-llm/expected-output.json b/examples/customer-support-llm/expected-output.json new file mode 100644 index 0000000..54f88a5 --- /dev/null +++ b/examples/customer-support-llm/expected-output.json @@ -0,0 +1,11 @@ +{ + "policy": "Global Toxicity Policy", + "version": "1.0.0", + "overall_result": true, + "details": { + "toxicity_score": 0.04, + "threshold": 0.1, + "compliant": true + }, + "recommendations": [] +} diff --git a/examples/customer-support-llm/input.json b/examples/customer-support-llm/input.json new file mode 100644 index 0000000..8b6aa2c --- /dev/null +++ b/examples/customer-support-llm/input.json @@ -0,0 +1,16 @@ +{ + "system": { + "name": "acme-support-bot", + "use_case": "Tier-1 customer support", + "model": "gpt-4o-2024-08-06" + }, + "evaluation": { + "toxicity_score": 0.04, + "sample_size": 10000, + "method": "Perspective API + human review on 1% sample", + "evaluation_date": "2026-05-01" + }, + "params": { + "toxicity_threshold": 0.1 + } +} diff --git a/examples/customer-support-llm/run.sh b/examples/customer-support-llm/run.sh new file mode 100755 index 0000000..c384a46 --- /dev/null +++ b/examples/customer-support-llm/run.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Evaluate input.json against the global toxicity policy. +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$HERE/../.." && pwd)" + +opa eval \ + -d "$REPO_ROOT/global/v1/toxicity" \ + --input "$HERE/input.json" \ + --format pretty \ + "data.global.v1.toxicity.compliance_report" diff --git a/examples/education-proctoring/README.md b/examples/education-proctoring/README.md new file mode 100644 index 0000000..dda9fe4 --- /dev/null +++ b/examples/education-proctoring/README.md @@ -0,0 +1,79 @@ +# Example — Education AI proctoring + +Evaluates a remote-proctoring AI system against the [`responsible_ai_proctoring`](../../industry_specific/education/v1/assessment_and_evaluation/responsible_ai_proctoring.rego) policy. The policy enforces a minimum due-process and privacy baseline that's broadly aligned with FERPA, COPPA, ETS guidance, and state-level AI-proctoring rules. + +## Run + +```bash +./run.sh +``` + +Expected verdict: + +```json +{ + "compliant": true, + "deny_reasons": [] +} +``` + +## What the policy checks + +The session is **compliant** only when **all** of the following are true: + +| Check | Field | Required | +|---|---|---| +| Student consent | `proctoring_session.student_consent_given` | `true` | +| Data encrypted | `proctoring_session.data_handling.encryption_enabled` | `true` | +| Bounded retention | `proctoring_session.data_handling.data_retention_period_days` | `<= 30` | +| Human review of every flag | `proctoring_session.review_process.human_review_required_for_all_flags` | `true` | +| Student appeal possible | `proctoring_session.review_process.student_appeal_possible` | `true` | + +## See it fail + +Flip any of those fields and re-run `./run.sh`. The policy returns a `deny_reasons` array citing which check failed: + +```json +{ + "compliant": false, + "deny_reasons": [ + "AI proctoring session is not compliant. Failures: {\"Lack of human review or appeals process\"}" + ] +} +``` + +The deny messages compose, so flipping multiple fields at once produces a multi-failure verdict. + +## Why this matters + +AI proctoring is one of the highest-scrutiny consumer-facing AI use cases. The risks are concrete: + +- **Privacy** — webcam + microphone + screen capture is highly sensitive +- **Due process** — a flagged student needs a path to challenge the AI's decision +- **Bias** — facial-detection failure rates differ across demographics +- **Consent** — implicit consent in an enrollment agreement is not the same as informed consent for AI surveillance + +This policy encodes the minimum signal a university or ed-tech vendor's compliance team needs to demonstrate. It's the floor, not the ceiling — most institutions will want stricter retention windows and broader bias evaluations on top. + +## Plug it into your eval pipeline + +```yaml +# .github/workflows/proctoring-compliance.yaml +- name: AI proctoring compliance check + run: | + opa eval \ + -d gopal/industry_specific/education/v1/assessment_and_evaluation \ + --input proctoring-session.json \ + --fail-defined \ + "data.industry_specific.education.v1.assessment_and_evaluation.responsible_ai_proctoring_compliant == false" +``` + +The build fails if any session in your pipeline lacks consent, leaks encryption, retains too long, skips human review, or lacks an appeals process. + +## Related policies + +| Concern | Policy | +|---|---| +| Student data privacy (FERPA / COPPA) | [`industry_specific/education/v1/student_data_privacy/`](../../industry_specific/education/v1/student_data_privacy) | +| Academic integrity | [`industry_specific/education/v1/academic_integrity/`](../../industry_specific/education/v1/academic_integrity) | +| Fair and equitable treatment | [`industry_specific/education/v1/fairness_and_equity/`](../../industry_specific/education/v1/fairness_and_equity) | diff --git a/examples/education-proctoring/expected-output.json b/examples/education-proctoring/expected-output.json new file mode 100644 index 0000000..9eeeba1 --- /dev/null +++ b/examples/education-proctoring/expected-output.json @@ -0,0 +1,4 @@ +{ + "compliant": true, + "deny_reasons": [] +} diff --git a/examples/education-proctoring/input.json b/examples/education-proctoring/input.json new file mode 100644 index 0000000..55e8f27 --- /dev/null +++ b/examples/education-proctoring/input.json @@ -0,0 +1,29 @@ +{ + "system": { + "name": "exam-proctor-ai", + "vendor": "Acme EdTech", + "use_case": "Remote-proctoring AI for university final exams" + }, + "proctoring_session": { + "session_id": "exam-2026-spring-final-cs101-student-42", + "student_consent_given": true, + "consent_artifacts": { + "consent_form_url": "https://example.edu/proctoring/consent-2026.pdf", + "consent_timestamp": "2026-04-30T18:00:00Z", + "opt_out_available": true, + "accommodations_offered": ["non-AI proctor on request", "extended time"] + }, + "data_handling": { + "encryption_enabled": true, + "encryption_method": "AES-256 at rest, TLS 1.3 in transit", + "data_retention_period_days": 14, + "deletion_verified": true + }, + "review_process": { + "human_review_required_for_all_flags": true, + "student_appeal_possible": true, + "appeal_deadline_days": 30, + "appeal_outcomes_tracked": true + } + } +} diff --git a/examples/education-proctoring/run.sh b/examples/education-proctoring/run.sh new file mode 100755 index 0000000..669d114 --- /dev/null +++ b/examples/education-proctoring/run.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# Evaluate input.json against the education responsible AI proctoring policy. +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$HERE/../.." && pwd)" + +opa eval \ + -d "$REPO_ROOT/industry_specific/education/v1/assessment_and_evaluation" \ + --input "$HERE/input.json" \ + --format pretty \ + "{ + \"compliant\": data.industry_specific.education.v1.assessment_and_evaluation.responsible_ai_proctoring_compliant, + \"deny_reasons\": data.industry_specific.education.v1.assessment_and_evaluation.deny + }" diff --git a/examples/eu-ai-act-transparency/README.md b/examples/eu-ai-act-transparency/README.md new file mode 100644 index 0000000..3931c5c --- /dev/null +++ b/examples/eu-ai-act-transparency/README.md @@ -0,0 +1,64 @@ +# Example — EU AI Act transparency + +Evaluates an AI system's documentation and toxicity metrics against the EU AI Act transparency policy at [`international/eu_ai_act/v1/transparency`](../../international/eu_ai_act/v1/transparency). + +The policy implements obligations from: + +- **Article 13** — Transparency and provision of information to users +- **Article 52** — Transparency obligations for certain AI systems + +## Run + +```bash +./run.sh +``` + +Expected verdict: + +```json +{ + "compliant": true, + "policy_name": "EU AI Act Transparency Requirements", + "reason": "The system meets EU AI Act transparency requirements with sufficient documentation and low toxicity levels", + "recommendations": [] +} +``` + +## What the policy checks + +The system is **compliant** when **all** of the following are true: + +| Check | Field | Required value | +|---|---|---| +| Technical documentation is reasonably complete | `documentation.technical_documentation.completeness` | `>= 0.7` | +| Model card is reasonably complete | `documentation.model_card.completeness` | `>= 0.7` | +| Explainability information is reasonably complete | `documentation.explainability.completeness` | `>= 0.7` | +| Outputs are not highly toxic | `metrics.toxicity.max_toxicity` | `<= 0.7` | + +## See it fail + +Edit [`input.json`](input.json) and lower any `completeness` below `0.7`, or raise `max_toxicity` above `0.7`. Re-run `./run.sh`: + +```json +{ + "compliant": false, + "reason": "The system's documentation is not sufficiently complete to meet EU AI Act transparency requirements", + "recommendations": [ + "Improve the completeness of technical documentation, model cards, and explainability information" + ] +} +``` + +## Plug it into CI + +```yaml +# .github/workflows/ai-compliance.yaml +- name: EU AI Act transparency check + run: | + opa eval -d gopal/international/eu_ai_act/v1/transparency \ + --input my-system.json \ + --fail-defined \ + "data.international.eu_ai_act.v1.transparency.compliance_report.compliant == false" +``` + +The build fails if the policy returns `compliant: false`. diff --git a/examples/eu-ai-act-transparency/expected-output.json b/examples/eu-ai-act-transparency/expected-output.json new file mode 100644 index 0000000..c05b0d9 --- /dev/null +++ b/examples/eu-ai-act-transparency/expected-output.json @@ -0,0 +1,6 @@ +{ + "compliant": true, + "policy_name": "EU AI Act Transparency Requirements", + "reason": "The system meets EU AI Act transparency requirements withsufficient documentation and low toxicity levels", + "recommendations": [] +} diff --git a/examples/eu-ai-act-transparency/input.json b/examples/eu-ai-act-transparency/input.json new file mode 100644 index 0000000..e086a94 --- /dev/null +++ b/examples/eu-ai-act-transparency/input.json @@ -0,0 +1,28 @@ +{ + "system": { + "name": "customer-support-llm-v3", + "provider": "Acme Corp", + "deployment": "production" + }, + "documentation": { + "technical_documentation": { + "completeness": 0.85, + "last_updated": "2026-04-12" + }, + "model_card": { + "completeness": 0.80, + "url": "https://example.com/model-cards/customer-support-llm-v3" + }, + "explainability": { + "completeness": 0.75, + "method": "SHAP + chain-of-thought logging" + } + }, + "metrics": { + "toxicity": { + "max_toxicity": 0.12, + "mean_toxicity": 0.03, + "sample_size": 5000 + } + } +} diff --git a/examples/eu-ai-act-transparency/run.sh b/examples/eu-ai-act-transparency/run.sh new file mode 100755 index 0000000..387eef3 --- /dev/null +++ b/examples/eu-ai-act-transparency/run.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Evaluate input.json against the EU AI Act transparency policy. +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$HERE/../.." && pwd)" + +opa eval \ + -d "$REPO_ROOT/international/eu_ai_act/v1/transparency" \ + --input "$HERE/input.json" \ + --format pretty \ + "data.international.eu_ai_act.v1.transparency.compliance_report" diff --git a/examples/nist-ai-rmf-govern/README.md b/examples/nist-ai-rmf-govern/README.md new file mode 100644 index 0000000..0ffc7f5 --- /dev/null +++ b/examples/nist-ai-rmf-govern/README.md @@ -0,0 +1,56 @@ +# Example — NIST AI RMF (Govern function) + +Evaluates an AI system's governance posture against the **Govern** function of the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework). + +Policy: [`international/nist/v1/govern`](../../international/nist/v1/govern). + +## Run + +```bash +./run.sh +``` + +Expected verdict — `allow: true` with all three sub-functions passing: + +```json +{ + "allow": true, + "accountability": { "allow": true, "msg": "Accountability requirements met." }, + "transparency": { "allow": true, "msg": "Transparency requirements met." }, + "fairness": { "allow": true, "msg": "Fairness requirements met." } +} +``` + +## What the policy checks + +The Govern function in this policy composes three checks. **All three must pass**: + +### Accountability +| Field | Required | +|---|---| +| `governance.roles_and_responsibilities_defined` | `true` | +| `governance.oversight_mechanisms_in_place` | `true` | + +### Transparency +| Field | Required | +|---|---| +| `transparency.public_documentation_available` | `true` | +| `transparency.decision_explanations_provided` | `true` | + +### Fairness +| Field | Required | +|---|---| +| `fairness.bias_assessments_conducted` | `true` | +| `fairness.bias_mitigation_strategies_in_place` | `true` | + +## See it fail + +Flip any of those booleans to `false` in [`input.json`](input.json) and re-run. The corresponding sub-function will return `{"allow": false, "msg": "... requirements not met."}` and the top-level `allow` will be `false`. + +## Why this matters + +The NIST AI RMF is the de-facto AI risk baseline for US federal procurement and increasingly for state and enterprise programs. The Govern function is the foundation — without it, the Map / Measure / Manage functions cannot be relied on. + +This example shows the minimum signal a governance team needs to encode: documented roles, documented oversight, documented transparency commitments, and documented bias-mitigation activities. + +For Map / Measure / Manage functions, see [`international/nist/v1/`](../../international/nist/v1/). diff --git a/examples/nist-ai-rmf-govern/expected-output.json b/examples/nist-ai-rmf-govern/expected-output.json new file mode 100644 index 0000000..dfb6bc7 --- /dev/null +++ b/examples/nist-ai-rmf-govern/expected-output.json @@ -0,0 +1,24 @@ +{ + "allow": true, + "accountability": { + "allow": true, + "msg": "Accountability requirements met." + }, + "transparency": { + "allow": true, + "msg": "Transparency requirements met." + }, + "fairness": { + "allow": true, + "msg": "Fairness requirements met." + }, + "metadata": { + "title": "NIST AI RMF - Govern", + "description": "Policies for the Govern function of the NIST AI Risk Management Framework.", + "version": "1.0.0", + "category": "NIST AI RMF", + "references": [ + "NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework" + ] + } +} diff --git a/examples/nist-ai-rmf-govern/input.json b/examples/nist-ai-rmf-govern/input.json new file mode 100644 index 0000000..02ab62b --- /dev/null +++ b/examples/nist-ai-rmf-govern/input.json @@ -0,0 +1,25 @@ +{ + "system": { + "name": "loan-approval-model-v2", + "provider": "Acme Bank", + "use_case": "consumer credit decisioning" + }, + "governance": { + "roles_and_responsibilities_defined": true, + "oversight_mechanisms_in_place": true, + "policy_documents": [ + "model-risk-policy-v3.pdf", + "ai-ethics-charter.pdf" + ] + }, + "transparency": { + "public_documentation_available": true, + "decision_explanations_provided": true, + "explanation_method": "adverse-action codes per Reg B + SHAP feature attribution" + }, + "fairness": { + "bias_assessments_conducted": true, + "bias_mitigation_strategies_in_place": true, + "protected_attributes_evaluated": ["race", "gender", "age", "national_origin"] + } +} diff --git a/examples/nist-ai-rmf-govern/run.sh b/examples/nist-ai-rmf-govern/run.sh new file mode 100755 index 0000000..a583622 --- /dev/null +++ b/examples/nist-ai-rmf-govern/run.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Evaluate input.json against the NIST AI RMF Govern function. +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$HERE/../.." && pwd)" + +opa eval \ + -d "$REPO_ROOT/international/nist/v1/govern" \ + --input "$HERE/input.json" \ + --format pretty \ + "data.international.nist.v1.govern"