diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..1d4c074 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,87 @@ +# Agent Instructions — AICertify + +This file is the canonical operational guide for AI coding agents working in this repository (Claude Code, Cursor, Codex, Windsurf, Gemini CLI, Copilot, etc.). Tool-specific files (`CLAUDE.md`, `GEMINI.md`) inherit from this and add only platform-specific notes. + +## What this project is + +**AICertify** is a Python framework that evaluates AI applications against regulatory frameworks using [Open Policy Agent (OPA)](https://www.openpolicyagent.org/) policies sourced from the sister project [gopal](https://github.com/Principled-Evolution/gopal). It produces audit-ready compliance reports in PDF, Markdown, JSON, and HTML. + +The user-facing surface is: + +1. **Python API** — `aicertify.application`, `aicertify.regulations`, evaluator classes +2. **CLI** — `python -m aicertify.cli` +3. **Reports** — generated artifacts in `examples/outputs/` you can show to an auditor + +## Repository layout + +``` +aicertify/ Python package +├── __init__.py Public API (re-exports the surface a user sees) +├── cli.py Argparse CLI entry — see "Useful commands" below +├── application.py / regulations.py User-facing fluent API +├── api.py / contract_models.py Contract data model (Pydantic) +├── evaluators/ Pluggable evaluators (Fairness, ContentSafety, …) +├── opa_policies/ Vendored Rego policy tree (mirrors gopal layout) +│ ├── global/v1/ Cross-cutting categories +│ ├── international/ EU AI Act, NIST, India, … +│ ├── industry_specific/ Healthcare, BFS, Automotive, Aviation, … +│ └── operational/ AIOps, cost, corporate +├── report_generation/ ReportLab-based PDF + Markdown + HTML + JSON writers +└── assets/ Logos, images +examples/ +├── quickstart.py End-to-end demo — the canonical "does this work?" script +├── sample_contract.json Contract structure reference +└── outputs/ Pre-generated reports for inspection +tests/ pytest tests +pyproject.toml Poetry-managed project metadata +``` + +## Useful commands + +```bash +# Install (editable) +pip install -e . + +# Run the end-to-end demo (writes reports/ into the repo) +python examples/quickstart.py + +# CLI evaluation +python -m aicertify.cli \ + --contract examples/sample_contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf \ + --output-dir reports/ + +# Tests +pytest tests/ -v + +# Lint / format +ruff check aicertify/ +black aicertify/ +``` + +## Conventions + +- **Python 3.12 only** (see `requires-python` in `pyproject.toml`). Do not introduce syntax incompatible with 3.12 — do not assume 3.13. +- **Line length 88** (black + ruff default). Don't change this. +- **Pydantic v2** — when defining models, use `model_config` not `class Config`. +- **OPA policies** — never hand-edit `aicertify/opa_policies/*.rego` files in this repo without considering whether the change should land upstream in [gopal](https://github.com/Principled-Evolution/gopal) first. They are a vendored copy. +- **Reports are user-facing legal artifacts** — when changing report generation, run the quickstart and inspect the output PDF before claiming success. +- **Avoid breaking the public API** — anything re-exported from `aicertify/__init__.py` is part of the user-facing contract. Bump the version and note in CHANGELOG.md if you must. + +## What NOT to do + +- Don't pin new dependencies aggressively — many users will install AICertify alongside their own stack; tight pins create conflicts. +- Don't replace OPA with another policy engine. The whole value proposition is policy-as-code in Rego. +- Don't commit `reports/` or `examples/outputs/` content unless deliberately updating the canonical samples. +- Don't introduce calls to closed-source SaaS services (Credo AI, etc.) — this is a self-hostable, on-prem-capable framework. +- Don't add a feature that requires `python >= 3.13` — the project is intentionally pinned to 3.12. + +## Sister projects + +- **[gopal](https://github.com/Principled-Evolution/gopal)** — the upstream Rego policy library. If you find yourself adding a new framework, contribute the `.rego` there, then vendor it here. +- **[Open Policy Agent](https://www.openpolicyagent.org/)** — the engine. + +## Conservatism + +The author prefers **surgical changes**: do only what was asked, present the plan first when there's any ambiguity, and ask before introducing new abstractions. Critique your own design once for elegance, DRY, KISS, and explainability before presenting it. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..2317e2f --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,43 @@ +# Changelog + +All notable changes to **AICertify** are documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added +- Centered HTML hero, ordered badge wall, value-prop tagline, and 5 programmatically-generated marketing diagrams in the README (regulatory coverage, architecture, comparison vs Fairlearn / AIF360 / MS RAI / Credo AI, report anatomy, end-to-end flow). +- `AGENTS.md` and `CLAUDE.md` — operational instructions for AI coding agents working in this repository. +- `skills/` directory with 4 Claude Code skills: `run-compliance-check`, `evaluate-contract`, `explain-regulation`, `draft-policy`. Each ships as a slash command once installed into `~/.claude/skills/`. +- Comparison table vs Fairlearn / IBM AI Fairness 360 / Microsoft RAI Toolbox / Credo AI in the README. +- `diagrams/generate_diagrams.py` — reproducible matplotlib script that regenerates every marketing PNG. + +### Changed +- README rewritten for product-page clarity: value-prop first, then quickstart, then differentiation, then coverage. + +## [0.7.0] — 2025-04 + +### Added +- Reporting subsystem (`aicertify.report_generation`) producing audit-ready artifacts in PDF (via ReportLab), Markdown, JSON, and HTML. +- Quickstart example (`examples/quickstart.py`) wiring sample interactions through the EU AI Act policy set and emitting a full report. +- Pluggable evaluator classes — `FairnessEvaluator`, `ContentSafetyEvaluator`, `RiskManagementEvaluator`, `ComplianceEvaluator`. +- Sample pre-generated reports under `examples/outputs/` for EU AI Act, loan evaluation, and medical diagnosis use cases. + +### Changed +- OPA policies migrated to the standalone [gopal](https://github.com/Principled-Evolution/gopal) library; AICertify now vendors the policy tree under `aicertify/opa_policies/` via Git submodule. +- Enhanced logging across the evaluation pipeline. +- Pre-commit hooks: `ruff`, `black`, security checks. + +### Fixed +- Security: bumped `protobuf` to 5.29.5 and `pycares` to 4.9.0 to resolve advisory exposure. +- Security: bumped `transformers` and `setuptools` to resolve security alerts. +- Auto-labeling workflow no longer produces excessive labels. + +## Earlier history + +For changes prior to 0.7.0, see the [Git log](https://github.com/Principled-Evolution/aicertify/commits/main). + +[Unreleased]: https://github.com/Principled-Evolution/aicertify/compare/v0.7.0...HEAD +[0.7.0]: https://github.com/Principled-Evolution/aicertify/releases/tag/v0.7.0 diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..90f649c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,51 @@ +# Claude Code — Project Context for AICertify + +> **Read [AGENTS.md](AGENTS.md) first.** This file inherits all of those instructions and only adds Claude Code-specific notes. + +## What you're working on + +**AICertify** evaluates AI applications against regulatory frameworks (EU AI Act, NIST AI RMF, +13 more) using [OPA](https://www.openpolicyagent.org/) policies from [gopal](https://github.com/Principled-Evolution/gopal). The deliverable is an audit-ready compliance report (PDF / Markdown / JSON / HTML). + +## Fast orientation + +When the user asks a question about the codebase, in order: + +1. **For "how does the eval flow work?"** → read [examples/quickstart.py](examples/quickstart.py). It's the canonical user path. +2. **For "what's covered by regulation X?"** → look in `aicertify/opa_policies///v1/`. +3. **For "what does a report look like?"** → open one in `examples/outputs/`. +4. **For "what's the public Python API?"** → read `aicertify/__init__.py` — it's the surface contract. + +## Conventions Claude Code should respect + +- **Use TodoWrite** for multi-step tasks (this is the user's preferred tracking). +- **Run the quickstart** before claiming a feature works end-to-end. Type-check and tests verify code correctness; the quickstart verifies *user-facing* correctness. +- **When editing OPA policies in `aicertify/opa_policies/`**, remind the user that these are vendored from [gopal](https://github.com/Principled-Evolution/gopal) and the upstream copy should usually be edited first. +- **Reports** — when changing report templates, open the generated PDF (`examples/outputs/eu_ai_act/`) and inspect it visually. Don't assume rendering correctness from the code alone. + +## Useful skills + +This project ships Claude Code skills in [`skills/`](skills/). To register them, copy into your Claude Code skills directory: + +```bash +mkdir -p ~/.claude/skills && cp -r skills/* ~/.claude/skills/ +``` + +Available slash commands once registered: + +- `/run-compliance-check` — Run the end-to-end quickstart and surface the generated report +- `/evaluate-contract` — Evaluate a user-supplied contract JSON against any supported framework +- `/explain-regulation` — Walk every policy in a framework directory and explain what it checks +- `/draft-policy` — Scaffold a new Rego policy with metadata, default rule, and a test sibling + +See [`skills/README.md`](skills/README.md) for the full list and conventions. + +## What NOT to do in this repo + +- Don't run `python examples/quickstart.py` repeatedly without cleaning `reports/` — the directory accumulates artifacts. +- Don't add Python ≥3.13 syntax (project pins to 3.12). +- Don't switch package manager — poetry is canonical, `pip install -e .` works for users. +- Don't claim MCP-compatibility yet — an MCP server is on the roadmap, not shipped. + +## Session etiquette + +The author prefers terse, surgical responses. Present the plan first when ambiguous; ask before introducing new abstractions. diff --git a/README.hi-IN.md b/README.hi-IN.md new file mode 100644 index 0000000..8509214 --- /dev/null +++ b/README.hi-IN.md @@ -0,0 +1,242 @@ +
+ AICertify +
+ +

AICertify

+ +

+ English | + 简体中文 | + 日本語 | + 한국어 | + हिन्दी +

+ +

+ AI सिस्टम्स के लिए Compliance-as-code। +

+ +

+ अपने AI का ऑडिट EU AI Act, NIST AI RMF, और 13 और फ्रेमवर्क्स के विरुद्ध करें — एक कॉन्ट्रैक्ट, एक कमांड, एक रिपोर्ट। +

+ +

+ CI + Stars + Version 0.7.0 + Python 3.12+ + Apache 2.0 + Built on OPA + 94 Rego Policies + Beta + PRs Welcome +

+ +

+ AI ऐप से ऑडिट-तैयार रिपोर्ट तक: AI Application -> AICertify Contract -> OPA Policy Evaluation -> Compliance Report +

+ +
+ +रेगुलेटर्स आपके गवर्नेंस डॉक्यूमेंट्स से तेज़ी से आगे बढ़ रहे हैं। EU AI Act लागू हो चुका है। NIST AI RMF अमेरिका का डी-फैक्टो स्टैंडर्ड है। भारत, ब्राज़ील, और सिंगापुर अगले हैं। `AICertify` आपको इन दायित्वों को निष्पादन योग्य [Open Policy Agent](https://www.openpolicyagent.org/) पॉलिसीज़ के रूप में एनकोड करने, कैप्चर की गई AI इंटरैक्शन्स के विरुद्ध चलाने, और PDF, Markdown, JSON, या HTML में ऑडिट-तैयार रिपोर्ट्स तैयार करने की सुविधा देता है। + +यह *"हमारे पास एक responsible-AI पॉलिसी है"* और *"हम इसे सिद्ध कर सकते हैं"* के बीच की लुप्त कड़ी है। + +--- + +## Quick Start + +```bash +git clone https://github.com/Principled-Evolution/aicertify.git +cd aicertify +pip install -e . +python examples/quickstart.py +``` + +यह क्विकस्टार्ट एक सैंपल AI एप्लिकेशन को EU AI Act पॉलिसी सेट के माध्यम से जोड़ता है और `reports/` में एक कंप्लायंस रिपोर्ट लिखता है। उसे खोलिए। यही आपके ऑडिट डिलिवरेबल का स्वरूप है — हाथ से नहीं, जनरेट होकर। + +### न्यूनतम Python उपयोग + +```python +from aicertify import regulations, application + +# 1. Pick the regulations you want to certify against +regs = regulations.create("my_regulations") +regs.add("eu_ai_act") + +# 2. Wrap your AI app +app = application.create( + name="customer-support-bot", + model_name="gpt-4o", + model_version="2024-08-06", +) + +# 3. Feed it real interactions +app.add_interaction( + input_text="I want a refund for my order", + output_text="I can help with that. Could you share your order number?", +) + +# 4. Evaluate and get reports back +await app.evaluate(regulations=regs, report_format="pdf", output_dir="reports") +``` + +यही पूरा लूप है। **Contract → interactions → evaluate → report.** + +--- + +## AICertify क्यों + +अधिकांश AI-गवर्नेंस टूलिंग या तो: + +- **एक वेंडर SaaS** है जो आपके ऑडिट ट्रेल को लॉगिन के पीछे बंद रखता है (Credo AI, Holistic AI), या +- **एक रिसर्च टूलकिट** है जो एक ही आयाम पर केंद्रित है — फेयरनेस मेट्रिक्स (Fairlearn, AI Fairness 360) या व्याख्यात्मकता (Microsoft RAI Toolbox)। + +दोनों में से कोई भी वह डॉक्यूमेंट तैयार नहीं करता जिसकी रेगुलेटर वास्तव में मांग करता है: *प्रमाण कि आपने इस AI सिस्टम का परीक्षण एक नामित विनियमन के विरुद्ध किया है, पुनरुत्पादनीय पॉलिसीज़ और दिनांकित रिपोर्ट के साथ।* + +AICertify उसी आर्टिफैक्ट के लिए बनाया गया है। + +

+ AICertify बनाम विकल्प: AICertify एकमात्र ओपन-सोर्स, policy-as-code विकल्प है जिसमें नामित रेगुलेटरी फ्रेमवर्क्स, इंडस्ट्री वर्टिकल्स, और बॉक्स से बाहर ऑडिट-तैयार रिपोर्ट्स हैं +

+ +| | AICertify | Fairlearn / AIF360 | MS RAI Toolbox | Credo AI | +|---|---|---|---|---| +| ओपन सोर्स | ✅ Apache 2.0 | ✅ MIT | ✅ MIT | ❌ क्लोज़्ड | +| On-prem / air-gapped | ✅ | ✅ | ✅ | ❌ | +| नामित रेगुलेटरी फ्रेमवर्क्स | **EU AI Act, NIST RMF, Brazil AI Bill, India DPDP, +11 और** | ❌ (केवल फेयरनेस) | ❌ (टूलकिट) | ✅ | +| Policy-as-code (ऑडिटेबल, diff-able) | ✅ OPA / Rego | ❌ | ❌ | ❌ | +| बॉक्स से बाहर इंडस्ट्री वर्टिकल्स | Aviation, Banking, Healthcare, Automotive, Education | ❌ | ❌ | आंशिक | +| ऑडिट-तैयार रिपोर्ट्स जनरेट करता है | ✅ PDF / MD / JSON / HTML | ❌ | आंशिक | ✅ | +| कस्टम पॉलिसीज़ | ✅ एक `.rego` फ़ाइल ड्रॉप करें | ❌ | N/A | ✅ (पेड) | + +--- + +## यह कैसे काम करता है + +

+ AICertify आर्किटेक्चर: आपका AI ऐप एक Contract को फीड करता है, जो Evaluators (Fairness, ContentSafety, RiskManagement, Compliance) के माध्यम से 94 Rego पॉलिसीज़ वाले OPA Engine में जाता है, और Report Generator के द्वारा एक ऑडिट डिलिवरेबल तैयार करता है +

+ +1. **Contract** — आपके AI एप्लिकेशन का एक JSON विवरण: model, version, कैप्चर की गई interactions, metadata। +2. **Evaluators** — प्लग करने योग्य Python evaluators (Fairness, ContentSafety, RiskManagement, Compliance) आपकी interactions से मेट्रिक्स निकालते हैं। +3. **OPA policies** — मेट्रिक्स का मूल्यांकन विनियमन की Rego पॉलिसीज़ ([gopal](https://github.com/Principled-Evolution/gopal) पॉलिसी लाइब्रेरी से प्राप्त) के विरुद्ध किया जाता है। +4. **Report** — एक फॉर्मेटेड, दिनांकित आर्टिफैक्ट जिसे आप कानूनी टीम, ऑडिटर, या अपनी AI रिस्क कमेटी को सौंप सकते हैं। + +चूंकि पॉलिसीज़ डिक्लेरेटिव Rego हैं, वे किसी भी अन्य कोड की तरह वर्ज़न, diff, और रिव्यू होती हैं। जब कोई विनियमन बदलता है, तो आप पॉलिसी अपडेट करते हैं — अपनी मूल्यांकन हार्नेस नहीं। + +--- + +## रेगुलेटरी कवरेज + +

+ रेगुलेटरी कवरेज: 15+ फ्रेमवर्क्स और 5 इंडस्ट्रीज़ में 94 पॉलिसीज़ -- EU AI Act, NIST AI RMF, India DPDP, Brazil AI Bill, RTCA DO-365/366, FAA Part 107, EASA SORA, ICAO Doc 10019, Healthcare, Banking and Financial Services, Automotive, Education, Global, Aviation, AIOps, Corporate +

+ +AICertify [gopal](https://github.com/Principled-Evolution/gopal) पॉलिसी लाइब्रेरी के विरुद्ध चलता है — इन फ्रेमवर्क्स में **94 प्रोडक्शन OPA पॉलिसीज़**: + +### अंतर्राष्ट्रीय +- **EU AI Act** — 29 पॉलिसीज़ जो निषिद्ध प्रथाओं, बायोमेट्रिक ID, मैनिपुलेशन, पारदर्शिता, तकनीकी डॉक्यूमेंटेशन, मानवीय निगरानी, GPAI दायित्वों को कवर करती हैं +- **NIST AI RMF** — Govern, Map, Measure, Manage + AI 600-1 +- **India Digital Policy** — DPDP-aligned दायित्व +- **Brazil AI Governance Bill** — एल्गोरिदमिक गवर्नेंस आवश्यकताएँ +- **एविएशन स्टैंडर्ड्स** — ICAO Doc 10019, RTCA DO-365/366, ASTM F3442, ISO 21384, FAA Part 107, EASA SORA + +### इंडस्ट्री-विशिष्ट +- **Aviation** (17 पॉलिसीज़) — Detect-and-avoid, certification, design, इंटीग्रेशन वैलिडेशन +- **Education** (12 पॉलिसीज़) — FERPA, COPPA, प्रॉक्टरिंग, human-in-the-loop ग्रेडिंग +- **Banking & Financial Services** — मॉडल रिस्क, fair lending +- **Healthcare** — पेशेंट सेफ्टी, डायग्नोस्टिक सेफ्टी +- **Automotive** — व्हीकल सेफ्टी इंटीग्रेशन + +### Global & Operational +- **Global** — जवाबदेही, फेयरनेस, पारदर्शिता, व्याख्यात्मकता, कंटेंट सेफ्टी, रिस्क मैनेजमेंट, सिक्योरिटी +- **Corporate** — InfoSec, गवर्नेंस +- **AIOps & Cost** — स्केलेबिलिटी, संसाधन दक्षता + +अपना विनियमन यहाँ नहीं देखा? [एक Rego फ़ाइल जोड़ें](https://github.com/Principled-Evolution/gopal/blob/main/CONTRIBUTING.md)। लाइब्रेरी विस्तार के लिए डिज़ाइन की गई है। + +--- + +## CLI + +```bash +python -m aicertify.cli \ + --contract path/to/contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf \ + --output-dir reports/ +``` + +उपयोगी फ़्लैग्स: + +| Flag | उद्देश्य | +|---|---| +| `--contract` | AI एप्लिकेशन कॉन्ट्रैक्ट JSON का पथ | +| `--policy` | जिसके विरुद्ध मूल्यांकन करना है उस OPA पॉलिसी फ़ोल्डर का पथ | +| `--report-format` | `pdf`, `markdown`, `json`, `html` (डिफ़ॉल्ट: `pdf`) | +| `--evaluators` | विशिष्ट evaluators तक सीमित करें (जैसे `Fairness ContentSafety`) | +| `--output-dir` | जहाँ रिपोर्ट्स लैंड होती हैं (डिफ़ॉल्ट: `./reports`) | +| `--verbose` | वर्बोज़ लॉगिंग | + +पूर्ण Python API के लिए [`examples/quickstart.py`](examples/quickstart.py) देखें। + +--- + +## सैंपल रिपोर्ट्स + +

+ एक ऑडिट-तैयार रिपोर्ट की संरचना: फ्रेमवर्क नाम, एप्लिकेशन, मॉडल और दिनांक के साथ हेडर; executive summary; policy results table; risk assessment bar chart; remediation guidance; AICertify v0.7.0 का श्रेय देने वाला फुटर +

+ +`examples/outputs/` डायरेक्टरी में वास्तविक मूल्यांकनों से जनरेट की गई रिपोर्ट्स हैं जिन्हें आप कुछ भी चलाने से पहले देख सकते हैं: + +- `eu_ai_act/` — EU AI Act के विरुद्ध मूल्यांकन किया गया एक customer-support agent +- `loan_evaluation/` — fair lending के लिए मूल्यांकन किया गया एक credit-scoring मॉडल +- `medical_diagnosis/` — पेशेंट सेफ्टी के लिए मूल्यांकन किया गया एक clinical-decision-support मॉडल + +PDFs खोलिए। यही आपका ऑडिटर चाहता है। + +--- + +## स्थिति + +AICertify **beta (v0.7.0)** में है — 1.0 रिलीज़ से पहले API विकसित हो सकता है। आज प्रोडक्शन-तैयार फ्रेमवर्क्स: + +- ✅ EU AI Act +- ✅ Global evaluators (fairness, content safety, transparency) +- ✅ Healthcare, BFS, Automotive इंडस्ट्री पॉलिसीज़ +- ✅ Aviation पॉलिसी सेट (RTCA, ASTM, FAA, EASA) +- 🚧 NIST AI RMF — आंशिक कवरेज +- 🚧 India Digital Policy — प्रारंभिक चरण + +[पॉलिसी लाइब्रेरी रोडमैप](https://github.com/Principled-Evolution/gopal) में प्रगति ट्रैक करें। + +--- + +## योगदान + +हम स्वागत करते हैं: + +- नए रेगुलेटरी फ्रेमवर्क्स (स्कोप संरेखित करने के लिए पहले एक issue खोलें) +- इंडस्ट्री-विशिष्ट पॉलिसीज़ जिन्हें आपने वास्तविक उपयोग में परखा है +- नए evaluators (fairness, safety, robustness — `aicertify/evaluators/` देखें) +- न्यूनतम पुनरुत्पादनीय कॉन्ट्रैक्ट के साथ बग रिपोर्ट्स + +[CONTRIBUTING.md](CONTRIBUTING.md) और [Code of Conduct](CODE_OF_CONDUCT.md) से शुरुआत करें। + +--- + +## संबंधित प्रोजेक्ट्स + +- **[gopal](https://github.com/Principled-Evolution/gopal)** — वह OPA पॉलिसी लाइब्रेरी जिसका AICertify उपयोग करता है। यदि आपको Python फ्रेमवर्क की आवश्यकता नहीं है तो OPA CLI के साथ स्टैंडअलोन उपयोग करें। +- **[Open Policy Agent](https://www.openpolicyagent.org/)** — पॉलिसी इंजन। +- **[Regal](https://github.com/StyraInc/regal)** — पॉलिसीज़ को साफ़ रखने के लिए उपयोग किया जाने वाला Rego linter। + +--- + +## लाइसेंस + +Apache License 2.0 — [LICENSE](LICENSE) देखें। + +

Principled Evolution द्वारा निर्मित · पॉलिसीज़ जिन्हें आप पढ़ सकते हैं, चला सकते हैं, और सिद्ध कर सकते हैं।

diff --git a/README.ja-JP.md b/README.ja-JP.md new file mode 100644 index 0000000..e273aea --- /dev/null +++ b/README.ja-JP.md @@ -0,0 +1,242 @@ +
+ AICertify +
+ +

AICertify

+ +

+ English | + 简体中文 | + 日本語 | + 한국어 | + हिन्दी +

+ +

+ AI システムのためのコンプライアンス・アズ・コード。 +

+ +

+ EU AI Act、NIST AI RMF をはじめとする 15 のフレームワークに対して、AI を監査する。契約 1 つ、コマンド 1 つ、レポート 1 つで完結します。 +

+ +

+ CI + Stars + Version 0.7.0 + Python 3.12+ + Apache 2.0 + Built on OPA + 94 Rego Policies + Beta + PRs Welcome +

+ +

+ AI アプリケーションから監査対応レポートまで: AI アプリケーション → AICertify 契約 → OPA ポリシー評価 → コンプライアンスレポート +

+ +
+ +規制当局は、社内のガバナンス文書よりも速いペースで動いています。EU AI Act はすでに発効済み、NIST AI RMF は米国における事実上の標準であり、インド、ブラジル、シンガポールが次に続きます。`AICertify` は、こうした義務を実行可能な [Open Policy Agent](https://www.openpolicyagent.org/) ポリシーとして表現し、収集した AI とのやり取りに対して評価を実行し、PDF、Markdown、JSON、HTML 形式の監査対応レポートを生成します。 + +「責任ある AI のポリシーがあります」と「それを証明できます」の間にあった、欠けていた橋渡しです。 + +--- + +## クイックスタート + +```bash +git clone https://github.com/Principled-Evolution/aicertify.git +cd aicertify +pip install -e . +python examples/quickstart.py +``` + +クイックスタートでは、サンプル AI アプリケーションを EU AI Act のポリシーセットに通し、コンプライアンスレポートを `reports/` に出力します。それを開いてみてください。手書きではなく、生成された監査成果物の実例です。 + +### 最小限の Python での使い方 + +```python +from aicertify import regulations, application + +# 1. 認証対象としたい規制を選択 +regs = regulations.create("my_regulations") +regs.add("eu_ai_act") + +# 2. AI アプリケーションをラップ +app = application.create( + name="customer-support-bot", + model_name="gpt-4o", + model_version="2024-08-06", +) + +# 3. 実際のやり取りを投入 +app.add_interaction( + input_text="I want a refund for my order", + output_text="I can help with that. Could you share your order number?", +) + +# 4. 評価を実行してレポートを取得 +await app.evaluate(regulations=regs, report_format="pdf", output_dir="reports") +``` + +これがループの全体像です。**契約 → インタラクション → 評価 → レポート。** + +--- + +## AICertify が選ばれる理由 + +既存の AI ガバナンスツールの多くは、次のいずれかに該当します。 + +- **ベンダー SaaS**: 監査ログがログイン画面の奥に閉じ込められている (Credo AI、Holistic AI)、または +- **研究用ツールキット**: 公平性指標 (Fairlearn、AI Fairness 360) や説明可能性 (Microsoft RAI Toolbox) など、単一の側面のみに特化している。 + +どちらも、規制当局が実際に求める文書、すなわち *「名前の付いた規制に対してこの AI システムをテストし、再現可能なポリシーと日付入りのレポートで裏付けた証拠」* を生み出しません。 + +AICertify はまさにその成果物のために構築されています。 + +

+ AICertify と代替ツールの比較: AICertify は、名前の付いた規制フレームワーク、業種別ポリシー、監査対応レポートを標準装備した唯一のオープンソース・ポリシー・アズ・コード型ツールです +

+ +| | AICertify | Fairlearn / AIF360 | MS RAI Toolbox | Credo AI | +|---|---|---|---|---| +| オープンソース | ✅ Apache 2.0 | ✅ MIT | ✅ MIT | ❌ クローズド | +| オンプレミス / エアギャップ環境 | ✅ | ✅ | ✅ | ❌ | +| 名前の付いた規制フレームワーク | **EU AI Act、NIST RMF、ブラジル AI 法案、インド DPDP ほか 11 件** | ❌ (公平性のみ) | ❌ (ツールキット) | ✅ | +| ポリシー・アズ・コード (監査・差分比較可) | ✅ OPA / Rego | ❌ | ❌ | ❌ | +| 業種別ポリシー標準装備 | 航空、銀行、医療、自動車、教育 | ❌ | ❌ | 部分対応 | +| 監査対応レポートの生成 | ✅ PDF / MD / JSON / HTML | ❌ | 部分対応 | ✅ | +| カスタムポリシー | ✅ `.rego` ファイルを配置するだけ | ❌ | 該当なし | ✅ (有償) | + +--- + +## 仕組み + +

+ AICertify のアーキテクチャ: AI アプリが契約を生成し、評価器 (公平性、コンテンツ安全性、リスク管理、コンプライアンス) を経由して 94 個の Rego ポリシーを持つ OPA エンジンに送られ、レポート生成器が監査成果物を出力 +

+ +1. **契約 (Contract)** — AI アプリケーションを記述した JSON です。モデル、バージョン、収集したやり取り、メタデータを含みます。 +2. **評価器 (Evaluators)** — プラガブルな Python 評価器 (公平性、コンテンツ安全性、リスク管理、コンプライアンス) が、やり取りからメトリクスを抽出します。 +3. **OPA ポリシー** — 抽出されたメトリクスは、規制ごとの Rego ポリシー ([gopal](https://github.com/Principled-Evolution/gopal) ポリシーライブラリ由来) に対して評価されます。 +4. **レポート** — 日付入りのフォーマット済み成果物として、法務、監査人、AI リスク委員会へ提出できます。 + +ポリシーは宣言的な Rego で書かれているため、他のコードと同じくバージョン管理、差分比較、レビューが可能です。規制が変わったら、評価ハーネスではなくポリシーを更新するだけで済みます。 + +--- + +## 規制カバレッジ + +

+ 規制カバレッジ: 15 以上のフレームワークと 5 つの業種にわたる 94 ポリシー -- EU AI Act、NIST AI RMF、インド DPDP、ブラジル AI 法案、RTCA DO-365/366、FAA Part 107、EASA SORA、ICAO Doc 10019、医療、銀行・金融サービス、自動車、教育、グローバル、航空、AIOps、コーポレート +

+ +AICertify は [gopal](https://github.com/Principled-Evolution/gopal) ポリシーライブラリ — **本番運用可能な 94 個の OPA ポリシー** — を用いて、以下のフレームワークに対する評価を実行します。 + +### 国際フレームワーク +- **EU AI Act** — 29 ポリシー。禁止行為、生体識別、操作 (manipulation)、透明性、技術文書、人間による監督、GPAI 義務をカバー +- **NIST AI RMF** — Govern、Map、Measure、Manage に加え AI 600-1 +- **インド Digital Policy** — DPDP に整合する義務 +- **ブラジル AI ガバナンス法案** — アルゴリズム・ガバナンス要件 +- **航空標準** — ICAO Doc 10019、RTCA DO-365/366、ASTM F3442、ISO 21384、FAA Part 107、EASA SORA + +### 業種別 +- **航空** (17 ポリシー) — Detect-and-Avoid、認証、設計、統合検証 +- **教育** (12 ポリシー) — FERPA、COPPA、試験監督、ヒューマン・イン・ザ・ループでの採点 +- **銀行・金融サービス** — モデルリスク、公正融資 +- **医療** — 患者安全、診断安全 +- **自動車** — 車両安全統合 + +### グローバル & オペレーショナル +- **グローバル** — アカウンタビリティ、公平性、透明性、説明可能性、コンテンツ安全性、リスク管理、セキュリティ +- **コーポレート** — 情報セキュリティ、ガバナンス +- **AIOps & コスト** — スケーラビリティ、リソース効率 + +該当する規制が見当たらない場合は [Rego ファイルを追加](https://github.com/Principled-Evolution/gopal/blob/main/CONTRIBUTING.md) してください。ライブラリは拡張可能な設計になっています。 + +--- + +## CLI + +```bash +python -m aicertify.cli \ + --contract path/to/contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf \ + --output-dir reports/ +``` + +主なフラグ: + +| フラグ | 用途 | +|---|---| +| `--contract` | AI アプリケーション契約 JSON のパス | +| `--policy` | 評価対象とする OPA ポリシーフォルダのパス | +| `--report-format` | `pdf`、`markdown`、`json`、`html` (デフォルト: `pdf`) | +| `--evaluators` | 特定の評価器に限定 (例: `Fairness ContentSafety`) | +| `--output-dir` | レポート出力先 (デフォルト: `./reports`) | +| `--verbose` | 詳細ログ出力 | + +Python API の全体像は [`examples/quickstart.py`](examples/quickstart.py) を参照してください。 + +--- + +## サンプルレポート + +

+ 監査対応レポートの構成: フレームワーク名・アプリケーション・モデル・日付を含むヘッダー、エグゼクティブサマリー、ポリシー結果テーブル、リスク評価の棒グラフ、是正ガイダンス、AICertify v0.7.0 を示すフッター +

+ +`examples/outputs/` ディレクトリには、実評価から生成されたレポートが含まれており、実行前に内容を確認できます。 + +- `eu_ai_act/` — EU AI Act に対して評価したカスタマーサポートエージェント +- `loan_evaluation/` — 公正融資の観点で評価した信用スコアリングモデル +- `medical_diagnosis/` — 患者安全の観点で評価した臨床意思決定支援モデル + +PDF を開いてみてください。監査人が求めているのは、まさにこの形式の文書です。 + +--- + +## ステータス + +AICertify は現在 **ベータ版 (v0.7.0)** です。1.0 リリースまでに API が変更される可能性があります。本日時点で本番運用可能なフレームワークは次のとおりです。 + +- ✅ EU AI Act +- ✅ グローバル評価器 (公平性、コンテンツ安全性、透明性) +- ✅ 医療、BFS、自動車の業種別ポリシー +- ✅ 航空ポリシーセット (RTCA、ASTM、FAA、EASA) +- 🚧 NIST AI RMF — 部分対応 +- 🚧 インド Digital Policy — 初期段階 + +進捗状況は [ポリシーライブラリのロードマップ](https://github.com/Principled-Evolution/gopal) で確認できます。 + +--- + +## コントリビューション + +以下のような貢献を歓迎しています。 + +- 新しい規制フレームワーク (スコープのすり合わせのため、まず issue を立ててください) +- 実運用で鍛え上げた業種別ポリシー +- 新しい評価器 (公平性、安全性、堅牢性 — `aicertify/evaluators/` を参照) +- 最小限の再現用契約を添えたバグレポート + +まずは [CONTRIBUTING.md](CONTRIBUTING.md) と [行動規範](CODE_OF_CONDUCT.md) をご確認ください。 + +--- + +## 関連プロジェクト + +- **[gopal](https://github.com/Principled-Evolution/gopal)** — AICertify が内部で使用している OPA ポリシーライブラリです。Python フレームワークが不要な場合は、OPA CLI と組み合わせて単体で利用できます。 +- **[Open Policy Agent](https://www.openpolicyagent.org/)** — ポリシーエンジン本体。 +- **[Regal](https://github.com/StyraInc/regal)** — ポリシーを清潔に保つために使用している Rego リンター。 + +--- + +## ライセンス + +Apache License 2.0 — [LICENSE](LICENSE) を参照してください。 + +

Built by Principled Evolution · 読める、動かせる、証明できるポリシー。

diff --git a/README.ko-KR.md b/README.ko-KR.md new file mode 100644 index 0000000..58925e5 --- /dev/null +++ b/README.ko-KR.md @@ -0,0 +1,245 @@ +
+ AICertify +
+ +

AICertify

+ +

+ English | + 简体中文 | + 日本語 | + 한국어 | + हिन्दी +

+ +

+ AI 시스템을 위한 컴플라이언스 코드화(compliance-as-code). +

+ +

+ EU AI Act, NIST AI RMF를 비롯한 13개 이상의 프레임워크에 대해 AI를 감사합니다 — 하나의 계약, 하나의 명령, 하나의 리포트. +

+ +

+ CI + Stars + Version 0.7.0 + Python 3.12+ + Apache 2.0 + Built on OPA + 94 Rego Policies + Beta + PRs Welcome +

+ +

+ AI 애플리케이션에서 감사 준비 리포트까지: AI 애플리케이션 -> AICertify 계약 -> OPA 정책 평가 -> 컴플라이언스 리포트 +

+ +
+ +규제는 거버넌스 문서보다 빠르게 움직이고 있습니다. EU AI Act는 발효되었고, NIST AI RMF는 미국의 사실상 표준이 되었습니다. 인도, 브라질, 싱가포르가 그 뒤를 잇고 있습니다. `AICertify`는 이러한 의무를 실행 가능한 [Open Policy Agent](https://www.openpolicyagent.org/) 정책으로 인코딩하고, 수집된 AI 상호작용에 대해 실행하며, PDF, Markdown, JSON 또는 HTML 형식의 감사 준비 리포트를 생성할 수 있게 해줍니다. + +이는 *"우리에게는 책임 있는 AI 정책이 있다"* 와 *"우리는 그것을 증명할 수 있다"* 사이의 잃어버린 연결고리입니다. + +--- + +## 빠른 시작 + +```bash +git clone https://github.com/Principled-Evolution/aicertify.git +cd aicertify +pip install -e . +python examples/quickstart.py +``` + +빠른 시작 스크립트는 샘플 AI 애플리케이션을 EU AI Act 정책 세트에 통과시키고 컴플라이언스 리포트를 `reports/`에 작성합니다. 파일을 열어 보세요. 이것이 바로 여러분의 감사 산출물의 모습입니다 — 손으로 작성하지 않고 생성된 리포트입니다. + +### 최소 Python 사용 예시 + +```python +from aicertify import regulations, application + +# 1. 인증받을 규제를 선택합니다 +regs = regulations.create("my_regulations") +regs.add("eu_ai_act") + +# 2. AI 앱을 래핑합니다 +app = application.create( + name="customer-support-bot", + model_name="gpt-4o", + model_version="2024-08-06", +) + +# 3. 실제 상호작용을 입력합니다 +app.add_interaction( + input_text="I want a refund for my order", + output_text="I can help with that. Could you share your order number?", +) + +# 4. 평가 후 리포트를 받습니다 +await app.evaluate(regulations=regs, report_format="pdf", output_dir="reports") +``` + +이것이 전체 흐름입니다. **계약 → 상호작용 → 평가 → 리포트.** + +--- + +## AICertify를 선택해야 하는 이유 + +대부분의 AI 거버넌스 도구는 다음 중 하나입니다. + +- **벤더 SaaS** — 감사 추적을 로그인 뒤에 가두어 둡니다(Credo AI, Holistic AI). +- **연구용 툴킷** — 공정성 지표(Fairlearn, AI Fairness 360)나 설명 가능성(Microsoft RAI Toolbox)과 같은 단일 차원에만 집중합니다. + +이 중 어느 것도 규제 당국이 실제로 요구하는 문서, 즉 *명시된 규제에 대해 이 AI 시스템을 테스트했다는 증거, 재현 가능한 정책과 날짜가 기재된 리포트* 를 만들어 내지 못합니다. + +AICertify는 바로 그 산출물을 위해 만들어졌습니다. + +

+ AICertify와 대안 비교: AICertify는 명명된 규제 프레임워크, 산업별 수직 영역, 그리고 기본 제공되는 감사 준비 리포트를 갖춘 유일한 오픈소스 정책 코드화 옵션입니다 +

+ +| | AICertify | Fairlearn / AIF360 | MS RAI Toolbox | Credo AI | +|---|---|---|---|---| +| 오픈소스 | ✅ Apache 2.0 | ✅ MIT | ✅ MIT | ❌ 비공개 | +| 온프레미스 / 에어갭 환경 | ✅ | ✅ | ✅ | ❌ | +| 명명된 규제 프레임워크 | **EU AI Act, NIST RMF, Brazil AI Bill, India DPDP, +11개 이상** | ❌ (공정성만 해당) | ❌ (툴킷) | ✅ | +| 정책 코드화 (감사 가능, 비교 가능) | ✅ OPA / Rego | ❌ | ❌ | ❌ | +| 기본 제공 산업 수직 영역 | 항공, 금융, 의료, 자동차, 교육 | ❌ | ❌ | 일부 | +| 감사 준비 리포트 생성 | ✅ PDF / MD / JSON / HTML | ❌ | 일부 | ✅ | +| 맞춤형 정책 | ✅ `.rego` 파일 추가만으로 가능 | ❌ | 해당 없음 | ✅ (유료) | + +--- + +## 작동 방식 + +

+ AICertify 아키텍처: AI 앱이 계약을 제공하고, 이는 평가기(Fairness, ContentSafety, RiskManagement, Compliance)를 거쳐 94개의 Rego 정책을 포함한 OPA 엔진으로 흐르며, 리포트 생성기를 통해 감사 산출물을 만들어 냅니다 +

+ +1. **계약(Contract)** — AI 애플리케이션에 대한 JSON 설명: 모델, 버전, 수집된 상호작용, 메타데이터. +2. **평가기(Evaluators)** — 플러그형 Python 평가기(Fairness, ContentSafety, RiskManagement, Compliance)가 상호작용에서 메트릭을 추출합니다. +3. **OPA 정책** — 메트릭은 규제의 Rego 정책에 대해 평가됩니다([gopal](https://github.com/Principled-Evolution/gopal) 정책 라이브러리에서 제공). +4. **리포트** — 법무팀, 감사관 또는 AI 리스크 위원회에 전달할 수 있는 형식화되고 날짜가 기재된 산출물입니다. + +정책이 선언적 Rego이기 때문에, 다른 코드와 마찬가지로 버전 관리, 차이 비교, 리뷰가 가능합니다. 규제가 변경되면 평가 하네스가 아닌 정책을 업데이트하기만 하면 됩니다. + +--- + +## 규제 커버리지 + +

+ 규제 커버리지: 15개 이상의 프레임워크와 5개 산업에 걸친 94개 정책 -- EU AI Act, NIST AI RMF, India DPDP, Brazil AI Bill, RTCA DO-365/366, FAA Part 107, EASA SORA, ICAO Doc 10019, 의료, 금융, 자동차, 교육, 글로벌, 항공, AIOps, 기업 +

+ +AICertify는 [gopal](https://github.com/Principled-Evolution/gopal) 정책 라이브러리에서 실행됩니다 — 다음 프레임워크들에 걸친 **94개의 프로덕션 OPA 정책**입니다. + +### 국제 + +- **EU AI Act** — 금지된 관행, 생체 인식, 조작, 투명성, 기술 문서, 인간의 감독, GPAI 의무 사항을 포함하는 29개 정책 +- **NIST AI RMF** — Govern, Map, Measure, Manage + AI 600-1 +- **India Digital Policy** — DPDP 정렬 의무 사항 +- **Brazil AI Governance Bill** — 알고리즘 거버넌스 요구사항 +- **항공 표준** — ICAO Doc 10019, RTCA DO-365/366, ASTM F3442, ISO 21384, FAA Part 107, EASA SORA + +### 산업별 + +- **항공** (17개 정책) — 감지 및 회피, 인증, 설계, 통합 검증 +- **교육** (12개 정책) — FERPA, COPPA, 시험 감독, 사람이 참여하는 채점 +- **금융 서비스** — 모델 리스크, 공정 대출 +- **의료** — 환자 안전, 진단 안전 +- **자동차** — 차량 안전 통합 + +### 글로벌 및 운영 + +- **글로벌** — 책임성, 공정성, 투명성, 설명 가능성, 콘텐츠 안전, 리스크 관리, 보안 +- **기업** — 정보 보안, 거버넌스 +- **AIOps 및 비용** — 확장성, 리소스 효율성 + +찾는 규제가 보이지 않나요? [Rego 파일을 추가하세요](https://github.com/Principled-Evolution/gopal/blob/main/CONTRIBUTING.md). 이 라이브러리는 확장 가능하도록 설계되었습니다. + +--- + +## CLI + +```bash +python -m aicertify.cli \ + --contract path/to/contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf \ + --output-dir reports/ +``` + +유용한 플래그: + +| 플래그 | 용도 | +|---|---| +| `--contract` | AI 애플리케이션 계약 JSON 파일 경로 | +| `--policy` | 평가에 사용할 OPA 정책 폴더 경로 | +| `--report-format` | `pdf`, `markdown`, `json`, `html` (기본값: `pdf`) | +| `--evaluators` | 특정 평가기로 제한 (예: `Fairness ContentSafety`) | +| `--output-dir` | 리포트 출력 위치 (기본값: `./reports`) | +| `--verbose` | 상세 로깅 | + +전체 Python API는 [`examples/quickstart.py`](examples/quickstart.py)를 참고하세요. + +--- + +## 샘플 리포트 + +

+ 감사 준비 리포트의 구성: 프레임워크 이름, 애플리케이션, 모델, 날짜를 포함한 헤더, 요약 정리, 정책 결과 표, 리스크 평가 막대 차트, 시정 안내, AICertify v0.7.0 표시 푸터 +

+ +`examples/outputs/` 디렉터리에는 실제 평가에서 생성된 리포트가 들어 있어, 직접 실행하기 전에 살펴볼 수 있습니다. + +- `eu_ai_act/` — EU AI Act에 대해 평가된 고객 지원 에이전트 +- `loan_evaluation/` — 공정 대출에 대해 평가된 신용 평가 모델 +- `medical_diagnosis/` — 환자 안전에 대해 평가된 임상 의사 결정 지원 모델 + +PDF를 열어 보세요. 감사관이 원하는 것이 바로 이것입니다. + +--- + +## 상태 + +AICertify는 **베타 (v0.7.0)** 단계로, 1.0 릴리스 이전에 API가 변경될 수 있습니다. 현재 프로덕션 사용이 가능한 프레임워크는 다음과 같습니다. + +- ✅ EU AI Act +- ✅ 글로벌 평가기 (공정성, 콘텐츠 안전, 투명성) +- ✅ 의료, 금융, 자동차 산업 정책 +- ✅ 항공 정책 세트 (RTCA, ASTM, FAA, EASA) +- 🚧 NIST AI RMF — 부분 커버리지 +- 🚧 India Digital Policy — 초기 단계 + +진행 상황은 [정책 라이브러리 로드맵](https://github.com/Principled-Evolution/gopal)에서 확인하세요. + +--- + +## 기여 + +다음과 같은 기여를 환영합니다. + +- 새로운 규제 프레임워크 (범위 정렬을 위해 먼저 이슈를 열어 주세요) +- 현장 검증된 산업별 정책 +- 새로운 평가기 (공정성, 안전성, 견고성 — `aicertify/evaluators/` 참고) +- 최소 재현 가능 계약을 포함한 버그 리포트 + +[CONTRIBUTING.md](CONTRIBUTING.md)와 [행동 강령](CODE_OF_CONDUCT.md)부터 확인해 주세요. + +--- + +## 관련 프로젝트 + +- **[gopal](https://github.com/Principled-Evolution/gopal)** — AICertify가 내부적으로 사용하는 OPA 정책 라이브러리입니다. Python 프레임워크가 필요하지 않다면 OPA CLI와 함께 단독으로 사용할 수 있습니다. +- **[Open Policy Agent](https://www.openpolicyagent.org/)** — 정책 엔진. +- **[Regal](https://github.com/StyraInc/regal)** — 정책을 깔끔하게 유지하기 위해 사용하는 Rego 린터. + +--- + +## 라이선스 + +Apache License 2.0 — [LICENSE](LICENSE) 참고. + +

Principled Evolution이 제작 · 읽고, 실행하고, 증명할 수 있는 정책.

diff --git a/README.md b/README.md index 63c17af..2d34ebe 100644 --- a/README.md +++ b/README.md @@ -1,71 +1,242 @@ -
- AICertify Logo -
- -# AICertify: AI Regulatory Compliance Framework - -[![Build Status](https://github.com/Principled-Evolution/aicertify/actions/workflows/aicertify-ci.yaml/badge.svg)](https://github.com/Principled-Evolution/aicertify/actions/workflows/aicertify-ci.yaml) -[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) -[![Linting: ruff](https://img.shields.io/badge/linting-ruff-yellow.svg)](https://github.com/astral-sh/ruff) -[![Python Version](https://img.shields.io/badge/python-3.12-blue.svg)](https://www.python.org/) -[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -[![Status: Beta](https://img.shields.io/badge/status-beta-orange.svg)](https://github.com/Principled-Evolution/aicertify) -[![Version: 0.7.0](https://img.shields.io/badge/version-0.7.0-brightgreen.svg)](https://github.com/Principled-Evolution/aicertify) -[![Poetry](https://img.shields.io/badge/poetry-managed-blue)](https://python-poetry.org/) -[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://makeapullrequest.com) - -AICertify is a comprehensive open-source framework for evaluating AI systems against regulatory requirements and ethical standards. It provides a powerful set of tools for assessing AI applications, generating compliance reports, and identifying areas for improvement. - -> **Note:** AICertify is currently in beta stage (v0.7.0). The API may change between versions as we work toward a stable 1.0.0 release. - -## Overview - -AICertify provides tools to: - -- Create and manage AI application contracts -- Evaluate AI interactions against regulatory frameworks -- Generate compliance reports -- Identify potential risks and issues in AI systems - -## Installation - -```bash -# Install from source -pip install -e . -``` - -## Quickstart - -The simplest way to get started with AICertify is to run the quickstart example: - -```bash -python examples/quickstart.py -``` - -This example demonstrates: -- Creating a regulations set -- Selecting target regulations -- Creating AI applications -- Adding interactions to applications -- Evaluating applications against regulations -- Generating and viewing reports - -## Core Components - -- **Application Management**: Create and configure AI applications -- **Regulation Sets**: Select and manage regulatory frameworks -- **Evaluation**: Evaluate applications against regulations -- **Reporting**: Generate detailed compliance reports - -## OPA Policy Integration - -AICertify uses Open Policy Agent (OPA) for policy definition and evaluation. Policies are organized in the following structure: - -- `global/`: Globally applicable policies -- `industry_specific/`: Industry-specific policies -- `international/`: Policies for international regulations -- `custom/`: Directory for user-defined policies - -## License - -This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. +
+ AICertify +
+ +

AICertify

+ +

+ English | + 简体中文 | + 日本語 | + 한국어 | + हिन्दी +

+ +

+ Compliance-as-code for AI systems. +

+ +

+ Audit your AI against the EU AI Act, NIST AI RMF, and 13 more frameworks — one contract, one command, one report. +

+ +

+ CI + Stars + Version 0.7.0 + Python 3.12+ + Apache 2.0 + Built on OPA + 94 Rego Policies + Beta + PRs Welcome +

+ +

+ From AI app to audit-ready report: AI Application -> AICertify Contract -> OPA Policy Evaluation -> Compliance Report +

+ +
+ +Regulators are moving faster than your governance docs. The EU AI Act is in force. NIST AI RMF is the de-facto US standard. India, Brazil, and Singapore are next. `AICertify` lets you encode those obligations as executable [Open Policy Agent](https://www.openpolicyagent.org/) policies, run them against captured AI interactions, and produce audit-ready reports in PDF, Markdown, JSON, or HTML. + +It's the missing link between *"we have a responsible-AI policy"* and *"we can prove it."* + +--- + +## Quick Start + +```bash +git clone https://github.com/Principled-Evolution/aicertify.git +cd aicertify +pip install -e . +python examples/quickstart.py +``` + +The quickstart wires a sample AI application through the EU AI Act policy set and writes a compliance report into `reports/`. Open it. That's what your audit deliverable looks like — generated, not handwritten. + +### Minimal Python usage + +```python +from aicertify import regulations, application + +# 1. Pick the regulations you want to certify against +regs = regulations.create("my_regulations") +regs.add("eu_ai_act") + +# 2. Wrap your AI app +app = application.create( + name="customer-support-bot", + model_name="gpt-4o", + model_version="2024-08-06", +) + +# 3. Feed it real interactions +app.add_interaction( + input_text="I want a refund for my order", + output_text="I can help with that. Could you share your order number?", +) + +# 4. Evaluate and get reports back +await app.evaluate(regulations=regs, report_format="pdf", output_dir="reports") +``` + +That's the whole loop. **Contract → interactions → evaluate → report.** + +--- + +## Why AICertify + +Most AI-governance tooling is either: + +- **A vendor SaaS** that locks your audit trail behind a login (Credo AI, Holistic AI), or +- **A research toolkit** focused on a single dimension — fairness metrics (Fairlearn, AI Fairness 360) or explainability (Microsoft RAI Toolbox). + +Neither produces the document a regulator actually asks for: *evidence that you tested this AI system against a named regulation, with reproducible policies and a dated report.* + +AICertify is built for that artifact. + +

+ AICertify vs alternatives: AICertify is the only open-source, policy-as-code option with named regulatory frameworks, industry verticals, and audit-ready reports out of the box +

+ +| | AICertify | Fairlearn / AIF360 | MS RAI Toolbox | Credo AI | +|---|---|---|---|---| +| Open source | ✅ Apache 2.0 | ✅ MIT | ✅ MIT | ❌ Closed | +| On-prem / air-gapped | ✅ | ✅ | ✅ | ❌ | +| Named regulatory frameworks | **EU AI Act, NIST RMF, Brazil AI Bill, India DPDP, +11 more** | ❌ (fairness only) | ❌ (toolkit) | ✅ | +| Policy-as-code (auditable, diff-able) | ✅ OPA / Rego | ❌ | ❌ | ❌ | +| Industry verticals out of the box | Aviation, Banking, Healthcare, Automotive, Education | ❌ | ❌ | Partial | +| Generates audit-ready reports | ✅ PDF / MD / JSON / HTML | ❌ | Partial | ✅ | +| Custom policies | ✅ Drop a `.rego` file | ❌ | N/A | ✅ (paid) | + +--- + +## How It Works + +

+ AICertify architecture: Your AI App feeds a Contract, which flows through Evaluators (Fairness, ContentSafety, RiskManagement, Compliance) into the OPA Engine with 94 Rego policies, producing an audit deliverable via the Report Generator +

+ +1. **Contract** — A JSON description of your AI application: model, version, captured interactions, metadata. +2. **Evaluators** — Pluggable Python evaluators (Fairness, ContentSafety, RiskManagement, Compliance) extract metrics from your interactions. +3. **OPA policies** — The metrics get evaluated against the regulation's Rego policies (sourced from the [gopal](https://github.com/Principled-Evolution/gopal) policy library). +4. **Report** — A formatted, dated artifact you can hand to legal, an auditor, or your AI risk committee. + +Because the policies are declarative Rego, they version, diff, and review like any other code. When a regulation changes, you bump the policy — not your evaluation harness. + +--- + +## Regulatory Coverage + +

+ Regulatory coverage: 94 policies across 15+ frameworks and 5 industries -- EU AI Act, NIST AI RMF, India DPDP, Brazil AI Bill, RTCA DO-365/366, FAA Part 107, EASA SORA, ICAO Doc 10019, Healthcare, Banking and Financial Services, Automotive, Education, Global, Aviation, AIOps, Corporate +

+ +AICertify runs against the [gopal](https://github.com/Principled-Evolution/gopal) policy library — **94 production OPA policies** across these frameworks: + +### International +- **EU AI Act** — 29 policies covering prohibited practices, biometric ID, manipulation, transparency, technical documentation, human oversight, GPAI obligations +- **NIST AI RMF** — Govern, Map, Measure, Manage + AI 600-1 +- **India Digital Policy** — DPDP-aligned obligations +- **Brazil AI Governance Bill** — Algorithmic governance requirements +- **Aviation standards** — ICAO Doc 10019, RTCA DO-365/366, ASTM F3442, ISO 21384, FAA Part 107, EASA SORA + +### Industry-specific +- **Aviation** (17 policies) — Detect-and-avoid, certification, design, integration validation +- **Education** (12 policies) — FERPA, COPPA, proctoring, human-in-the-loop grading +- **Banking & Financial Services** — Model risk, fair lending +- **Healthcare** — Patient safety, diagnostic safety +- **Automotive** — Vehicle safety integration + +### Global & Operational +- **Global** — Accountability, fairness, transparency, explainability, content safety, risk management, security +- **Corporate** — InfoSec, governance +- **AIOps & Cost** — Scalability, resource efficiency + +Don't see your regulation? [Add a Rego file](https://github.com/Principled-Evolution/gopal/blob/main/CONTRIBUTING.md). The library is designed to be extended. + +--- + +## CLI + +```bash +python -m aicertify.cli \ + --contract path/to/contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf \ + --output-dir reports/ +``` + +Useful flags: + +| Flag | Purpose | +|---|---| +| `--contract` | Path to the AI application contract JSON | +| `--policy` | Path to the OPA policy folder to evaluate against | +| `--report-format` | `pdf`, `markdown`, `json`, `html` (default: `pdf`) | +| `--evaluators` | Restrict to specific evaluators (e.g. `Fairness ContentSafety`) | +| `--output-dir` | Where reports land (default: `./reports`) | +| `--verbose` | Verbose logging | + +See [`examples/quickstart.py`](examples/quickstart.py) for the full Python API. + +--- + +## Sample Reports + +

+ Anatomy of an audit-ready report: header with framework name, application, model and date; executive summary; policy results table; risk assessment bar chart; remediation guidance; footer attributing AICertify v0.7.0 +

+ +The `examples/outputs/` directory contains generated reports from real evaluations you can inspect before running anything: + +- `eu_ai_act/` — A customer-support agent evaluated against the EU AI Act +- `loan_evaluation/` — A credit-scoring model evaluated for fair lending +- `medical_diagnosis/` — A clinical-decision-support model evaluated for patient safety + +Open the PDFs. That's what your auditor wants. + +--- + +## Status + +AICertify is in **beta (v0.7.0)** — the API may evolve before the 1.0 release. Production-ready frameworks today: + +- ✅ EU AI Act +- ✅ Global evaluators (fairness, content safety, transparency) +- ✅ Healthcare, BFS, Automotive industry policies +- ✅ Aviation policy set (RTCA, ASTM, FAA, EASA) +- 🚧 NIST AI RMF — partial coverage +- 🚧 India Digital Policy — early stage + +Track progress in the [policy library roadmap](https://github.com/Principled-Evolution/gopal). + +--- + +## Contributing + +We welcome: + +- New regulatory frameworks (open an issue first to align scope) +- Industry-specific policies you've battle-tested +- New evaluators (fairness, safety, robustness — see `aicertify/evaluators/`) +- Bug reports with a minimal reproducing contract + +Start with [CONTRIBUTING.md](CONTRIBUTING.md) and the [Code of Conduct](CODE_OF_CONDUCT.md). + +--- + +## Related Projects + +- **[gopal](https://github.com/Principled-Evolution/gopal)** — The OPA policy library AICertify uses under the hood. Use it standalone with the OPA CLI if you don't need the Python framework. +- **[Open Policy Agent](https://www.openpolicyagent.org/)** — The policy engine. +- **[Regal](https://github.com/StyraInc/regal)** — Rego linter used to keep policies clean. + +--- + +## License + +Apache License 2.0 — see [LICENSE](LICENSE). + +

Built by Principled Evolution · Policies you can read, run, and prove.

diff --git a/README.zh-CN.md b/README.zh-CN.md new file mode 100644 index 0000000..33174f3 --- /dev/null +++ b/README.zh-CN.md @@ -0,0 +1,245 @@ +
+ AICertify +
+ +

AICertify

+ +

+ English | + 简体中文 | + 日本語 | + 한국어 | + हिन्दी +

+ +

+ 面向 AI 系统的合规即代码。 +

+ +

+ 依据 EU AI Act、NIST AI RMF 等 15 项框架审计您的 AI —— 一份合约、一条命令、一份报告。 +

+ +

+ 持续集成 + Star 数 + 版本 0.7.0 + Python 3.12+ + Apache 2.0 许可证 + 基于 OPA 构建 + 94 条 Rego 策略 + Beta 阶段 + 欢迎提交 PR +

+ +

+ 从 AI 应用到审计就绪的报告:AI 应用 -> AICertify 合约 -> OPA 策略评估 -> 合规报告 +

+ +
+ +监管机构推进的速度比您的治理文档更快。EU AI Act 已经生效。NIST AI RMF 已成为美国事实上的标准。印度、巴西和新加坡也将紧随其后。`AICertify` 让您能够将这些义务编码为可执行的 [Open Policy Agent](https://www.openpolicyagent.org/) 策略,在采集的 AI 交互数据上运行,并生成 PDF、Markdown、JSON 或 HTML 格式的审计就绪报告。 + +它是连接"我们有负责任的 AI 策略"与"我们能够证明这一点"之间缺失的一环。 + +--- + +## 快速开始 + +```bash +git clone https://github.com/Principled-Evolution/aicertify.git +cd aicertify +pip install -e . +python examples/quickstart.py +``` + +quickstart 会将一个示例 AI 应用接入 EU AI Act 策略集,并将合规报告写入 `reports/`。打开看看 —— 这就是您的审计交付物的样貌:由系统生成,而非手工撰写。 + +### 最简 Python 用法 + +```python +from aicertify import regulations, application + +# 1. 选择需要认证的法规 +regs = regulations.create("my_regulations") +regs.add("eu_ai_act") + +# 2. 包装您的 AI 应用 +app = application.create( + name="customer-support-bot", + model_name="gpt-4o", + model_version="2024-08-06", +) + +# 3. 输入真实的交互数据 +app.add_interaction( + input_text="I want a refund for my order", + output_text="I can help with that. Could you share your order number?", +) + +# 4. 评估并取回报告 +await app.evaluate(regulations=regs, report_format="pdf", output_dir="reports") +``` + +整个闭环就这么简单。**合约 → 交互 → 评估 → 报告。** + +--- + +## 为何选择 AICertify + +大多数 AI 治理工具不外乎两类: + +- **厂商 SaaS**,将您的审计轨迹锁在登录页之后(Credo AI、Holistic AI),或 +- **研究工具包**,只聚焦单一维度 —— 公平性指标(Fairlearn、AI Fairness 360)或可解释性(Microsoft RAI Toolbox)。 + +二者都无法产出监管机构真正索要的文件:*您依据某项具名法规对该 AI 系统进行了测试的证据,包含可复现的策略与带日期的报告。* + +AICertify 正是为这份交付物而生。 + +

+ AICertify 与同类工具对比:AICertify 是唯一开箱即用、采用策略即代码、覆盖具名法规框架与行业垂直领域并生成审计就绪报告的开源选项 +

+ +| | AICertify | Fairlearn / AIF360 | MS RAI Toolbox | Credo AI | +|---|---|---|---|---| +| 开源 | ✅ Apache 2.0 | ✅ MIT | ✅ MIT | ❌ 闭源 | +| 本地部署 / 内网隔离 | ✅ | ✅ | ✅ | ❌ | +| 具名法规框架 | **EU AI Act、NIST RMF、Brazil AI Bill、India DPDP 等 15+** | ❌(仅公平性) | ❌(工具包) | ✅ | +| 策略即代码(可审计、可对比) | ✅ OPA / Rego | ❌ | ❌ | ❌ | +| 开箱即用的行业垂直领域 | 航空、银行、医疗、汽车、教育 | ❌ | ❌ | 部分 | +| 生成审计就绪报告 | ✅ PDF / MD / JSON / HTML | ❌ | 部分 | ✅ | +| 自定义策略 | ✅ 放入一个 `.rego` 文件即可 | ❌ | 不适用 | ✅(付费) | + +--- + +## 工作原理 + +

+ AICertify 架构:您的 AI 应用提供合约,合约流经评估器(Fairness、ContentSafety、RiskManagement、Compliance)进入承载 94 条 Rego 策略的 OPA 引擎,并通过报告生成器产出审计交付物 +

+ +1. **合约(Contract)** —— 用 JSON 描述您的 AI 应用:模型、版本、采集的交互、元数据。 +2. **评估器(Evaluators)** —— 可插拔的 Python 评估器(Fairness、ContentSafety、RiskManagement、Compliance),从交互数据中提取指标。 +3. **OPA 策略** —— 指标依据法规对应的 Rego 策略进行评估(策略来自 [gopal](https://github.com/Principled-Evolution/gopal) 策略库)。 +4. **报告** —— 一份格式化、带日期的交付物,可直接交给法务、审计师或 AI 风险委员会。 + +由于策略是声明式的 Rego,它们可像任何代码一样进行版本管理、差异比对和评审。法规变更时,您只需升级策略,而无需修改评估流水线。 + +--- + +## 法规覆盖 + +

+ 法规覆盖:94 条策略,横跨 15+ 框架与 5 个行业 —— EU AI Act、NIST AI RMF、India DPDP、Brazil AI Bill、RTCA DO-365/366、FAA Part 107、EASA SORA、ICAO Doc 10019、医疗、银行金融服务、汽车、教育、全球、航空、AIOps、企业 +

+ +AICertify 运行于 [gopal](https://github.com/Principled-Evolution/gopal) 策略库之上 —— **94 条生产级 OPA 策略**,覆盖以下框架: + +### 国际 + +- **EU AI Act** —— 29 条策略,涵盖禁止性实践、生物识别、操纵行为、透明度、技术文档、人类监督、GPAI 义务 +- **NIST AI RMF** —— Govern、Map、Measure、Manage 以及 AI 600-1 +- **India Digital Policy** —— 对齐 DPDP 的义务要求 +- **Brazil AI Governance Bill** —— 算法治理要求 +- **航空标准** —— ICAO Doc 10019、RTCA DO-365/366、ASTM F3442、ISO 21384、FAA Part 107、EASA SORA + +### 行业专属 + +- **航空**(17 条策略) —— 感知与规避、适航认证、设计、集成验证 +- **教育**(12 条策略) —— FERPA、COPPA、在线监考、人类参与的评分 +- **银行与金融服务** —— 模型风险、公平借贷 +- **医疗** —— 患者安全、诊断安全 +- **汽车** —— 车辆安全集成 + +### 全球与运营 + +- **全球** —— 问责、公平、透明、可解释性、内容安全、风险管理、安全 +- **企业** —— 信息安全、治理 +- **AIOps 与成本** —— 可扩展性、资源效率 + +没有看到您关心的法规?[添加一个 Rego 文件](https://github.com/Principled-Evolution/gopal/blob/main/CONTRIBUTING.md)即可。该策略库本就为可扩展而设计。 + +--- + +## 命令行(CLI) + +```bash +python -m aicertify.cli \ + --contract path/to/contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf \ + --output-dir reports/ +``` + +常用参数: + +| 参数 | 用途 | +|---|---| +| `--contract` | AI 应用合约 JSON 的路径 | +| `--policy` | 要使用的 OPA 策略目录路径 | +| `--report-format` | `pdf`、`markdown`、`json`、`html`(默认:`pdf`) | +| `--evaluators` | 限定特定评估器(例如 `Fairness ContentSafety`) | +| `--output-dir` | 报告输出目录(默认:`./reports`) | +| `--verbose` | 详细日志 | + +完整的 Python API 请参阅 [`examples/quickstart.py`](examples/quickstart.py)。 + +--- + +## 示例报告 + +

+ 审计就绪报告的结构:含框架名称、应用、模型与日期的页眉;执行摘要;策略结果表;风险评估柱状图;整改建议;以及标注 AICertify v0.7.0 的页脚 +

+ +`examples/outputs/` 目录中包含来自真实评估的生成报告,您可以在运行前先行查阅: + +- `eu_ai_act/` —— 一个客户支持智能体依据 EU AI Act 的评估 +- `loan_evaluation/` —— 一个信用评分模型的公平借贷评估 +- `medical_diagnosis/` —— 一个临床决策支持模型的患者安全评估 + +打开这些 PDF,这正是审计师想要的样子。 + +--- + +## 状态 + +AICertify 目前处于 **beta(v0.7.0)** 阶段 —— 1.0 版本发布前 API 仍可能演进。已可用于生产的框架包括: + +- ✅ EU AI Act +- ✅ 全球评估器(公平性、内容安全、透明度) +- ✅ 医疗、银行金融服务、汽车行业策略 +- ✅ 航空策略集(RTCA、ASTM、FAA、EASA) +- 🚧 NIST AI RMF —— 部分覆盖 +- 🚧 India Digital Policy —— 早期阶段 + +在[策略库路线图](https://github.com/Principled-Evolution/gopal)中跟踪进展。 + +--- + +## 参与贡献 + +我们欢迎: + +- 新的法规框架(请先开 issue 对齐范围) +- 您已在生产中验证的行业策略 +- 新的评估器(公平性、安全性、鲁棒性 —— 参见 `aicertify/evaluators/`) +- 附带最小可复现合约的缺陷报告 + +请从 [CONTRIBUTING.md](CONTRIBUTING.md) 与[行为准则](CODE_OF_CONDUCT.md)开始。 + +--- + +## 相关项目 + +- **[gopal](https://github.com/Principled-Evolution/gopal)** —— AICertify 底层使用的 OPA 策略库。如不需要 Python 框架,可搭配 OPA CLI 独立使用。 +- **[Open Policy Agent](https://www.openpolicyagent.org/)** —— 策略引擎。 +- **[Regal](https://github.com/StyraInc/regal)** —— 用于保持策略整洁的 Rego 代码检查工具。 + +--- + +## 许可证 + +Apache License 2.0 —— 参见 [LICENSE](LICENSE)。 + +

Principled Evolution 构建 · 可读、可运行、可证明的策略。

diff --git a/diagrams/diagram1_hero_flow.png b/diagrams/diagram1_hero_flow.png new file mode 100644 index 0000000..03760fe Binary files /dev/null and b/diagrams/diagram1_hero_flow.png differ diff --git a/diagrams/diagram2_architecture.png b/diagrams/diagram2_architecture.png new file mode 100644 index 0000000..cef83ac Binary files /dev/null and b/diagrams/diagram2_architecture.png differ diff --git a/diagrams/diagram3_regulatory_coverage.png b/diagrams/diagram3_regulatory_coverage.png new file mode 100644 index 0000000..82b344c Binary files /dev/null and b/diagrams/diagram3_regulatory_coverage.png differ diff --git a/diagrams/diagram4_comparison.png b/diagrams/diagram4_comparison.png new file mode 100644 index 0000000..8cdad65 Binary files /dev/null and b/diagrams/diagram4_comparison.png differ diff --git a/diagrams/diagram5_report_anatomy.png b/diagrams/diagram5_report_anatomy.png new file mode 100644 index 0000000..bb55b47 Binary files /dev/null and b/diagrams/diagram5_report_anatomy.png differ diff --git a/diagrams/generate_diagrams.py b/diagrams/generate_diagrams.py new file mode 100644 index 0000000..27920a3 --- /dev/null +++ b/diagrams/generate_diagrams.py @@ -0,0 +1,1028 @@ +"""Generate marketing diagrams for the AICertify README. + +Run from repo root: python diagrams/generate_diagrams.py +Outputs 5 PNGs (1600x900, ~retina-2x) into the diagrams/ directory. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +import matplotlib.pyplot as plt +from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle + +# ---------- shared style ---------- +PURPLE = "#7D4698" +PURPLE_DARK = "#5C2E76" +BLUE = "#1971c2" +GREEN = "#2f9e44" +ORANGE = "#e8590c" +TEXT = "#212529" +MUTED = "#495057" +LIGHT_BG = "#f1f3f5" +LIGHTER_BG = "#f8f9fa" +BORDER = "#dee2e6" +WHITE = "#ffffff" +RED = "#c92a2a" + +# Figure size: 16x9 inches at 100 dpi -> 1600x900px (retina-friendly) +FIGSIZE = (16, 9) +DPI = 100 + +# Typography +TITLE_SIZE = 30 +SUBTITLE_SIZE = 18 +BODY_SIZE = 16 +SMALL_SIZE = 13 + +HERE = Path(__file__).resolve().parent + + +# ---------- helpers ---------- +def new_fig(): + """Create a clean 16:9 figure with white background.""" + fig, ax = plt.subplots(figsize=FIGSIZE, dpi=DPI) + fig.patch.set_facecolor(WHITE) + ax.set_facecolor(WHITE) + ax.set_xlim(0, 100) + ax.set_ylim(0, 56.25) # keeps a 16:9 aspect in data coords + ax.set_xticks([]) + ax.set_yticks([]) + for spine in ax.spines.values(): + spine.set_visible(False) + return fig, ax + + +def title(ax, text, y=52, color=TEXT, size=TITLE_SIZE, weight="bold"): + ax.text( + 50, + y, + text, + ha="center", + va="center", + fontsize=size, + fontweight=weight, + color=color, + ) + + +def subtitle(ax, text, y=47, color=MUTED, size=SUBTITLE_SIZE): + ax.text(50, y, text, ha="center", va="center", fontsize=size, color=color) + + +def draw_box( + ax, + x, + y, + w, + h, + facecolor=WHITE, + edgecolor=PURPLE, + linewidth=2.2, + rounding=0.4, + shadow=False, +): + """Draw a rounded rectangle centered on (x, y) with width w, height h.""" + if shadow: + # offset shadow + shadow_box = FancyBboxPatch( + (x - w / 2 + 0.25, y - h / 2 - 0.35), + w, + h, + boxstyle=f"round,pad=0.02,rounding_size={rounding}", + linewidth=0, + facecolor="#000000", + alpha=0.08, + zorder=1, + ) + ax.add_patch(shadow_box) + box = FancyBboxPatch( + (x - w / 2, y - h / 2), + w, + h, + boxstyle=f"round,pad=0.02,rounding_size={rounding}", + linewidth=linewidth, + edgecolor=edgecolor, + facecolor=facecolor, + zorder=2, + ) + ax.add_patch(box) + return box + + +def labeled_box( + ax, + cx, + cy, + w, + h, + title_text, + subtitle_text=None, + facecolor=WHITE, + edgecolor=PURPLE, + title_color=None, + subtitle_color=None, + title_size=BODY_SIZE + 2, + subtitle_size=SMALL_SIZE, + shadow=True, +): + draw_box( + ax, + cx, + cy, + w, + h, + facecolor=facecolor, + edgecolor=edgecolor, + shadow=shadow, + ) + tc = ( + title_color + if title_color + else (WHITE if facecolor in (PURPLE, PURPLE_DARK) else TEXT) + ) + sc = ( + subtitle_color + if subtitle_color + else (WHITE if facecolor in (PURPLE, PURPLE_DARK) else MUTED) + ) + if subtitle_text: + title_lines = title_text.count("\n") + 1 + sub_lines = subtitle_text.count("\n") + 1 + # vertical spacing roughly proportional to font size (1 unit ~ 14pt) + title_line_h = title_size / 14.0 + sub_line_h = subtitle_size / 14.0 + gap = 0.8 + total_h = title_lines * title_line_h + gap + sub_lines * sub_line_h + # top of text block + top = cy + total_h / 2 + # title block center + title_cy = top - (title_lines * title_line_h) / 2 + sub_cy = ( + title_cy + - (title_lines * title_line_h) / 2 + - gap + - (sub_lines * sub_line_h) / 2 + ) + ax.text( + cx, + title_cy, + title_text, + ha="center", + va="center", + fontsize=title_size, + fontweight="bold", + color=tc, + zorder=3, + linespacing=1.0, + ) + ax.text( + cx, + sub_cy, + subtitle_text, + ha="center", + va="center", + fontsize=subtitle_size, + color=sc, + zorder=3, + linespacing=1.1, + ) + else: + ax.text( + cx, + cy, + title_text, + ha="center", + va="center", + fontsize=title_size, + fontweight="bold", + color=tc, + zorder=3, + ) + + +def draw_arrow(ax, x1, y1, x2, y2, color=PURPLE, lw=2.8, mutation_scale=28): + arr = FancyArrowPatch( + (x1, y1), + (x2, y2), + arrowstyle="-|>", + mutation_scale=mutation_scale, + linewidth=lw, + color=color, + zorder=2, + ) + ax.add_patch(arr) + + +def save_fig(fig, name): + out = HERE / name + fig.savefig(out, dpi=DPI, bbox_inches="tight", facecolor=WHITE, pad_inches=0.25) + plt.close(fig) + size_kb = os.path.getsize(out) / 1024 + print(f" wrote {name} ({size_kb:.1f} KB)") + + +# ---------- diagram 1: hero flow ---------- +def diagram1_hero_flow(): + fig, ax = new_fig() + title(ax, "From AI app to audit-ready report") + subtitle( + ax, + "One contract. One command. One report your auditor accepts.", + ) + + steps = [ + ("AI\nApplication", "model + interactions\n+ metadata", BLUE), + ("AICertify\nContract", "captured\nas JSON", PURPLE), + ("OPA Policy\nEvaluation", "EU AI Act, NIST\n+ 13 more", GREEN), + ("Compliance\nReport", "PDF, Markdown,\nJSON, HTML", ORANGE), + ] + + box_w = 19 + box_h = 18 + y = 26 + # 4 boxes spaced with clear gaps for arrows + centers = [12, 37.33, 62.66, 88] + + for i, (cx, (t, s, color)) in enumerate(zip(centers, steps)): + # numbered circle on top of each box + circle = plt.Circle( + (cx - box_w / 2 + 1.8, y + box_h / 2 - 1.2), + 1.3, + color=color, + zorder=4, + ) + ax.add_patch(circle) + ax.text( + cx - box_w / 2 + 1.8, + y + box_h / 2 - 1.2, + str(i + 1), + ha="center", + va="center", + color=WHITE, + fontsize=14, + fontweight="bold", + zorder=5, + ) + labeled_box( + ax, + cx, + y, + box_w, + box_h, + t, + s, + facecolor=WHITE, + edgecolor=color, + title_size=18, + subtitle_size=13, + shadow=True, + ) + + # arrows + for i in range(3): + x_from = centers[i] + box_w / 2 + 0.2 + x_to = centers[i + 1] - box_w / 2 - 0.2 + draw_arrow(ax, x_from, y, x_to, y, color=MUTED, lw=2.5, mutation_scale=24) + + # footer tag + ax.text( + 50, + 7, + "Open source - Apache 2.0 | Built on Open Policy Agent | 94 Rego policies", + ha="center", + va="center", + fontsize=14, + color=MUTED, + ) + + save_fig(fig, "diagram1_hero_flow.png") + + +# ---------- diagram 2: architecture ---------- +def diagram2_architecture(): + fig, ax = new_fig() + title(ax, "How AICertify works", y=53) + subtitle( + ax, + "Pluggable evaluators feed OPA. OPA produces the report.", + y=48.5, + ) + + # LEFT: your AI app + labeled_box( + ax, + 10, + 24, + 14, + 11, + "Your AI App", + "any LLM, agent,\nor pipeline", + facecolor=BLUE, + edgecolor=BLUE, + title_color=WHITE, + subtitle_color=WHITE, + title_size=16, + subtitle_size=11, + shadow=True, + ) + + # CENTER stack + cx = 50 + box_w = 44 + box_h = 7.2 + rows = [ + ( + 38.5, + "Contract", + "model_name | version | interactions | metadata", + PURPLE, + ), + ( + 29, + "Evaluators", + "Fairness | ContentSafety | RiskManagement | Compliance", + PURPLE, + ), + ( + 19.5, + "OPA Engine + Rego Policies", + "94 policies sourced from the gopal library", + PURPLE_DARK, + ), + ( + 10, + "Report Generator", + "ReportLab PDF | Markdown | JSON | HTML", + PURPLE, + ), + ] + + for y, t, s, color in rows: + labeled_box( + ax, + cx, + y, + box_w, + box_h, + t, + s, + facecolor=color, + edgecolor=color, + title_color=WHITE, + subtitle_color="#e9d5f0", + title_size=17, + subtitle_size=12, + shadow=True, + ) + + # arrows down the stack + for i in range(len(rows) - 1): + y_from = rows[i][0] - box_h / 2 - 0.05 + y_to = rows[i + 1][0] + box_h / 2 + 0.05 + draw_arrow(ax, cx, y_from, cx, y_to, color=MUTED, lw=2.4, mutation_scale=22) + + # arrow from left box into center stack (top) + draw_arrow(ax, 10 + 14 / 2 + 0.2, 24, cx - box_w / 2 - 0.2, rows[0][0], color=MUTED) + + # RIGHT: audit deliverable + labeled_box( + ax, + 90, + 24, + 14, + 11, + "Audit\nDeliverable", + "dated, signed,\nreproducible", + facecolor=ORANGE, + edgecolor=ORANGE, + title_color=WHITE, + subtitle_color=WHITE, + title_size=16, + subtitle_size=11, + shadow=True, + ) + # arrow from bottom of stack out to the right box + draw_arrow( + ax, + cx + box_w / 2 + 0.2, + rows[-1][0], + 90 - 14 / 2 - 0.2, + 24, + color=MUTED, + ) + + save_fig(fig, "diagram2_architecture.png") + + +# ---------- diagram 3: regulatory coverage ---------- +def diagram3_regulatory_coverage(): + fig, ax = new_fig() + title(ax, "Regulatory coverage - 15+ frameworks", y=53) + ax.text( + 50, + 47.5, + "94 policies | 15+ frameworks | 5 industries", + ha="center", + va="center", + fontsize=22, + fontweight="bold", + color=PURPLE, + ) + + # row category labels + row_labels = [ + "International", + "Aviation Safety", + "Industry", + "Cross-cutting", + ] + + cells = [ + # row 1: International + [ + ("EU AI Act", "29"), + ("NIST AI RMF", "5"), + ("India DPDP", "1"), + ("Brazil AI Bill", "1"), + ], + # row 2: Aviation Safety + [ + ("RTCA DO-365/366", "2"), + ("FAA Part 107", "2"), + ("EASA SORA", "2"), + ("ICAO Doc 10019", "1"), + ], + # row 3: Industry + [ + ("Healthcare", "2"), + ("Banking & FS", "2"), + ("Automotive", "1"), + ("Education", "12"), + ], + # row 4: Cross-cutting + [ + ("Global", "9"), + ("Aviation Vertical", "17"), + ("AIOps", "1"), + ("Corporate", "2"), + ], + ] + + # grid layout area: rows centered at y values, columns centered at x values + col_x = [22, 41, 60, 79] + row_y = [37, 28, 19, 10] + cell_w = 16.5 + cell_h = 7.4 + + # row labels on the far left + for ry, label in zip(row_y, row_labels): + ax.text( + 6.5, + ry, + label, + ha="center", + va="center", + fontsize=13, + fontweight="bold", + color=MUTED, + rotation=0, + ) + + for r_i, row in enumerate(cells): + for c_i, (name, count) in enumerate(row): + cx = col_x[c_i] + cy = row_y[r_i] + draw_box( + ax, + cx, + cy, + cell_w, + cell_h, + facecolor=PURPLE, + edgecolor=PURPLE_DARK, + linewidth=1.5, + rounding=0.3, + shadow=True, + ) + # framework name + ax.text( + cx, + cy + 1.1, + name, + ha="center", + va="center", + fontsize=14, + fontweight="bold", + color=WHITE, + zorder=3, + ) + # policy count pill + ax.text( + cx, + cy - 1.5, + f"{count} {'policy' if count == '1' else 'policies'}", + ha="center", + va="center", + fontsize=12, + color="#e9d5f0", + zorder=3, + ) + + save_fig(fig, "diagram3_regulatory_coverage.png") + + +# ---------- diagram 4: comparison ---------- +def diagram4_comparison(): + fig, ax = new_fig() + title(ax, "AICertify vs alternatives", y=53.5) + subtitle( + ax, + "The only open-source, policy-as-code option with named regulatory coverage.", + y=48.8, + ) + + columns = [ + "Feature", + "AICertify", + "Fairlearn", + "AI Fairness 360", + "MS RAI Toolbox", + "Credo AI", + ] + rows = [ + ( + "Open source (Apache 2.0)", + ["YES", "YES (MIT)", "YES (MIT)", "YES (MIT)", "NO"], + ), + ("On-prem capable", ["YES", "YES", "YES", "YES", "NO"]), + ("Policy-as-code (OPA / Rego)", ["YES", "NO", "NO", "NO", "NO"]), + ("Named regulatory frameworks (15+)", ["YES", "NO", "NO", "NO", "PARTIAL"]), + ("Industry verticals out of box", ["YES", "NO", "NO", "NO", "PARTIAL"]), + ("Audit-ready report output", ["YES", "NO", "NO", "PARTIAL", "YES"]), + ("Custom policies via `.rego`", ["YES", "NO", "NO", "NO", "PAID"]), + ] + + # Layout: columns + col_x = [18, 39, 52, 65, 78, 91] + col_w = [26, 11, 13, 13, 13, 11] + header_y = 42 + row_h = 4.2 + first_row_y = 38 + + # column header strip + for cx, cw, name in zip(col_x, col_w, columns): + is_aic = name == "AICertify" + ax.add_patch( + Rectangle( + (cx - cw / 2, header_y - row_h / 2), + cw, + row_h, + facecolor=PURPLE if is_aic else "#343a40", + edgecolor="none", + zorder=2, + ) + ) + ax.text( + cx, + header_y, + name, + ha="center", + va="center", + fontsize=14 if is_aic else 12, + fontweight="bold", + color=WHITE, + zorder=3, + ) + + # highlight strip for AICertify column behind rows + aic_x = col_x[1] + aic_w = col_w[1] + rows_total_h = row_h * len(rows) + ax.add_patch( + Rectangle( + (aic_x - aic_w / 2, first_row_y - row_h * (len(rows) - 1) - row_h / 2), + aic_w, + rows_total_h, + facecolor="#f3e8f7", + edgecolor="none", + zorder=1, + ) + ) + + def cell_marker(value): + if value == "YES": + return ("Yes", GREEN, "bold") + if value == "NO": + return ("No", RED, "bold") + if value == "PARTIAL": + return ("Partial", ORANGE, "bold") + if value == "PAID": + return ("Paid", ORANGE, "bold") + if value.startswith("YES "): + return (value.replace("YES ", "Yes "), GREEN, "bold") + return (value, TEXT, "normal") + + for r_i, (feature, vals) in enumerate(rows): + y = first_row_y - r_i * row_h + # alternating row band + if r_i % 2 == 0: + ax.add_patch( + Rectangle( + (col_x[0] - col_w[0] / 2, y - row_h / 2), + sum(col_w) + + (col_x[-1] + col_w[-1] / 2 - (col_x[0] - col_w[0] / 2)) + - sum(col_w), + row_h, + facecolor=LIGHTER_BG, + edgecolor="none", + zorder=0, + ) + ) + # feature label (left col) + ax.text( + col_x[0] - col_w[0] / 2 + 1, + y, + feature, + ha="left", + va="center", + fontsize=13, + color=TEXT, + zorder=3, + ) + # value cells + for c_i, v in enumerate(vals): + label, color, weight = cell_marker(v) + ax.text( + col_x[c_i + 1], + y, + label, + ha="center", + va="center", + fontsize=13, + color=color, + fontweight=weight, + zorder=3, + ) + + # bottom border + ax.plot( + [col_x[0] - col_w[0] / 2, col_x[-1] + col_w[-1] / 2], + [ + first_row_y - len(rows) * row_h + row_h / 2, + first_row_y - len(rows) * row_h + row_h / 2, + ], + color=BORDER, + lw=1.5, + zorder=2, + ) + + # footer + ax.text( + 50, + 4, + "Source: public docs and source repos as of 2025. AICertify is community-maintained.", + ha="center", + va="center", + fontsize=11, + color=MUTED, + style="italic", + ) + + save_fig(fig, "diagram4_comparison.png") + + +# ---------- diagram 5: report anatomy ---------- +def diagram5_report_anatomy(): + fig, ax = new_fig() + title(ax, "Anatomy of an audit-ready report", y=53.5) + subtitle( + ax, + "What the generated PDF actually contains.", + y=48.8, + ) + + # The "document" page + page_x, page_y = 50, 23.5 + page_w, page_h = 70, 41 + + # shadow + shadow_rect = Rectangle( + (page_x - page_w / 2 + 0.5, page_y - page_h / 2 - 0.6), + page_w, + page_h, + facecolor="#000000", + alpha=0.10, + edgecolor="none", + zorder=1, + ) + ax.add_patch(shadow_rect) + # page background + ax.add_patch( + Rectangle( + (page_x - page_w / 2, page_y - page_h / 2), + page_w, + page_h, + facecolor=WHITE, + edgecolor=BORDER, + linewidth=1.5, + zorder=2, + ) + ) + + left = page_x - page_w / 2 + 2.5 + right = page_x + page_w / 2 - 2.5 + top = page_y + page_h / 2 + + # Header band + ax.add_patch( + Rectangle( + (page_x - page_w / 2, top - 5.5), + page_w, + 5.5, + facecolor=PURPLE, + edgecolor="none", + zorder=3, + ) + ) + ax.text( + left, + top - 2.0, + "EU AI Act Compliance Report", + ha="left", + va="center", + fontsize=17, + fontweight="bold", + color=WHITE, + zorder=4, + ) + ax.text( + left, + top - 4.0, + "customer-support-bot | gpt-4o | 2026-05-14", + ha="left", + va="center", + fontsize=12, + color="#e9d5f0", + zorder=4, + ) + # PASS stamp on header + ax.add_patch( + FancyBboxPatch( + (right - 9, top - 4.6), + 8, + 3.4, + boxstyle="round,pad=0.02,rounding_size=0.4", + facecolor=GREEN, + edgecolor=GREEN, + zorder=4, + ) + ) + ax.text( + right - 5, + top - 2.9, + "PASS", + ha="center", + va="center", + fontsize=15, + fontweight="bold", + color=WHITE, + zorder=5, + ) + + # Section 1: Executive Summary + sec_y = top - 8.5 + ax.text( + left, + sec_y, + "1. Executive Summary", + ha="left", + va="center", + fontsize=14, + fontweight="bold", + color=PURPLE, + zorder=4, + ) + for i, line in enumerate( + [ + "27 of 29 EU AI Act policies passed. 2 advisory findings.", + "Risk classification: limited risk (Article 52 transparency).", + ] + ): + ax.text( + left, + sec_y - 2 - i * 1.7, + line, + ha="left", + va="center", + fontsize=11.5, + color=TEXT, + zorder=4, + ) + + # Section 2: Policy Results table + sec_y2 = sec_y - 7.5 + ax.text( + left, + sec_y2, + "2. Policy Results", + ha="left", + va="center", + fontsize=14, + fontweight="bold", + color=PURPLE, + zorder=4, + ) + + tbl_top = sec_y2 - 2 + # header row + ax.add_patch( + Rectangle( + (left, tbl_top - 1.4), + right - left - 22, + 1.4, + facecolor=LIGHT_BG, + edgecolor=BORDER, + zorder=3, + ) + ) + headers = ["Policy", "Status", "Evidence"] + hxs = [left + 0.5, left + 22, left + 30] + for hx, h in zip(hxs, headers): + ax.text( + hx, + tbl_top - 0.7, + h, + ha="left", + va="center", + fontsize=11, + fontweight="bold", + color=MUTED, + zorder=4, + ) + + rows = [ + ("Art. 5 Prohibited practices", "Pass", "0/127 interactions flagged"), + ("Art. 10 Data governance", "Pass", "training set documented"), + ("Art. 13 Transparency", "Pass", "user notice present"), + ("Art. 14 Human oversight", "Advisory", "review queue partial"), + ] + for i, (p, status, ev) in enumerate(rows): + y = tbl_top - 1.4 - 1.4 - i * 1.5 + color = GREEN if status == "Pass" else ORANGE + ax.text( + left + 0.5, + y, + p, + ha="left", + va="center", + fontsize=10.5, + color=TEXT, + zorder=4, + ) + ax.text( + left + 22, + y, + status, + ha="left", + va="center", + fontsize=10.5, + fontweight="bold", + color=color, + zorder=4, + ) + ax.text( + left + 30, + y, + ev, + ha="left", + va="center", + fontsize=10.5, + color=MUTED, + zorder=4, + ) + + # Section 3: Risk Assessment with mini bar chart + sec_y3 = sec_y2 - 12.5 + ax.text( + left, + sec_y3, + "3. Risk Assessment", + ha="left", + va="center", + fontsize=14, + fontweight="bold", + color=PURPLE, + zorder=4, + ) + bar_labels = ["Fair", "Safe", "Priv", "Robust"] + bar_vals = [0.92, 0.88, 0.95, 0.71] + bar_colors = [GREEN, GREEN, GREEN, ORANGE] + bx_start = left + 0.5 + bar_w = 3.6 + gap = 1.6 + base_y = sec_y3 - 6.2 + max_h = 4.5 + for i, (lbl, v, c) in enumerate(zip(bar_labels, bar_vals, bar_colors)): + bx = bx_start + i * (bar_w + gap) + ax.add_patch( + Rectangle( + (bx, base_y), + bar_w, + v * max_h, + facecolor=c, + edgecolor="none", + zorder=4, + ) + ) + ax.text( + bx + bar_w / 2, + base_y - 0.7, + lbl, + ha="center", + va="top", + fontsize=10, + color=MUTED, + zorder=4, + ) + ax.text( + bx + bar_w / 2, + base_y + v * max_h + 0.3, + f"{int(v * 100)}", + ha="center", + va="bottom", + fontsize=9.5, + color=TEXT, + zorder=4, + ) + + # Section 4: Remediation Guidance (right side, next to chart) + ax.text( + left + 24, + sec_y3, + "4. Remediation Guidance", + ha="left", + va="center", + fontsize=14, + fontweight="bold", + color=PURPLE, + zorder=4, + ) + for i, line in enumerate( + [ + "- Expand human-in-loop for low-confidence replies", + "- Add adversarial test set for robustness", + "- Document data-source provenance", + ] + ): + ax.text( + left + 24, + sec_y3 - 2 - i * 1.6, + line, + ha="left", + va="center", + fontsize=11, + color=TEXT, + zorder=4, + ) + + # Footer band + foot_y = page_y - page_h / 2 + ax.add_patch( + Rectangle( + (page_x - page_w / 2, foot_y), + page_w, + 2.0, + facecolor=LIGHTER_BG, + edgecolor="none", + zorder=3, + ) + ) + ax.text( + page_x, + foot_y + 1.0, + "Generated by AICertify v0.7.0 | Apache 2.0 | reproducible from contract.json", + ha="center", + va="center", + fontsize=10.5, + color=MUTED, + zorder=4, + ) + + save_fig(fig, "diagram5_report_anatomy.png") + + +# ---------- entrypoint ---------- +def main(): + print(f"Generating diagrams into {HERE}") + diagram1_hero_flow() + diagram2_architecture() + diagram3_regulatory_coverage() + diagram4_comparison() + diagram5_report_anatomy() + print("done.") + + +if __name__ == "__main__": + main() diff --git a/docs/INDEX.md b/docs/INDEX.md new file mode 100644 index 0000000..828d85d --- /dev/null +++ b/docs/INDEX.md @@ -0,0 +1,39 @@ +# AICertify Documentation + +> **Looking for an overview?** Start with the [README](../README.md) — it covers value prop, quickstart, regulatory coverage, and a comparison vs alternatives. + +The documentation here is organized along [Diátaxis](https://diataxis.fr/) lines: tutorials get you running, how-tos solve specific problems, reference describes the API, explanation explores design. + +## 🎓 Tutorials — get from zero to a working compliance report + +- [Quickstart](../examples/quickstart.py) — clone, install, run, inspect the report. ~5 minutes. +- [Sample contract](../examples/sample_contract.json) — the JSON shape a real contract takes. +- [Examples README](../examples/README.md) — index of all shipped examples and their pre-generated outputs. + +## 🛠️ How-To Guides — solve a specific problem + +- [Run a compliance check against a custom contract](../skills/evaluate-contract/SKILL.md) (Claude Code skill) +- [Add coverage for a new regulation](../skills/draft-policy/SKILL.md) (Claude Code skill) +- [Understand what a framework's policies enforce](../skills/explain-regulation/SKILL.md) (Claude Code skill) +- [Generate a report in PDF / Markdown / JSON / HTML](../examples/quickstart.py) — see the `report_format` argument. + +## 📚 Reference — look up specific names + +- [Python API](../aicertify/__init__.py) — the public surface re-exported from the package root. +- [CLI](../aicertify/cli.py) — `python -m aicertify.cli` flags and behavior. See also [README#cli](../README.md#cli). +- [Regulatory coverage table](../README.md#regulatory-coverage) — every framework with its policy count. +- [pyproject.toml](../pyproject.toml) — metadata, dependencies, entry points. +- [CHANGELOG](../CHANGELOG.md) — release history. + +## 💡 Explanation — understand the design + +- [Project overview](../PROJECT_OVERVIEW.md) — the long-form architectural narrative. +- [AGENTS.md](../AGENTS.md) — how AI coding agents (and humans) should work in the repo. +- [Why policy-as-code?](../README.md#why-aicertify) — the differentiation argument. + +## 🤝 Contributing & community + +- [CONTRIBUTING.md](../CONTRIBUTING.md) +- [CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md) +- [Issues](https://github.com/Principled-Evolution/aicertify/issues) +- [Sister project: gopal](https://github.com/Principled-Evolution/gopal) — the policy library AICertify consumes. diff --git a/poetry.lock b/poetry.lock index 2d5f0d5..5d804ec 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry 2.1.1 and should not be changed by hand. +# This file is automatically @generated by Poetry 2.0.1 and should not be changed by hand. [[package]] name = "absl-py" @@ -156,7 +156,7 @@ propcache = ">=0.2.0" yarl = ">=1.17.0,<2.0" [package.extras] -speedups = ["Brotli ; platform_python_implementation == \"CPython\"", "aiodns (>=3.2.0) ; sys_platform == \"linux\" or sys_platform == \"darwin\"", "brotlicffi ; platform_python_implementation != \"CPython\""] +speedups = ["Brotli", "aiodns (>=3.2.0)", "brotlicffi"] [[package]] name = "aiosignal" @@ -248,7 +248,7 @@ typing_extensions = {version = ">=4.5", markers = "python_version < \"3.13\""} [package.extras] doc = ["Sphinx (>=7.4,<8.0)", "packaging", "sphinx-autodoc-typehints (>=1.2.0)", "sphinx_rtd_theme"] -test = ["anyio[trio]", "coverage[toml] (>=7)", "exceptiongroup (>=1.2.0)", "hypothesis (>=4.0)", "psutil (>=5.9)", "pytest (>=7.0)", "trustme", "truststore (>=0.9.1) ; python_version >= \"3.10\"", "uvloop (>=0.21) ; platform_python_implementation == \"CPython\" and platform_system != \"Windows\" and python_version < \"3.14\""] +test = ["anyio[trio]", "coverage[toml] (>=7)", "exceptiongroup (>=1.2.0)", "hypothesis (>=4.0)", "psutil (>=5.9)", "pytest (>=7.0)", "trustme", "truststore (>=0.9.1)", "uvloop (>=0.21)"] trio = ["trio (>=0.26.1)"] [[package]] @@ -278,12 +278,12 @@ files = [ ] [package.extras] -benchmark = ["cloudpickle ; platform_python_implementation == \"CPython\"", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pympler", "pytest (>=4.3.0)", "pytest-codspeed", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pytest-xdist[psutil]"] -cov = ["cloudpickle ; platform_python_implementation == \"CPython\"", "coverage[toml] (>=5.3)", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pytest-xdist[psutil]"] -dev = ["cloudpickle ; platform_python_implementation == \"CPython\"", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pre-commit-uv", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pytest-xdist[psutil]"] +benchmark = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-codspeed", "pytest-mypy-plugins", "pytest-xdist[psutil]"] +cov = ["cloudpickle", "coverage[toml] (>=5.3)", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"] +dev = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pre-commit-uv", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"] docs = ["cogapp", "furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-towncrier", "towncrier (<24.7)"] -tests = ["cloudpickle ; platform_python_implementation == \"CPython\"", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pytest-xdist[psutil]"] -tests-mypy = ["mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\"", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.10\""] +tests = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"] +tests-mypy = ["mypy (>=1.11.1)", "pytest-mypy-plugins"] [[package]] name = "banks" @@ -583,7 +583,6 @@ description = "Foreign Function Interface for Python calling C code." optional = false python-versions = ">=3.8" groups = ["main"] -markers = "platform_python_implementation != \"CPython\" or sys_platform == \"linux\" or sys_platform == \"darwin\"" files = [ {file = "cffi-1.17.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14"}, {file = "cffi-1.17.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67"}, @@ -897,17 +896,17 @@ tqdm = ">=4.66.3" xxhash = "*" [package.extras] -audio = ["librosa", "soundfile (>=0.12.1)", "soxr (>=0.4.0) ; python_version >= \"3.9\""] +audio = ["librosa", "soundfile (>=0.12.1)", "soxr (>=0.4.0)"] benchmarks = ["tensorflow (==2.12.0)", "torch (==2.0.1)", "transformers (==4.30.1)"] -dev = ["Pillow (>=9.4.0)", "absl-py", "decorator", "decord (==0.6.0)", "elasticsearch (>=7.17.12,<8.0.0)", "faiss-cpu (>=1.8.0.post1)", "jax (>=0.3.14) ; sys_platform != \"win32\"", "jaxlib (>=0.3.14) ; sys_platform != \"win32\"", "joblib (<1.3.0)", "joblibspark", "librosa", "lz4", "moto[server]", "polars[timezone] (>=0.20.0)", "protobuf (<4.0.0)", "py7zr", "pyspark (>=3.4)", "pytest", "pytest-datadir", "pytest-xdist", "rarfile (>=4.0)", "ruff (>=0.3.0)", "s3fs", "s3fs (>=2021.11.1)", "soundfile (>=0.12.1)", "soundfile (>=0.12.1)", "soxr (>=0.4.0) ; python_version >= \"3.9\"", "sqlalchemy", "tensorflow (>=2.16.0) ; python_version >= \"3.10\"", "tensorflow (>=2.6.0)", "tensorflow (>=2.6.0) ; python_version < \"3.10\"", "tiktoken", "torch", "torch (>=2.0.0)", "torchdata", "transformers", "transformers (>=4.42.0)", "zstandard"] +dev = ["Pillow (>=9.4.0)", "absl-py", "decorator", "decord (==0.6.0)", "elasticsearch (>=7.17.12,<8.0.0)", "faiss-cpu (>=1.8.0.post1)", "jax (>=0.3.14)", "jaxlib (>=0.3.14)", "joblib (<1.3.0)", "joblibspark", "librosa", "lz4", "moto[server]", "polars[timezone] (>=0.20.0)", "protobuf (<4.0.0)", "py7zr", "pyspark (>=3.4)", "pytest", "pytest-datadir", "pytest-xdist", "rarfile (>=4.0)", "ruff (>=0.3.0)", "s3fs", "s3fs (>=2021.11.1)", "soundfile (>=0.12.1)", "soundfile (>=0.12.1)", "soxr (>=0.4.0)", "sqlalchemy", "tensorflow (>=2.16.0)", "tensorflow (>=2.6.0)", "tensorflow (>=2.6.0)", "tiktoken", "torch", "torch (>=2.0.0)", "torchdata", "transformers", "transformers (>=4.42.0)", "zstandard"] docs = ["s3fs", "tensorflow (>=2.6.0)", "torch", "transformers"] jax = ["jax (>=0.3.14)", "jaxlib (>=0.3.14)"] quality = ["ruff (>=0.3.0)"] s3 = ["s3fs"] tensorflow = ["tensorflow (>=2.6.0)"] tensorflow-gpu = ["tensorflow (>=2.6.0)"] -tests = ["Pillow (>=9.4.0)", "absl-py", "decorator", "decord (==0.6.0)", "elasticsearch (>=7.17.12,<8.0.0)", "faiss-cpu (>=1.8.0.post1)", "jax (>=0.3.14) ; sys_platform != \"win32\"", "jaxlib (>=0.3.14) ; sys_platform != \"win32\"", "joblib (<1.3.0)", "joblibspark", "librosa", "lz4", "moto[server]", "polars[timezone] (>=0.20.0)", "protobuf (<4.0.0)", "py7zr", "pyspark (>=3.4)", "pytest", "pytest-datadir", "pytest-xdist", "rarfile (>=4.0)", "s3fs (>=2021.11.1)", "soundfile (>=0.12.1)", "soundfile (>=0.12.1)", "soxr (>=0.4.0) ; python_version >= \"3.9\"", "sqlalchemy", "tensorflow (>=2.16.0) ; python_version >= \"3.10\"", "tensorflow (>=2.6.0) ; python_version < \"3.10\"", "tiktoken", "torch (>=2.0.0)", "torchdata", "transformers (>=4.42.0)", "zstandard"] -tests-numpy2 = ["Pillow (>=9.4.0)", "absl-py", "decorator", "decord (==0.6.0)", "elasticsearch (>=7.17.12,<8.0.0)", "jax (>=0.3.14) ; sys_platform != \"win32\"", "jaxlib (>=0.3.14) ; sys_platform != \"win32\"", "joblib (<1.3.0)", "joblibspark", "lz4", "moto[server]", "polars[timezone] (>=0.20.0)", "protobuf (<4.0.0)", "py7zr", "pyspark (>=3.4)", "pytest", "pytest-datadir", "pytest-xdist", "rarfile (>=4.0)", "s3fs (>=2021.11.1)", "soundfile (>=0.12.1)", "soundfile (>=0.12.1)", "soxr (>=0.4.0) ; python_version >= \"3.9\"", "sqlalchemy", "tiktoken", "torch (>=2.0.0)", "torchdata", "transformers (>=4.42.0)", "zstandard"] +tests = ["Pillow (>=9.4.0)", "absl-py", "decorator", "decord (==0.6.0)", "elasticsearch (>=7.17.12,<8.0.0)", "faiss-cpu (>=1.8.0.post1)", "jax (>=0.3.14)", "jaxlib (>=0.3.14)", "joblib (<1.3.0)", "joblibspark", "librosa", "lz4", "moto[server]", "polars[timezone] (>=0.20.0)", "protobuf (<4.0.0)", "py7zr", "pyspark (>=3.4)", "pytest", "pytest-datadir", "pytest-xdist", "rarfile (>=4.0)", "s3fs (>=2021.11.1)", "soundfile (>=0.12.1)", "soundfile (>=0.12.1)", "soxr (>=0.4.0)", "sqlalchemy", "tensorflow (>=2.16.0)", "tensorflow (>=2.6.0)", "tiktoken", "torch (>=2.0.0)", "torchdata", "transformers (>=4.42.0)", "zstandard"] +tests-numpy2 = ["Pillow (>=9.4.0)", "absl-py", "decorator", "decord (==0.6.0)", "elasticsearch (>=7.17.12,<8.0.0)", "jax (>=0.3.14)", "jaxlib (>=0.3.14)", "joblib (<1.3.0)", "joblibspark", "lz4", "moto[server]", "polars[timezone] (>=0.20.0)", "protobuf (<4.0.0)", "py7zr", "pyspark (>=3.4)", "pytest", "pytest-datadir", "pytest-xdist", "rarfile (>=4.0)", "s3fs (>=2021.11.1)", "soundfile (>=0.12.1)", "soundfile (>=0.12.1)", "soxr (>=0.4.0)", "sqlalchemy", "tiktoken", "torch (>=2.0.0)", "torchdata", "transformers (>=4.42.0)", "zstandard"] torch = ["torch"] vision = ["Pillow (>=9.4.0)"] @@ -971,7 +970,7 @@ files = [ wrapt = ">=1.10,<2" [package.extras] -dev = ["PyTest", "PyTest-Cov", "bump2version (<1)", "setuptools ; python_version >= \"3.12\"", "tox"] +dev = ["PyTest", "PyTest-Cov", "bump2version (<1)", "setuptools", "tox"] [[package]] name = "detoxify" @@ -1202,7 +1201,7 @@ files = [ [package.extras] docs = ["furo (>=2024.8.6)", "sphinx (>=8.1.3)", "sphinx-autodoc-typehints (>=3)"] testing = ["covdefaults (>=2.3)", "coverage (>=7.6.10)", "diff-cover (>=9.2.1)", "pytest (>=8.3.4)", "pytest-asyncio (>=0.25.2)", "pytest-cov (>=6)", "pytest-mock (>=3.14)", "pytest-timeout (>=2.3.1)", "virtualenv (>=20.28.1)"] -typing = ["typing-extensions (>=4.12.2) ; python_version < \"3.11\""] +typing = ["typing-extensions (>=4.12.2)"] [[package]] name = "filetype" @@ -1697,7 +1696,7 @@ httpcore = "==1.*" idna = "*" [package.extras] -brotli = ["brotli ; platform_python_implementation == \"CPython\"", "brotlicffi ; platform_python_implementation != \"CPython\""] +brotli = ["brotli", "brotlicffi"] cli = ["click (==8.*)", "pygments (==2.*)", "rich (>=10,<14)"] http2 = ["h2 (>=3,<5)"] socks = ["socksio (==1.*)"] @@ -1796,12 +1795,12 @@ files = [ zipp = ">=3.20" [package.extras] -check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1) ; sys_platform != \"cygwin\""] +check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1)"] cover = ["pytest-cov"] doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (>=3.5)", "sphinx-lint"] enabler = ["pytest-enabler (>=2.2)"] perf = ["ipython"] -test = ["flufl.flake8", "importlib-resources (>=1.3) ; python_version < \"3.9\"", "jaraco.test (>=5.4)", "packaging", "pyfakefs", "pytest (>=6,!=8.1.*)", "pytest-perf (>=0.9.2)"] +test = ["flufl.flake8", "importlib-resources (>=1.3)", "jaraco.test (>=5.4)", "packaging", "pyfakefs", "pytest (>=6,!=8.1.*)", "pytest-perf (>=0.9.2)"] type = ["pytest-mypy"] [[package]] @@ -2629,7 +2628,7 @@ files = [ [package.extras] develop = ["codecov", "pycodestyle", "pytest (>=4.6)", "pytest-cov", "wheel"] docs = ["sphinx"] -gmpy = ["gmpy2 (>=2.1.0a4) ; platform_python_implementation != \"PyPy\""] +gmpy = ["gmpy2 (>=2.1.0a4)"] tests = ["pytest (>=4.6)"] [[package]] @@ -3575,7 +3574,7 @@ docs = ["furo", "olefile", "sphinx (>=8.1)", "sphinx-copybutton", "sphinx-inline fpx = ["olefile"] mic = ["olefile"] tests = ["check-manifest", "coverage (>=7.4.2)", "defusedxml", "markdown2", "olefile", "packaging", "pyroma", "pytest", "pytest-cov", "pytest-timeout", "trove-classifiers (>=2024.10.12)"] -typing = ["typing-extensions ; python_version < \"3.10\""] +typing = ["typing-extensions"] xmp = ["defusedxml"] [[package]] @@ -3852,7 +3851,6 @@ description = "Python interface for c-ares" optional = false python-versions = ">=3.9" groups = ["main"] -markers = "sys_platform == \"linux\" or sys_platform == \"darwin\"" files = [ {file = "pycares-4.9.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:0b8bd9a3ee6e9bc990e1933dc7e7e2f44d4184f49a90fa444297ac12ab6c0c84"}, {file = "pycares-4.9.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:417a5c20861f35977240ad4961479a6778125bcac21eb2ad1c3aad47e2ff7fab"}, @@ -3945,7 +3943,6 @@ description = "C parser in Python" optional = false python-versions = ">=3.8" groups = ["main"] -markers = "platform_python_implementation != \"CPython\" or sys_platform == \"linux\" or sys_platform == \"darwin\"" files = [ {file = "pycparser-2.22-py3-none-any.whl", hash = "sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc"}, {file = "pycparser-2.22.tar.gz", hash = "sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6"}, @@ -3970,7 +3967,7 @@ typing-extensions = ">=4.12.2" [package.extras] email = ["email-validator (>=2.0.0)"] -timezone = ["tzdata ; python_version >= \"3.9\" and platform_system == \"Windows\""] +timezone = ["tzdata"] [[package]] name = "pydantic-ai" @@ -4831,7 +4828,7 @@ numpy = ">=1.23.5,<2.5" [package.extras] dev = ["cython-lint (>=0.12.2)", "doit (>=0.36.0)", "mypy (==1.10.0)", "pycodestyle", "pydevtool", "rich-click", "ruff (>=0.0.292)", "types-psutil", "typing_extensions"] doc = ["intersphinx_registry", "jupyterlite-pyodide-kernel", "jupyterlite-sphinx (>=0.16.5)", "jupytext", "matplotlib (>=3.5)", "myst-nb", "numpydoc", "pooch", "pydata-sphinx-theme (>=0.15.2)", "sphinx (>=5.0.0,<8.0.0)", "sphinx-copybutton", "sphinx-design (>=0.4.0)"] -test = ["Cython", "array-api-strict (>=2.0,<2.1.1)", "asv", "gmpy2", "hypothesis (>=6.30)", "meson", "mpmath", "ninja ; sys_platform != \"emscripten\"", "pooch", "pytest", "pytest-cov", "pytest-timeout", "pytest-xdist", "scikit-umfpack", "threadpoolctl"] +test = ["Cython", "array-api-strict (>=2.0,<2.1.1)", "asv", "gmpy2", "hypothesis (>=6.30)", "meson", "mpmath", "ninja", "pooch", "pytest", "pytest-cov", "pytest-timeout", "pytest-xdist", "scikit-umfpack", "threadpoolctl"] [[package]] name = "sentence-transformers" @@ -4991,13 +4988,13 @@ files = [ ] [package.extras] -check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1) ; sys_platform != \"cygwin\"", "ruff (>=0.8.0) ; sys_platform != \"cygwin\""] -core = ["importlib_metadata (>=6) ; python_version < \"3.10\"", "jaraco.functools (>=4)", "jaraco.text (>=3.7)", "more_itertools", "more_itertools (>=8.8)", "packaging (>=24.2)", "platformdirs (>=4.2.2)", "tomli (>=2.0.1) ; python_version < \"3.11\"", "wheel (>=0.43.0)"] +check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1)", "ruff (>=0.8.0)"] +core = ["importlib_metadata (>=6)", "jaraco.functools (>=4)", "jaraco.text (>=3.7)", "more_itertools", "more_itertools (>=8.8)", "packaging (>=24.2)", "platformdirs (>=4.2.2)", "tomli (>=2.0.1)", "wheel (>=0.43.0)"] cover = ["pytest-cov"] doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "pygments-github-lexers (==0.0.5)", "pyproject-hooks (!=1.1)", "rst.linker (>=1.9)", "sphinx (>=3.5)", "sphinx-favicon", "sphinx-inline-tabs", "sphinx-lint", "sphinx-notfound-page (>=1,<2)", "sphinx-reredirects", "sphinxcontrib-towncrier", "towncrier (<24.7)"] enabler = ["pytest-enabler (>=2.2)"] -test = ["build[virtualenv] (>=1.0.3)", "filelock (>=3.4.0)", "ini2toml[lite] (>=0.14)", "jaraco.develop (>=7.21) ; python_version >= \"3.9\" and sys_platform != \"cygwin\"", "jaraco.envs (>=2.2)", "jaraco.path (>=3.7.2)", "jaraco.test (>=5.5)", "packaging (>=24.2)", "pip (>=19.1)", "pyproject-hooks (!=1.1)", "pytest (>=6,!=8.1.*)", "pytest-home (>=0.5)", "pytest-perf ; sys_platform != \"cygwin\"", "pytest-subprocess", "pytest-timeout", "pytest-xdist (>=3)", "tomli-w (>=1.0.0)", "virtualenv (>=13.0.0)", "wheel (>=0.44.0)"] -type = ["importlib_metadata (>=7.0.2) ; python_version < \"3.10\"", "jaraco.develop (>=7.21) ; sys_platform != \"cygwin\"", "mypy (==1.14.*)", "pytest-mypy"] +test = ["build[virtualenv] (>=1.0.3)", "filelock (>=3.4.0)", "ini2toml[lite] (>=0.14)", "jaraco.develop (>=7.21)", "jaraco.envs (>=2.2)", "jaraco.path (>=3.7.2)", "jaraco.test (>=5.5)", "packaging (>=24.2)", "pip (>=19.1)", "pyproject-hooks (!=1.1)", "pytest (>=6,!=8.1.*)", "pytest-home (>=0.5)", "pytest-perf", "pytest-subprocess", "pytest-timeout", "pytest-xdist (>=3)", "tomli-w (>=1.0.0)", "virtualenv (>=13.0.0)", "wheel (>=0.44.0)"] +type = ["importlib_metadata (>=7.0.2)", "jaraco.develop (>=7.21)", "mypy (==1.14.*)", "pytest-mypy"] [[package]] name = "shellingham" @@ -5115,7 +5112,10 @@ files = [ ] [package.dependencies] -greenlet = {version = "!=0.4.17", optional = true, markers = "python_version < \"3.14\" and (platform_machine == \"aarch64\" or platform_machine == \"ppc64le\" or platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"win32\" or platform_machine == \"WIN32\") or extra == \"asyncio\""} +greenlet = [ + {version = "!=0.4.17", markers = "python_version < \"3.14\" and (platform_machine == \"aarch64\" or platform_machine == \"ppc64le\" or platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"win32\" or platform_machine == \"WIN32\")"}, + {version = "!=0.4.17", optional = true, markers = "python_version < \"3.14\" and (platform_machine == \"aarch64\" or platform_machine == \"ppc64le\" or platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"win32\" or platform_machine == \"WIN32\") or extra == \"asyncio\""}, +] typing-extensions = ">=4.6.0" [package.extras] @@ -5587,7 +5587,7 @@ files = [ ] [package.extras] -brotli = ["brotli (>=1.0.9) ; platform_python_implementation == \"CPython\"", "brotlicffi (>=0.8.0) ; platform_python_implementation != \"CPython\""] +brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)"] h2 = ["h2 (>=4,<5)"] socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"] zstd = ["zstandard (>=0.18.0)"] @@ -5609,7 +5609,7 @@ click = ">=7.0" h11 = ">=0.8" [package.extras] -standard = ["colorama (>=0.4) ; sys_platform == \"win32\"", "httptools (>=0.6.3)", "python-dotenv (>=0.13)", "pyyaml (>=5.1)", "uvloop (>=0.14.0,!=0.15.0,!=0.15.1) ; sys_platform != \"win32\" and sys_platform != \"cygwin\" and platform_python_implementation != \"PyPy\"", "watchfiles (>=0.13)", "websockets (>=10.4)"] +standard = ["colorama (>=0.4)", "httptools (>=0.6.3)", "python-dotenv (>=0.13)", "pyyaml (>=5.1)", "uvloop (>=0.14.0,!=0.15.0,!=0.15.1)", "watchfiles (>=0.13)", "websockets (>=10.4)"] [[package]] name = "vadersentiment" @@ -5645,7 +5645,7 @@ platformdirs = ">=3.9.1,<5" [package.extras] docs = ["furo (>=2023.7.26)", "proselint (>=0.13)", "sphinx (>=7.1.2,!=7.3)", "sphinx-argparse (>=0.4)", "sphinxcontrib-towncrier (>=0.2.1a0)", "towncrier (>=23.6)"] -test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess (>=1)", "flaky (>=3.7)", "packaging (>=23.1)", "pytest (>=7.4)", "pytest-env (>=0.8.2)", "pytest-freezer (>=0.4.8) ; platform_python_implementation == \"PyPy\" or platform_python_implementation == \"GraalVM\" or platform_python_implementation == \"CPython\" and sys_platform == \"win32\" and python_version >= \"3.13\"", "pytest-mock (>=3.11.1)", "pytest-randomly (>=3.12)", "pytest-timeout (>=2.1)", "setuptools (>=68)", "time-machine (>=2.10) ; platform_python_implementation == \"CPython\""] +test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess (>=1)", "flaky (>=3.7)", "packaging (>=23.1)", "pytest (>=7.4)", "pytest-env (>=0.8.2)", "pytest-freezer (>=0.4.8)", "pytest-mock (>=3.11.1)", "pytest-randomly (>=3.12)", "pytest-timeout (>=2.1)", "setuptools (>=68)", "time-machine (>=2.10)"] [[package]] name = "wrapt" @@ -6006,11 +6006,11 @@ files = [ ] [package.extras] -check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1) ; sys_platform != \"cygwin\""] +check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1)"] cover = ["pytest-cov"] doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (>=3.5)", "sphinx-lint"] enabler = ["pytest-enabler (>=2.2)"] -test = ["big-O", "importlib-resources ; python_version < \"3.9\"", "jaraco.functools", "jaraco.itertools", "jaraco.test", "more-itertools", "pytest (>=6,!=8.1.*)", "pytest-ignore-flaky"] +test = ["big-O", "importlib-resources", "jaraco.functools", "jaraco.itertools", "jaraco.test", "more-itertools", "pytest (>=6,!=8.1.*)", "pytest-ignore-flaky"] type = ["pytest-mypy"] [[package]] @@ -6129,4 +6129,4 @@ cffi = ["cffi (>=1.11)"] [metadata] lock-version = "2.1" python-versions = ">=3.12,<3.13" -content-hash = "7ddb9a0a8836a3e7a19b830ef048f4438a27c02e2f689675ff1e5f7e669424d8" +content-hash = "3b9e9d56a2e7bb57ecaa53e453327f26d42df359dd5bcb2247124e6f9cfab98a" diff --git a/pyproject.toml b/pyproject.toml index 2482142..f80bf96 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,84 +1,110 @@ -[project] -name = "aicertify" -version = "0.7.0" -description = "AICertify is a standardized framework for validating and certifying AI applications against regulatory, compliance, and operational (AIOps) requirements. It leverages declarative policies via Open Policy Agent (OPA)—to ensure AI systems meet cost, fairness, validation, and acceptance criteria." -authors = [ - {name = "kmadan",email = "kapil.madan@gmail.com"} -] -license = {text = "\"Apache-2.0\""} -readme = "README.md" -requires-python = ">=3.12,<3.13" -dependencies = [ - "langfair @ git+https://github.com/mantric/langfair-mantric.git@python-3.12-support", - "fastapi>=0.115.8,<0.116.0", - "uvicorn>=0.34.0,<0.35.0", - "opa-python-client>=0.1.0", - "requests>=2.32.3,<3.0.0", - "python-dotenv>=1.0.0", - "pandas>=2.2.0", - "langchain-openai>=0.0.5", - "pydantic-ai (>=0.0.24,<0.0.25)", - "markdown (>=3.7,<4.0)", - "reportlab (>=4.3.1,<5.0.0)", - "yfinance (>=0.2.54,<0.3.0)", - "pytest (>=8.3.4,<9.0.0)", - "datasets (>=3.3.2,<4.0.0)", - "huggingface-hub (>=0.29.1,<0.30.0)", - "deepeval (>=2.4.8,<3.0.0)", - "colorlog (>=6.9.0,<7.0.0)", - "pydantic (>=2.10.6,<3.0.0)", - "rich (>=13.9.4,<14.0.0)", - "black (>=25.1.0,<26.0.0)", - "h11>=0.14.0", - "torch>=2.7.0", - "transformers>=4.50.0", - "setuptools>=78.1.1" -] - - -[build-system] -requires = ["poetry-core>=2.0.0,<3.0.0"] -build-backend = "poetry.core.masonry.api" - -[tool.poetry] -name = "aicertify" -version = "0.1.0" -description = "A framework for validating and certifying AI applications" -authors = ["kmadan "] -license = "Apache-2.0" -readme = "README.md" -repository = "https://github.com/mantric/aicertify" - -# ✅ This ensures Poetry installs the package correctly -packages = [{ include = "aicertify" }] - -[tool.poetry.dependencies] -python = ">=3.12,<3.13" -fastapi = ">=0.115.8,<0.116.0" -uvicorn = ">=0.34.0,<0.35.0" -requests = ">=2.32.3,<3.0.0" -opa-python-client = ">=0.1.0" -langfair = {git = "https://github.com/mantric/langfair-mantric.git", branch = "python-3.12-support"} -python-dotenv = ">=1.0.0" -pandas = ">=2.2.0" -langchain-openai = ">=0.0.5" - -[tool.poetry.group.dev.dependencies] -pytest-asyncio = "^0.25.3" -ruff = "^0.5.5" -pre-commit = "^4.2.0" -black = ">=25.1.0,<26.0.0" - -[tool.ruff] -line-length = 88 # Match black's default -target-version = "py312" -select = ["E", "F", "I"] -ignore = [] - -[tool.ruff.per-file-ignores] -"__init__.py" = ["F401"] - -[tool.black] -line-length = 88 # Default black line length -target-version = ["py312"] -include = '\.pyi?$' +[project] +name = "aicertify" +version = "0.7.0" +description = "Compliance-as-code for AI systems. Audit your AI against the EU AI Act, NIST AI RMF, and 13+ regulatory frameworks using Open Policy Agent (OPA) — and produce audit-ready PDF, Markdown, JSON, or HTML reports." +authors = [ + {name = "Kapil Madan", email = "kapil.madan@gmail.com"}, + {name = "Principled Evolution", email = "info@principledevolution.ai"}, +] +license = "Apache-2.0" +readme = "README.md" +requires-python = ">=3.12,<3.13" +keywords = [ + "ai-governance", + "ai-compliance", + "ai-act", + "eu-ai-act", + "nist-ai-rmf", + "responsible-ai", + "open-policy-agent", + "opa", + "rego", + "regulatory-compliance", + "ai-audit", + "ai-safety", + "fairness", + "policy-as-code", + "ai-evaluation", + "ai-certification", +] +classifiers = [ + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "Intended Audience :: Information Technology", + "Intended Audience :: Legal Industry", + "Intended Audience :: Science/Research", + "License :: OSI Approved :: Apache Software License", + "Operating System :: OS Independent", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.12", + "Topic :: Scientific/Engineering :: Artificial Intelligence", + "Topic :: Security", + "Topic :: Software Development :: Quality Assurance", + "Topic :: Software Development :: Testing", +] +dependencies = [ + "langfair @ git+https://github.com/mantric/langfair-mantric.git@python-3.12-support", + "fastapi>=0.115.8,<0.116.0", + "uvicorn>=0.34.0,<0.35.0", + "opa-python-client>=0.1.0", + "requests>=2.32.3,<3.0.0", + "python-dotenv>=1.0.0", + "pandas>=2.2.0", + "langchain-openai>=0.0.5", + "pydantic-ai (>=0.0.24,<0.0.25)", + "markdown (>=3.7,<4.0)", + "reportlab (>=4.3.1,<5.0.0)", + "yfinance (>=0.2.54,<0.3.0)", + "pytest (>=8.3.4,<9.0.0)", + "datasets (>=3.3.2,<4.0.0)", + "huggingface-hub (>=0.29.1,<0.30.0)", + "deepeval (>=2.4.8,<3.0.0)", + "colorlog (>=6.9.0,<7.0.0)", + "pydantic (>=2.10.6,<3.0.0)", + "rich (>=13.9.4,<14.0.0)", + "black (>=25.1.0,<26.0.0)", + "h11>=0.14.0", + "torch>=2.7.0", + "transformers>=4.50.0", + "setuptools>=78.1.1", + "pycares (>=4.9.0,<5.0.0)", +] + +[project.urls] +Homepage = "https://github.com/Principled-Evolution/aicertify" +Repository = "https://github.com/Principled-Evolution/aicertify" +Documentation = "https://github.com/Principled-Evolution/aicertify#readme" +Issues = "https://github.com/Principled-Evolution/aicertify/issues" +Changelog = "https://github.com/Principled-Evolution/aicertify/blob/main/CHANGELOG.md" +"Policy Library (gopal)" = "https://github.com/Principled-Evolution/gopal" + +[project.scripts] +aicertify = "aicertify.cli:main" + +[build-system] +requires = ["poetry-core>=2.0.0,<3.0.0"] +build-backend = "poetry.core.masonry.api" + +[tool.poetry] +packages = [{ include = "aicertify" }] + +[tool.poetry.group.dev.dependencies] +pytest-asyncio = "^0.25.3" +ruff = "^0.5.5" +pre-commit = "^4.2.0" +black = ">=25.1.0,<26.0.0" + +[tool.ruff] +line-length = 88 +target-version = "py312" +select = ["E", "F", "I"] +ignore = [] + +[tool.ruff.per-file-ignores] +"__init__.py" = ["F401"] + +[tool.black] +line-length = 88 +target-version = ["py312"] +include = '\.pyi?$' diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 0000000..c932adc --- /dev/null +++ b/skills/README.md @@ -0,0 +1,33 @@ +# AICertify Claude Code Skills + +This directory ships [Claude Code](https://docs.claude.com/en/docs/claude-code/) skills for working with AICertify. Each subdirectory contains a `SKILL.md` file that Claude Code can invoke as a slash command. + +## Available skills + +| Skill | What it does | +|---|---| +| [`run-compliance-check`](run-compliance-check/SKILL.md) | Run the end-to-end quickstart and surface the generated report | +| [`evaluate-contract`](evaluate-contract/SKILL.md) | Evaluate a user-supplied contract JSON against any supported framework | +| [`explain-regulation`](explain-regulation/SKILL.md) | Walk every policy in a framework directory and explain what it checks | +| [`draft-policy`](draft-policy/SKILL.md) | Scaffold a new Rego policy with metadata, default rule, and a test sibling | + +## Installation + +Skills under `skills/` in this repository are intended to be installed into Claude Code's skills directory. The simplest way: + +```bash +# From the repo root +mkdir -p ~/.claude/skills +cp -r skills/* ~/.claude/skills/ +``` + +Restart Claude Code. The skills appear as slash commands: + +``` +/run-compliance-check +/evaluate-contract path/to/contract.json eu_ai_act pdf +/explain-regulation eu_ai_act +/draft-policy international eu_ai_act my_new_rule +``` + +A first-class `aicertify install` command is on the roadmap to register skills, MCP, and editor hooks in one step. diff --git a/skills/draft-policy/SKILL.md b/skills/draft-policy/SKILL.md new file mode 100644 index 0000000..a7686d7 --- /dev/null +++ b/skills/draft-policy/SKILL.md @@ -0,0 +1,90 @@ +--- +name: draft-policy +description: Scaffold a new Rego policy file (with metadata, default rule, and a test sibling) for AICertify, following the project's authoring conventions. Use this when the user wants to add coverage for a new regulation, article, or organizational rule. +argument-hint: " [--upstream]" +--- + +# Draft Policy + +Generate a new Rego policy + test file matching AICertify's conventions. + +## Inputs + +- **domain** — One of: `global`, `international`, `industry_specific`, `operational`. +- **framework** — Existing framework name (e.g. `eu_ai_act`) OR a new one. If new, prompt the user to confirm before creating the directory. +- **policy_name** — `snake_case` filename without `.rego` suffix. +- **--upstream** — Optional flag. If set, draft into [gopal](https://github.com/Principled-Evolution/gopal) instead of the vendored copy. **Strongly preferred** for any policy that should ship to all gopal consumers. + +## Steps + +1. **Decide repo** — if `--upstream`, target `/home/kapil/Projects/gopal/`. Otherwise the vendored copy at `aicertify/opa_policies/`. **Default recommendation: write upstream and vendor in.** + +2. **Confirm directory layout** — + ``` + //v1/.rego + //v1/_test.rego + ``` + Create the directory if it doesn't exist. + +3. **Write the policy file** with this skeleton — fill in title, description, and source from the user's input: + + ```rego + package ..v1. + + import data.helper_functions.reporting + + # METADATA + # title: + # description: + # version: 1 + # source: + + default allow := false + + allow if { + # TODO: encode the rule. Reference fields like: + # input.system.name + # input.system. + true + } + + report := reporting.compose_report( + ".", + allow, + [{"name": "", "value": allow, "control_passed": allow}], + ) + ``` + +4. **Write the test file**: + + ```rego + package ..v1._test + + import data...v1. + + test_allow_when_compliant if { + .allow with input as { + "system": { + # TODO: shape the compliant input + } + } + } + + test_deny_by_default if { + not .allow with input as {} + } + ``` + +5. **Update the framework README** at `//v1/README.md` to list the new policy. If the README doesn't exist, scaffold it with the standard source + disclaimer template. + +6. **Validate** — run from the target repo root: + ```bash + opa check --ignore custom/ . + regal lint --ignore-files custom/ . + ``` + Surface any errors and offer to fix them. + +7. **Remind the user**: + - Fill in the TODO blocks with the actual rule logic and input shape. + - Add `# METADATA` with a real `source:` URL — auditors will read this. + - If drafted upstream in gopal, the next AICertify vendor sync will pull it in. diff --git a/skills/evaluate-contract/SKILL.md b/skills/evaluate-contract/SKILL.md new file mode 100644 index 0000000..ef4c582 --- /dev/null +++ b/skills/evaluate-contract/SKILL.md @@ -0,0 +1,48 @@ +--- +name: evaluate-contract +description: Evaluate a user-provided AI application contract JSON against a chosen regulation and generate a compliance report. Use this when the user has their own contract file and wants to run AICertify against a specific framework. +argument-hint: " [report-format]" +--- + +# Evaluate Contract + +Run AICertify against a user-supplied contract file and produce a report. + +## Inputs + +The user provides: +- **contract** — Path to a contract JSON. See `examples/sample_contract.json` for the schema. +- **framework** — One of: `eu_ai_act`, `nist`, `india`, `brazil`, `healthcare`, `bfs`, `automotive`, `aviation`, `education`. Map this to the policy directory under `aicertify/opa_policies/`. +- **report-format** — Optional. `pdf` (default), `markdown`, `json`, `html`. + +## Steps + +1. **Validate the contract** — Read the file, confirm it parses as JSON and contains at minimum: `application.name`, `model`, `interactions`. If any are missing, stop and tell the user which fields are absent. + +2. **Resolve the policy path** — Look in `aicertify/opa_policies/` for the framework directory. Most frameworks live at: + - `international//v1/` + - `industry_specific//v1/` + If the framework directory doesn't exist or is marked as a placeholder, surface that to the user before running. + +3. **Run the evaluation**: + ```bash + python -m aicertify.cli \ + --contract \ + --policy \ + --report-format \ + --output-dir reports/ + ``` + +4. **Locate and summarize the report**: + - Identify the new file in `reports/`. + - Print: framework evaluated, headline pass/fail, count of policies evaluated, count of failures. + - If the report is JSON, parse and surface the top 3 failing rules with their `deny[msg]` strings. + +5. **Suggest next actions** if there are failures: + - For each top failure, identify the corresponding `.rego` file and quote the rule. + - Recommend either fixing the AI application input or filing an issue if the policy is wrong. + +## Notes + +- Never edit `aicertify/opa_policies/*.rego` files in this repo without considering whether the change should land upstream in [gopal](https://github.com/Principled-Evolution/gopal) first. They are a vendored copy. +- If the user's contract is missing captured interactions, point them at `examples/sample_contract.json` for the shape. diff --git a/skills/explain-regulation/SKILL.md b/skills/explain-regulation/SKILL.md new file mode 100644 index 0000000..ee075e1 --- /dev/null +++ b/skills/explain-regulation/SKILL.md @@ -0,0 +1,51 @@ +--- +name: explain-regulation +description: Walk through every Rego policy under a given regulatory framework and explain in plain English what each one checks. Use this when the user wants to understand what coverage a framework actually provides before running it. +argument-hint: "" +--- + +# Explain Regulation + +Produce a plain-English audit-grade summary of what a regulatory framework's policies enforce. + +## Steps + +1. **Resolve the framework directory** — locate it under `aicertify/opa_policies/`: + - `international//v1/` + - `industry_specific//v1/` + - `operational//v1/` + + If not found, list available frameworks instead and stop. + +2. **Enumerate policies** — list every `.rego` file in the directory (excluding `*_test.rego`). + +3. **For each policy**: + - Read the file. + - Extract the `# METADATA` block (title, description, source link). + - Identify the `default allow` (or equivalent) and the conditions under which it returns `true`. + - Translate the rule into one or two plain-English sentences. + +4. **Produce a structured output**: + + ``` + ## Framework: + + **** — + - **What it checks**: + - **Source**: + - **Status**: ✅ active | 🚧 placeholder + + + + ## Coverage Summary + - Policies: N + - Articles covered: + - Gaps: + ``` + +5. **Surface placeholders** — if a policy file has an empty body, only a default rule, or comments saying "TODO/Pending", explicitly flag it as placeholder in the output. Honesty here matters; auditors will read this summary. + +## Notes + +- Cross-link to the upstream [gopal](https://github.com/Principled-Evolution/gopal) policy if it exists. The `.rego` files in this repo are a vendored copy. +- Don't editorialize about whether the regulation is "good" — just describe what the policies check. diff --git a/skills/run-compliance-check/SKILL.md b/skills/run-compliance-check/SKILL.md new file mode 100644 index 0000000..0d76bf7 --- /dev/null +++ b/skills/run-compliance-check/SKILL.md @@ -0,0 +1,42 @@ +--- +name: run-compliance-check +description: Run the AICertify end-to-end quickstart against the EU AI Act policy set and inspect the generated compliance report. Use this when the user wants to verify AICertify works in their environment, see a sample audit deliverable, or smoke-test after a change. +argument-hint: "[optional: report-format pdf|markdown|json|html]" +--- + +# Run Compliance Check + +Execute the canonical AICertify quickstart and produce a real compliance report. + +## Steps + +1. **Verify environment** — confirm the user is in the AICertify repo root and `pip install -e .` has run. If not, instruct them once. + +2. **Choose the report format** — default to `pdf`. If the user passed an argument, honor it. + +3. **Run the quickstart**: + ```bash + python examples/quickstart.py + ``` + This wires a sample AI application through the EU AI Act policy set. + +4. **Locate the output** — reports land in `reports/` (relative to repo root). List the directory and identify the new files by timestamp. + +5. **Summarize for the user**: + - Path of the generated report(s) + - File size + format + - One-sentence summary of the headline pass/fail result (parse the JSON report if present, else open the markdown header) + - Inline a 5-line excerpt of the report's Executive Summary if available + +6. **Surface anything broken** — if the run errors, capture the stack trace, identify the failing module, and suggest the most likely fix (missing dep, OPA binary not in PATH, Python version mismatch). Do not silently retry. + +## Notes + +- The quickstart accumulates artifacts in `reports/`. If the directory gets large, suggest archiving but do not delete without confirmation. +- For CLI-style usage instead of the quickstart, run: + ```bash + python -m aicertify.cli \ + --contract examples/sample_contract.json \ + --policy aicertify/opa_policies/international/eu_ai_act/v1 \ + --report-format pdf + ```