Skip to content

Commit 66198c3

Browse files
varun369claude
varun369
and
claude
committed
feat: AgentAssert v0.1.0 — Formal Behavioral Contracts for AI Agents
309 tests, 293 benchmarks (100% across 12 domains), 14 operators, 3 framework adapters (LangGraph, CrewAI, OpenAI Agents SDK), SPRT certification, compositional safety proofs, drift detection. Paper: arXiv:2602.22302 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
0 parents  commit 66198c3

407 files changed

Lines changed: 23024 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
name: Bug Report
3+
about: Report a bug in AgentAssert
4+
title: "[BUG] "
5+
labels: bug
6+
assignees: ""
7+
---
8+
9+
## Description
10+
11+
A clear description of the bug.
12+
13+
## Steps to Reproduce
14+
15+
1. ...
16+
2. ...
17+
3. ...
18+
19+
## Expected Behavior
20+
21+
What should happen.
22+
23+
## Actual Behavior
24+
25+
What actually happens. Include error messages and tracebacks.
26+
27+
## Environment
28+
29+
- **Python version:** (e.g., 3.12.4)
30+
- **agentassert-abc version:** (e.g., 0.1.0)
31+
- **OS:** (e.g., macOS 15, Ubuntu 24.04)
32+
- **Installed extras:** (e.g., `yaml,math`)
33+
34+
## Contract YAML (if relevant)
35+
36+
```yaml
37+
# Paste your contract here
38+
```
39+
40+
## Minimal Reproduction
41+
42+
```python
43+
# Paste minimal code that reproduces the issue
44+
```

‎.github/ISSUE_TEMPLATE/contract_request.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
name: Contract Request
3+
about: Request a new domain contract
4+
title: "[CONTRACT] "
5+
labels: contract
6+
assignees: ""
7+
---
8+
9+
## Domain
10+
11+
What domain is this contract for? (e.g., legal advisory, autonomous driving)
12+
13+
## Description
14+
15+
What does the agent do in this domain?
16+
17+
## Hard Constraints (non-negotiable safety rules)
18+
19+
List the rules that must never be violated:
20+
21+
1. ...
22+
2. ...
23+
24+
## Soft Constraints (quality goals)
25+
26+
List the quality metrics to track:
27+
28+
1. ...
29+
2. ...
30+
31+
## Example Agent Behavior
32+
33+
Describe a typical agent interaction in this domain and what should/shouldn't happen.

‎.github/ISSUE_TEMPLATE/feature_request.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
name: Feature Request
3+
about: Suggest a new feature or enhancement
4+
title: "[FEATURE] "
5+
labels: enhancement
6+
assignees: ""
7+
---
8+
9+
## Problem Statement
10+
11+
What problem does this feature solve?
12+
13+
## Proposed Solution
14+
15+
How should it work?
16+
17+
## Alternatives Considered
18+
19+
What other approaches did you consider?
20+
21+
## Domain
22+
23+
Which part of AgentAssert does this affect?
24+
25+
- [ ] ContractSpec DSL (parser, operators, schema)
26+
- [ ] Evaluator (constraint evaluation engine)
27+
- [ ] Metrics (drift, compliance, Theta)
28+
- [ ] Monitor (session monitoring, recovery)
29+
- [ ] Certification (SPRT, composition)
30+
- [ ] Integration (framework adapters)
31+
- [ ] Benchmark (scenarios, runner)
32+
- [ ] Documentation
33+
- [ ] Other

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
## Description
2+
3+
What does this PR do?
4+
5+
## Type
6+
7+
- [ ] Bug fix
8+
- [ ] New feature
9+
- [ ] New domain contract
10+
- [ ] Documentation
11+
- [ ] Refactor
12+
- [ ] CI/CD
13+
14+
## Checklist
15+
16+
- [ ] Tests pass: `uv run pytest`
17+
- [ ] Lint clean: `uv run ruff check src/ tests/`
18+
- [ ] Types clean: `uv run basedpyright`
19+
- [ ] New files have copyright header
20+
- [ ] No breaking changes to public API
21+
- [ ] Documentation updated (if applicable)

‎.github/workflows/benchmarks.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Benchmarks
2+
3+
on:
4+
workflow_dispatch:
5+
6+
jobs:
7+
run-benchmarks:
8+
name: Run benchmarks
9+
runs-on: ubuntu-latest
10+
11+
steps:
12+
- uses: actions/checkout@v4
13+
14+
- uses: astral-sh/setup-uv@v4
15+
with:
16+
enable-cache: true
17+
18+
- name: Set up Python
19+
uses: actions/setup-python@v5
20+
with:
21+
python-version: "3.12"
22+
23+
- name: Install dependencies
24+
run: uv sync --group dev
25+
26+
- name: Run benchmarks
27+
run: uv run python benchmarks/runner.py
28+
29+
- name: Upload results
30+
uses: actions/upload-artifact@v4
31+
with:
32+
name: benchmark-results-${{ github.sha }}
33+
path: benchmarks/results/
34+
retention-days: 90

‎.github/workflows/ci.yml‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
concurrency:
10+
group: ci-${{ github.ref }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
lint-type-test:
15+
name: Python ${{ matrix.python-version }}
16+
runs-on: ubuntu-latest
17+
strategy:
18+
fail-fast: false
19+
matrix:
20+
python-version: ["3.12", "3.13", "3.14"]
21+
22+
steps:
23+
- uses: actions/checkout@v4
24+
25+
- uses: astral-sh/setup-uv@v4
26+
with:
27+
enable-cache: true
28+
29+
- name: Set up Python ${{ matrix.python-version }}
30+
uses: actions/setup-python@v5
31+
with:
32+
python-version: ${{ matrix.python-version }}
33+
34+
- name: Install dependencies
35+
run: uv sync --group dev
36+
37+
- name: Lint (ruff)
38+
run: uv run ruff check src/ tests/
39+
40+
- name: Type check (basedpyright)
41+
run: uv run basedpyright
42+
43+
- name: Test with coverage
44+
run: uv run pytest --cov=agentassert_abc --cov-report=xml -q
45+
46+
- name: Upload coverage to Codecov
47+
if: matrix.python-version == '3.12' && always()
48+
uses: codecov/codecov-action@v4
49+
with:
50+
files: coverage.xml
51+
fail_ci_if_error: false
52+
env:
53+
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

‎.github/workflows/publish-npm.yml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
name: Publish to npm
2+
3+
on:
4+
push:
5+
tags:
6+
- "v*"
7+
8+
jobs:
9+
publish:
10+
name: Publish npm placeholder
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v4
15+
16+
- uses: actions/setup-node@v4
17+
with:
18+
node-version: "20"
19+
registry-url: "https://registry.npmjs.org"
20+
21+
- name: Update package.json version from tag
22+
working-directory: claim/npm
23+
run: |
24+
TAG="${GITHUB_REF#refs/tags/v}"
25+
echo "Setting npm package version to ${TAG}"
26+
npm version "${TAG}" --no-git-tag-version --allow-same-version
27+
28+
- name: Publish to npm
29+
working-directory: claim/npm
30+
run: npm publish --access public
31+
env:
32+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

‎.github/workflows/publish-pypi.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Publish to PyPI
2+
3+
on:
4+
push:
5+
tags:
6+
- "v*"
7+
8+
permissions:
9+
id-token: write
10+
contents: read
11+
12+
jobs:
13+
publish:
14+
name: Build & Publish to PyPI
15+
runs-on: ubuntu-latest
16+
environment:
17+
name: pypi
18+
url: https://pypi.org/p/agentassert-abc
19+
20+
steps:
21+
- uses: actions/checkout@v4
22+
23+
- uses: astral-sh/setup-uv@v4
24+
with:
25+
enable-cache: true
26+
27+
- name: Set up Python
28+
uses: actions/setup-python@v5
29+
with:
30+
python-version: "3.12"
31+
32+
- name: Build package
33+
run: uv build
34+
35+
- name: Publish to PyPI (Trusted Publishers / OIDC)
36+
uses: pypa/gh-action-pypi-publish@release/v1
37+
with:
38+
# Falls back to token auth if Trusted Publishers not configured
39+
password: ${{ secrets.PYPI_TOKEN }}
40+
skip-existing: true

‎.gitignore‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Python-generated files
2+
__pycache__/
3+
*.py[cod]
4+
*.pyo
5+
build/
6+
dist/
7+
wheels/
8+
*.egg-info/
9+
10+
# Virtual environments
11+
.venv/
12+
venv/
13+
14+
# Environment / secrets
15+
.env
16+
.env.*
17+
*.env
18+
19+
# Test/lint artifacts
20+
.coverage
21+
htmlcov/
22+
.hypothesis/
23+
.pytest_cache/
24+
.ruff_cache/
25+
26+
# Benchmark outputs (regenerated per run; live_benchmark.py has Azure endpoints)
27+
benchmarks/live_results.json
28+
benchmarks/live_benchmark.py
29+
30+
# npm claim artifacts (not for public repo)
31+
claim/
32+
33+
# Internal dev files (never commit)
34+
.backup/
35+
36+
# Blockchain timestamp proofs (keep in .backup only)
37+
*.ots
38+
39+
# Lock file — library best practice: do NOT commit uv.lock for libraries
40+
# (applications should commit it; libraries should not, to avoid pinning transitive deps)
41+
uv.lock
42+
43+
# IDE
44+
.idea/
45+
.vscode/
46+
47+
# OS
48+
.DS_Store
49+
Thumbs.db

‎.npmignore‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# This .npmignore covers the npm placeholder package (claim/npm/)
2+
# Ensure nothing sensitive leaks into the npm registry.
3+
4+
.env
5+
.env.*
6+
*.env
7+
.backup/
8+
.venv/
9+
venv/
10+
__pycache__/
11+
*.pyc
12+
*.pyo
13+
*.egg-info/
14+
dist/
15+
build/
16+
tests/
17+
benchmarks/
18+
claim/
19+
.coverage
20+
htmlcov/
21+
.hypothesis/
22+
.pytest_cache/
23+
.ruff_cache/
24+
*.ots
25+
uv.lock
26+
.DS_Store
27+
.idea/
28+
.vscode/

0 commit comments

Comments
 (0)