Production monitoring and kill-switch platform for AI agents that automatically detects and stops misbehaving LLM workloads before they burn your budget.
Quick Start • Features • Examples • Contributing
AI Circuit Breaker is a production monitoring platform that automatically detects and terminates malfunctioning AI workloads to prevent cost overruns and system instability. It provides real-time anomaly detection and kill-switch capabilities for LLM-based applications.
$ python -m ai_circuit_breaker --config config.yaml
[INFO] Starting monitoring service...
[INFO] Connected to 3 LLM endpoints
[INFO] Thresholds: cost > $10/hr, latency > 5s, toxicity > 0.8
Companies running AI agents and LLM systems in production have no reliable way to automatically shut down misbehaving AI that's causing runaway costs latency spikes or inappropriate outputs. Current monitoring tools weren't built for the unique failure modes of AI systems leaving teams scrambling to manually intervene during incidents.
| Feature | Description |
|---|---|
| Real-time Cost Monitoring | Tracks token usage and API spend across all LLM providers with configurable alert thresholds |
| Latency Anomaly Detection | Identifies abnormal response time patterns using statistical modeling to catch degradation early |
| Output Toxicity Scoring | Analyzes LLM outputs for harmful content using integrated safety classifiers |
| Automatic Kill Switch | Terminates problematic workloads instantly when thresholds are exceeded |
| Multi-Provider Support | Works with OpenAI Anthropic Azure and self-hosted models via unified API |
| Dashboard Visualization | Provides real-time metrics and historical trends through an intuitive web interface |
| Custom Policy Engine | Allows defining complex rules combining cost latency and safety metrics |
| Audit Logging | Records all interventions and decisions for compliance and debugging |
- Clone the repository:
git clone https://github.com/your-org/ai-circuit-breaker.git - Install backend dependencies:
cd ai-circuit-breaker && uv pip install -r backend/requirements.txt - Install frontend dependencies:
cd frontend && bun install - Set environment variables:
cp .env.example .env && edit .env to add ANTHROPIC_API_KEY - Start the application:
../init.sh(from project root) or manually:- Backend:
uvicorn backend.main:app --reload --port 8000 - Frontend:
bun run dev --port 3000
- Backend:
- Open http://localhost:3000 to access the dashboard
Verify both services are running:
$ curl http://localhost:8000/api/health
{"status":"ok","timestamp":"2024-01-15T10:30:00Z","version":"0.1.0"}
$ curl http://localhost:3000
<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><title>AI Circuit Breaker</title>
Configure a monthly budget limit for GPT-4 usage:
$ curl -X POST http://localhost:8000/api/policies \
-H "Content-Type: application/json" \
-d '{"name":"gpt4_monthly_limit","conditions":[{"metric":"cost","operator":" > ","value":500,"window":"30d"}],"action":"terminate","enabled":true}'
{"id":"pol_123","name":"gpt4_monthly_limit","conditions":[{"metric":"cost","operator":" > ","value":500,"window":"30d"}],"action":"terminate","enabled":true}
Trigger a kill switch by exceeding latency threshold:
$ curl -X POST http://localhost:8000/api/simulate/anomaly \
-H "Content-Type: application/json" \
-d '{"type":"latency","value":8.5,"duration":"5m"}'
{"alert_id":"alt_456","triggered":true,"action_taken":"terminate","affected_endpoints":["gpt-4","claude-2"]}
AI Circuit Breaker/ backend/ # FastAPI server and core logic main.py # Application entry point routes.py # API endpoint definitions models.py # Pydantic data models database.py # Database connection and setup frontend/ # React Vite application src/ components/ # Reusable UI components (tables forms alerts) lib/ # API client and utility functions pages/ # Route-based pages (Dashboard Settings) App.tsx # Root application component main.tsx # DOM rendering entry point init.sh # Startup script for development .env.example # Template environment variables README.md # This file
| Technology | Purpose |
|---|---|
| React 19 | Frontend UI library |
| Vite | Frontend build tool |
| Tailwind CSS | Utility-first CSS framework |
| Shadcn/ui | Pre-built accessible components |
| FastAPI | Backend API framework |
| Python 3.11+ | Backend language |
| SQLite/PostgreSQL | Data storage |
| bun | Frontend package manager |
| uv | Backend package manager |
| Anthropic API | LLM safety classification |
Fork the repository. Make your changes in a feature branch. Test thoroughly before submitting. Open a pull request targeting main.
MIT
Matthew Snow -- M2AI | @m2ai-portfolio