Skip to content

README.md

Latest commit

 

History

History
256 lines (191 loc) · 9.9 KB

README.md

File metadata and controls

256 lines (191 loc) · 9.9 KB
Raw

AI Orchestration Patterns

Production-ready implementations of 8 AI orchestration patterns — from deterministic workflows to autonomous multi-agent systems.

Each pattern uses the same use case (tennis court booking) to highlight architectural differences, not domain complexity.

📖 Reference

This repo implements the patterns described in my blog post:

The Control Spectrum: 8 AI Orchestration Patterns from Full Control to Full Autonomy

Read the blog first for architecture diagrams, trade-offs, and decision guides.

Patterns

Pattern Style Runtime Status
A - AI as Service No agent, LLM parses only Single-Process ✅ Done
B - Workflow (Single-Process) Fixed sequence Single-Process ✅ Done
C - Workflow (Multi-Process) Fixed sequence Multi-Process ✅ Done
D - Function Calling LLM suggests, you control loop Single-Process ✅ Done
E - Single Agent Agent controls the loop Single-Process ✅ Done
F - Multi-Agent (Single-Process) Manager routes dynamically Single-Process ✅ Done
G - Multi-Agent (Multi-Process) Manager routes dynamically Multi-Process ✅ Done
H - Bedrock Agent AWS-managed agent Managed ✅ Done

Implementation Status

✅ Local Patterns: A → B → C → D → E → F → G → H (all done!)
☁️ AWS Deployed:   A → B → C → D → E → F → G → H (live on Lambda)

The Spectrum

Control ←——————————————————————————————————————————→ Autonomy

  A       B       C       D       E       F       G       H
  │       │       │       │       │       │       │       │
  No    Workflow Workflow Function Single Multi  Multi  Bedrock
 Agent  (Single) (Multi)  Calling  Agent  Agent  Agent  (Managed)
  │       │       │       │       │       │       │       │
 You    Fixed   Fixed    LLM     Agent  Manager Manager  AWS
control steps   steps  suggests controls routes  routes manages
 all   (single) (multi)  you      loop
                        control

MCP Layer

The spectrum answers who decides which function to call. MCP answers a different question — how tools are exposed and discovered — so it lives in mcp/ as a layer, not a ninth pattern. The same two booking tools are served over MCP, and a Pattern E-style agent consumes them with zero locally-defined tools:

Patterns D–G:  agent ──(in-process function call)──▶ tool code
MCP local:     agent ──(stdio, subprocess)─────────▶ MCP server ──▶ tool code
MCP remote:    agent ──(streamable HTTP)───────────▶ MCP server ──▶ tool code

Try the deployed MCP-backed agent:

curl -X POST https://vg35zfgrh1.execute-api.us-east-1.amazonaws.com/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Book tomorrow at 3pm"}'

The MCP server itself is bearer-token protected (the tool plane is not public) and holds no OpenAI key. See mcp/README.md for local stdio usage — including Claude Code registration — the raw JSON-RPC handshake, and the Lambda-specific design notes.

Live Demo

Try the deployed patterns (AWS Lambda + API Gateway):

Pattern Health Check Chat Endpoint
A - AI as Service Health curl -X POST https://7jtqo8ncu4.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
B - Workflow (Single) Health curl -X POST https://jwmoovw1se.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
C - Workflow (Multi) Health curl -X POST https://1ywzwz1hog.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
D - Function Calling Health curl -X POST https://3sd40p0zz4.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
E - Single Agent Health curl -X POST https://ok1ro2wdf1.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
F - Multi-Agent (Single) Health curl -X POST https://seymcwtuh9.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
G - Multi-Agent (Multi) Health curl -X POST https://nwnh1ys1u8.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'
H - Bedrock Agent Health curl -X POST https://dck1pppjal.execute-api.us-east-1.amazonaws.com/chat -H "Content-Type: application/json" -d '{"message": "Book tomorrow at 3pm"}'

Tech Stack

  • Language: Python
  • Package Manager: UV
  • AI Providers: OpenAI, Anthropic Claude, AWS Bedrock
  • Agent Framework: OpenAI Agents SDK
  • Tool Protocol: MCP (Model Context Protocol) — see mcp/
  • API: FastAPI
  • Infrastructure: AWS Lambda, Terraform

Use Case

All patterns implement a tennis court booking system:

check_availability(date, time) → returns available slots
book(slot_id, user_id)         → reserves a slot

The difference: who decides which function to call and when.

Repo Structure

The repository is organized into three main areas:

  • pattern-*/ folders - Each contains a complete implementation of one orchestration pattern with source code, dependencies, and sequence diagrams
  • terraform/ folder - AWS infrastructure (Lambda + API Gateway) to deploy each pattern. One subfolder per pattern.
  • shared/ folder - Common booking service logic reused across all patterns to keep the focus on orchestration differences, not business logic
ai-orchestration-patterns/
├── README.md
├── pattern-a-ai-as-service/
├── pattern-b-workflow-single-process/
├── pattern-c-workflow-multi-process/
├── pattern-d-function-calling/
├── pattern-e-single-agent/
├── pattern-f-multi-agent-single-process/
├── pattern-g-multi-agent-multi-process/
├── pattern-h-bedrock-agent/
├── mcp/                      # MCP layer: booking MCP server + consuming agent
├── scripts/
│   ├── package_lambda.py     # Build tool for patterns A-F
│   └── requirements-lambda.txt
├── shared/
│   └── booking_service.py    # Mock booking service (all patterns)
└── terraform/                # Infrastructure (Lambda + API Gateway)
    ├── pattern_a/
    ├── pattern_b/
    ├── ...
    ├── pattern_h/
    └── mcp/

Getting Started

Prerequisites

Required:

  • Python 3.12+
  • UV (package manager)
  • Docker (for AWS Lambda builds)

For AWS deployment:

  • AWS CLI configured
  • Terraform 1.5+
  • OpenAI API key (patterns A-G)
  • AWS account with Bedrock access (pattern H)

Local Development

Run patterns locally without AWS:

# Install dependencies
cd pattern-d-function-calling
uv sync

# Run demo
uv run src/demo.py

AWS Deployment

Step 1: Configure Secrets

# Create .env file (gitignored)
cat > .env << EOF
OPENAI_API_KEY=sk-...
EOF

Step 2: Build Lambda Package

For Patterns A-F (single Lambda):

python scripts/package_lambda.py pattern-a-ai-as-service

For Pattern G (3 Lambdas - manager, availability, booking):

cd pattern-g-multi-agent-multi-process
./build.sh

For Pattern H (2 Lambdas - action, invoker):

cd pattern-h-bedrock-agent
./build.sh

Step 3: Deploy with Terraform

cd terraform/pattern_a

# First time: copy example config
cp terraform.tfvars.example terraform.tfvars

# Edit terraform.tfvars:
# - Add your OpenAI API key
# - (Pattern H only) Choose foundation model

# Deploy
terraform init
terraform apply

Step 4: Test Deployment

# Get endpoint
terraform output api_endpoint

# Test health
curl $(terraform output -raw api_endpoint)/health

# Test chat
curl -X POST $(terraform output -raw api_endpoint)/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Book tomorrow at 3pm"}'

Pattern-Specific Notes

Pattern H (Bedrock Agent):

  • Uses AWS Nova Pro by default (no agreement required)
  • Alternative models require Bedrock model access:
    1. AWS Console → Bedrock → Model access
    2. Enable desired model (e.g., Claude Haiku 4.5)
    3. Update foundation_model in terraform.tfvars

Why This Repo?

  • Real infrastructure — not just pseudocode
  • Clear progression — deterministic → controlled → autonomous
  • Trade-off analysis — when to use each pattern
  • Production patterns — what actually works in enterprise

Author

Moss Gu

License

MIT