Enterprise Bedrock Proxy Gateway

An open-source sample implementation of an enterprise gateway for Amazon Bedrock that demonstrates OAuth 2.0 (Open Authorization 2.0) authentication, rate limiting, multi-account routing, and comprehensive observability patterns.

This code can serve as basis to make LLMs and guardrails accessible across large enterprises. In contrast to other code samples/solutions, the code addresses only this challenge in a way that is modular, extensible (e.g. by 3rd party model providers) and easy to maintain.

Note: This is a sample implementation provided for educational and demonstration purposes. You should review, test, and customize this code for your specific requirements before deploying to any environment.

To understand how this code could be integrated in an enterprise to accelerate GenAI adoption, please review this blog post.

What does this provide?

This sample demonstrates how to build a secure gateway for Amazon Bedrock with:

• OAuth 2.0 authentication - Integrate with identity providers like Okta, Auth0, Amazon Cognito, or Microsoft Entra ID
• Rate limiting - Token-based quotas per client to control costs
• Multi-account routing - Load balance across multiple AWS accounts with automatic failover
• Credential caching - Sub-10ms response times with Valkey cache
• Streaming support - Real-time model responses
• Observability - CloudWatch metrics, X-Ray tracing, and structured logging

Architecture

Client → ALB → ECS (Fargate) → Valkey → Amazon Bedrock
                ↓
         CloudWatch + X-Ray

The gateway runs on Amazon ECS with AWS Fargate and uses Application Load Balancer for traffic distribution.

• Step 1: Requests are sent to an Application Load Balancer (ALB) fronted by a WAF in the central account using the Bedrock runtime API. Use cases can use the standard Bedrock API.
• Step 2: The ALB routes to an Amazon ECS service implemented in FastAPI that processes the request.
• Step 3: The quota of the use case and target accounts is checked in ElastiCache.
• Step 4: Check if STS credentials for the use case are cached in ElastiCache to make Bedrock calls in the target accounts by assuming a role with web identity.
• Step 5 to 7: If credentials are not cached, refresh them by calling the Identity Provider (steps 5–7).
• Step 8: Check if the use case has included pre-defined Guardrails in the request and include the ID of the corresponding Guardrail in the target account (see the Guardrails section for details).
• Step 9: Forward the Bedrock request with the correct Guardrail ID to one of the target accounts based on available quota.

Quick start

# Clone repository
git clone https://github.com/aws-samples/sample-bedrock-proxy-gateway.git
cd sample-bedrock-proxy-gateway

# Configure for your environment
cd infrastructure
cp dev.tfvars dev.local.tfvars
# Edit dev.local.tfvars with your app_id, OAuth provider, and AWS account details

# Deploy
cd ..
./scripts/deploy.sh dev --apply

# Destroy
./scripts/destroy.sh dev --confirm

Prerequisites: AWS account, Terraform 1.5+, AWS CLI v2, OAuth 2.0 provider

Note: The .local.* files (.local.tfvars, .local.tfbackend, .local.yaml) are gitignored for your personal configurations. The base files contain generic examples for the open-source repository.

For detailed instructions, see the Quick Start Guide.

Documentation

Complete documentation is in the docs/gateway/ directory:

• Quick Start - Deploy in 15 minutes
• Setup Guide - Prerequisites, deployment, configuration
• Usage Guide - Authentication, API usage, code examples
• Architecture - Components, security, operations
• Troubleshooting - Common issues and solutions

Examples

Interactive Jupyter notebooks are available in the examples/ directory:

• Getting started and fundamentals
• Embeddings and RAG (Retrieval-Augmented Generation)
• Image generation and guardrails
• Rate limiting and operations

Resources

• Documentation
• Amazon Bedrock User Guide
• Amazon Bedrock API Reference
• OAuth 2.0 Specification

Security

Note

The gateway deploys with a self-signed TLS certificate by default. The example notebooks use verify=False to accommodate this. Traffic is still encrypted, but server identity is not verified. To enable full TLS verification, configure a custom domain with an ACM certificate and remove verify=False.

See CONTRIBUTING for more information.

License

This library is licensed under the Apache 2.0 License. See the LICENSE file.

Authors

The following authors have contributed this sample within AWS and prepared it for open-source release:

• Yuvaraj Kesavan
• Rumeshkrishnan Mohan
• Nicola D'Orazio
• Konstantin Zerebcov
• David Sauerwein

Special thanks to Olivier Brique for his thorough review and suggestions that helped improve the solution.