Skip to content

Fixes agent timeout issue by moving to a load balancer #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fixes agent timeout issue by moving to a load balancer #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6ad80df
update to use nlb
zjaco13 Sep 10, 2025
905a7da
make chat sync and update to support ingressnginx deployment
zjaco13 Sep 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions eks/agents/weather/src/agent_server_a2a.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,15 @@ def a2a_agent():
logger.info("Agent instance created successfully")

port = os.getenv("A2A_PORT", "9000")
hosting_http_url = os.getenv("A2A_URL", "0.0.0.0")
host_ip = os.getenv("A2A_HOST", "0.0.0.0")
hosting_url=os.getenv("A2A_URL", "http://weather-agent:9000")


strands_a2a_agent = A2AServer(
agent=strands_agent,
host=host_ip,
port=int(port),
http_url=hosting_http_url
http_url=hosting_url
)
logger.info("A2A Server wrapper created successfully")

Expand Down
7 changes: 6 additions & 1 deletion eks/infrastructure/terraform/eks.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,12 @@ module "eks_blueprints_addons" {
}

# Add-ons
enable_cert_manager = true
enable_cert_manager = true
enable_ingress_nginx = true

ingress_nginx = {
values = [templatefile("${path.module}/ingress_nginx_values.yaml", {}),]
}


tags = local.tags
Expand Down
10 changes: 10 additions & 0 deletions eks/infrastructure/terraform/ingress_nginx_values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
controller:
replicaCount: 2
service:
type: LoadBalancer
annotations:
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
2 changes: 2 additions & 0 deletions eks/manifests/helm/agent/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ spec:
value: "{{ .Values.mcp.port }}"
- name: A2A_PORT
value: "{{ .Values.a2a.port }}"
- name: A2A_HOST
value: "{{ .Values.a2a.host }}"
- name: A2A_URL
value: "{{ .Values.a2a.http_url }}"
- name: FASTAPI_PORT
Expand Down
2 changes: 2 additions & 0 deletions eks/manifests/helm/agent/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ fastapi:
enabled: false
a2a:
port: 9000
host: "0.0.0.0"
http_url: "http://weather-agent:9000"
exposedPort: 9000
ingress:
enabled: false
Expand Down
11 changes: 11 additions & 0 deletions eks/manifests/helm/ui/templates/ingress-fastapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,17 @@ spec:
{{- end }}
{{- end }}
rules:
{{- if .Values.ingress.defaultRule.enabled }}
- http:
paths:
- path: {{ .Values.ingress.defaultRule.path | default "/" }}
pathType: {{ .Values.ingress.defaultRule.pathType | default "Prefix" }}
backend:
service:
name: {{ include "ui.fullname" . }}
port:
name: fastapi
{{- end }}
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
Expand Down
4 changes: 4 additions & 0 deletions eks/manifests/helm/ui/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ ingress:
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
defaultRule:
enabled: false
path: "/"
pathType: "Prefix"
hosts:
- host: chart-example.local
paths:
Expand Down
2 changes: 2 additions & 0 deletions eks/scripts/terraform-prep-env-travel-agent.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,5 +156,7 @@ image:
env:
OAUTH_JWKS_URL: $OAUTH_JWKS_URL
SESSION_STORE_BUCKET_NAME: $SESSION_STORE_BUCKET_NAME
DISABLE_AUTH: "1"


EOF
1 change: 1 addition & 0 deletions eks/scripts/terraform-prep-env-weather-agent.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,5 +78,6 @@ image:
env:
OAUTH_JWKS_URL: $OAUTH_JWKS_URL
SESSION_STORE_BUCKET_NAME: $SESSION_STORE_BUCKET_NAME
DISABLE_AUTH: "1"

EOF
17 changes: 13 additions & 4 deletions eks/scripts/terraform-prep-env-weather-ui.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,26 @@ echo "OAUTH_LOGOUT_URL=$OAUTH_LOGOUT_URL" >> $UI_AGENT_DST_FILE_NAME
echo "OAUTH_WELL_KNOWN_URL=$OAUTH_WELL_KNOWN_URL" >> $UI_AGENT_DST_FILE_NAME
echo "OAUTH_JWKS_URL=$OAUTH_JWKS_URL" >> $UI_AGENT_DST_FILE_NAME


BASE_URL=$(kubectl get svc -n ingress-nginx ingress-nginx-controller -o jsonpath={.status.loadBalancer.ingress[0].hostname})

ECR_REPO_AGENT_UI_URI=$(terraform -chdir="$TERRAFORM_DIRECTORY" output -json ecr_agent_ui_repository_url)

cat <<EOF > $UI_AGENT_HELM_VALUES
image:
repository: $ECR_REPO_AGENT_UI_URI
env:
env:
AGENT_UI_ENDPOINT_URL_1: "http://weather-agent.agents/prompt"
AGENT_UI_ENDPOINT_URL_2: "http://travel-agent.agents/prompt"
BASE_PATH: "${IDE_URL:+/proxy/8000}"
BASE_URL: "${IDE_URL:-http://localhost:8000}"
BASE_PATH: "/"
BASE_URL: "http://${BASE_URL:-localhost:8000}"
AUTH_ENABLED: "false"
fastapi:
ingress:
enabled: true
ingress:
enabled: true
className: nginx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are using EKS Auto, we should leverage the AWS ALB Controller that comes with EKS Auto and not use nginx.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@csantanapr, what is your recommendation for getting the DNS of the ingress to pass as the BASE_URL of the application?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @csantanapr / @zjaco13 do you have an update for the PR here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still working on a solution @evgrenda

defaultRule:
enabled: true

EOF
Loading