diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/README.md b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/README.md new file mode 100644 index 00000000..3d8d1c56 --- /dev/null +++ b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/README.md @@ -0,0 +1,44 @@ +# CrewAI Agent with Amazon Bedrock AgentCore Runtime and Observability + +This tutorial demonstrates how to deploy a [CrewAI](https://www.crewai.com/) travel agent to Amazon Bedrock AgentCore Runtime with observability through Amazon CloudWatch. + +## Overview + +Learn to host a CrewAI agent using Amazon Bedrock models with comprehensive observability with AWS OpenTelemetry instrumentation and Amazon CloudWatch monitoring. + +## Prerequisites + +* Python 3.10+ +* AWS credentials configured with appropriate permissions +* Amazon Bedrock AgentCore SDK +* CrewAI framework +* Amazon CloudWatch access +* Enable [transaction search](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Enable-TransactionSearch.html) on Amazon CloudWatch + +## Getting Started + +1. Install dependencies: + ```bash + pip install -r requirements.txt + ``` + +2. Open the Jupyter notebook: `runtime-with-crewai-and-bedrock-models.ipynb` + +3. Follow the tutorial to: + - Create and test a CrewAI agent locally + - Deploy the agent to AgentCore Runtime + - Enable observability with OpenTelemetry + - Monitor performance on CloudWatch + +## Key Features + +* CrewAI travel agent with web search capabilities +* Amazon Bedrock models (Anthropic Claude Sonnet 3.7) +* AgentCore Runtime hosting +* CloudWatch observability and tracing + +## Cleanup + +After completing the tutorial: +1. Remove AgentCore Runtime deployments +2. Clean up ECR repositories \ No newline at end of file diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/architecture_runtime.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/architecture_runtime.png new file mode 100644 index 00000000..075964c5 Binary files /dev/null and b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/architecture_runtime.png differ diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/configure.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/configure.png new file mode 100644 index 00000000..13e7cc88 Binary files /dev/null and b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/configure.png differ diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/invoke.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/invoke.png new file mode 100644 index 00000000..cbc39a2c Binary files /dev/null and b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/invoke.png differ diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/launch.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/launch.png new file mode 100644 index 00000000..91196a01 Binary files /dev/null and b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/launch.png differ diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/span_crew_Ai.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/span_crew_Ai.png new file mode 100644 index 00000000..a2df0a1d Binary files /dev/null and b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/span_crew_Ai.png differ diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/span_details.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/span_details.png new file mode 100644 index 00000000..61925fdf Binary files /dev/null and b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/images/span_details.png differ diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/requirements.txt b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/requirements.txt new file mode 100644 index 00000000..7e3bd563 --- /dev/null +++ b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/requirements.txt @@ -0,0 +1,11 @@ +crewai +crewai-tools +bedrock-agentcore +bedrock-agentcore-starter-toolkit +langchain-community +ddgs +opentelemetry-instrumentation-crewai +aws-opentelemetry-distro==0.12.1 +boto3 +botocore +python-dotenv \ No newline at end of file diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/runtime-with-crewai-and-bedrock-models.ipynb b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/runtime-with-crewai-and-bedrock-models.ipynb new file mode 100644 index 00000000..046a11e5 --- /dev/null +++ b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/CrewAI/runtime-with-crewai-and-bedrock-models.ipynb @@ -0,0 +1,557 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5c0122e65c053f38", + "metadata": {}, + "source": [ + "# Hosting Simple CrewAI Agent with Amazon Bedrock models in Amazon Bedrock AgentCore Runtime\n", + "\n", + "## Overview\n", + "\n", + "In this tutorial we will learn how to host a simple CrewAI agent using Amazon Bedrock AgentCore Runtime. We will be using Opentelemetry instrumentation and AWS Opentelemetry python library to add observability to this agent and monitor it's performance on Amazon CloudWatch GenAI Observability Dashboard.\n", + "\n", + "### Tutorial Details\n", + "\n", + "| Information | Details |\n", + "|:--------------------|:---------------------------------------------------------------------------------|\n", + "| Tutorial type | Conversational |\n", + "| Agent type | Single |\n", + "| Agentic Framework | CrewAI |\n", + "| LLM model | Anthropic Claude Sonnet 3.7 |\n", + "| Tutorial components | Hosting agent on AgentCore Runtime. Using CrewAI and Amazon Bedrock Model |\n", + "| Tutorial vertical | Cross-vertical |\n", + "| Example complexity | Easy |\n", + "| SDK used | Amazon BedrockAgentCore Python SDK and boto3 |\n", + "\n", + "### Tutorial Key Features\n", + "\n", + "* Hosting Agents on Amazon Bedrock AgentCore Runtime\n", + "* Using Amazon Bedrock models\n", + "* Using CrewAI\n", + "* Amazon CloudWatch GenAI Observability\n", + "\n", + "\n", + "### Tutorial Architecture\n", + "\n", + "In this tutorial we will describe how to deploy an existing multi-agent crew to AgentCore runtime. \n", + "\n", + "For demonstration purposes, we will use a CrewAI crew using Amazon Bedrock models\n", + "\n", + "In our example we will use a travel agent with web search capabilities.\n", + "
\n", + " \n", + "
\n" + ] + }, + { + "cell_type": "markdown", + "id": "3a676f58ecf52b42", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "To execute this tutorial you will need:\n", + "* Python 3.10+\n", + "* AWS credentials with appropriate permissions\n", + "* Amazon Bedrock AgentCore SDK\n", + "* CrewAI\n", + "* Amazon CloudWatch Access\n", + "* Enable [transaction search](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Enable-TransactionSearch.html) on Amazon CloudWatch." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "initial_id", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --force-reinstall -U -r requirements.txt --quiet" + ] + }, + { + "cell_type": "markdown", + "id": "932110e6-fca6-47b6-b7c5-c4714a866a80", + "metadata": {}, + "source": [ + "## Preparing your agent for deployment on AgentCore Runtime\n", + "\n", + "Let's now deploy our agent to AgentCore Runtime. To do so we need to:\n", + "* Import the Runtime App with `from bedrock_agentcore.runtime import BedrockAgentCoreApp`\n", + "* Initialize the App in our code with `app = BedrockAgentCoreApp()`\n", + "* Decorate the invocation function with the `@app.entrypoint` decorator\n", + "* Let AgentCoreRuntime control the running of the agent with `app.run()`\n", + "\n", + "### CrewAI Agent with Amazon Bedrock model\n", + "Let's create our runtime-ready CrewAI Agent using Amazon Bedrock model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3b845b32-a03e-45c2-a2f0-2afba8069f47", + "metadata": {}, + "outputs": [], + "source": [ + "%%writefile crewai_runtime_agent.py\n", + "import os\n", + "\n", + "from crewai import Agent, Task, Crew, LLM\n", + "from crewai.tools import tool\n", + "from ddgs import DDGS\n", + "import logging\n", + "from bedrock_agentcore.runtime import BedrockAgentCoreApp\n", + "from opentelemetry.instrumentation.crewai import CrewAIInstrumentor\n", + "\n", + "# Instrument CrewAI with Opentelemetry\n", + "# Note: The AWS OpenTelemetry distro will automatically handle tracer provider setup\n", + "# when using opentelemetry-instrument command\n", + "CrewAIInstrumentor().instrument()\n", + "\n", + "app = BedrockAgentCoreApp()\n", + "\n", + "# Set up logging\n", + "logging.basicConfig(level=logging.INFO)\n", + "logger = logging.getLogger(__name__)\n", + "\n", + "@tool(\"web_search\")\n", + "def web_search(query: str) -> str:\n", + " \"\"\"Search the web for current information about travel destinations, attractions, and events.\"\"\"\n", + " try:\n", + " ddgs = DDGS()\n", + " results = ddgs.text(query, max_results=3)\n", + " \n", + " formatted_results = []\n", + " for i, result in enumerate(results, 1):\n", + " formatted_results.append(\n", + " f\"{i}. {result.get('title', 'No title')}\\n\"\n", + " f\" {result.get('body', 'No summary')}\\n\"\n", + " f\" Source: {result.get('href', 'No URL')}\\n\"\n", + " )\n", + " \n", + " return \"\\n\".join(formatted_results) if formatted_results else \"No results found.\"\n", + " \n", + " except Exception as e:\n", + " return f\"Search error: {str(e)}\"\n", + "\n", + "def get_llm():\n", + " model_id = os.getenv(\"BEDROCK_MODEL_ID\", \"us.anthropic.claude-3-7-sonnet-20250219-v1:0\")\n", + " region = os.getenv(\"AWS_DEFAULT_REGION\", \"us-west-2\")\n", + " \n", + " try:\n", + " llm = LLM(\n", + " model=f\"bedrock/{model_id}\",\n", + " temperature=0.7,\n", + " max_tokens=512,\n", + " aws_region_name=region\n", + " )\n", + " logger.info(f\"Successfully initialized Bedrock LLM with model: {model_id} in region: {region}\")\n", + " return llm\n", + " except Exception as e:\n", + " logger.error(f\"Failed to initialize Bedrock LLM: {str(e)}\")\n", + " logger.error(\"Please ensure you have proper AWS credentials configured and access to the Bedrock model\")\n", + " raise\n", + "\n", + "@app.entrypoint\n", + "def crewai_agent_bedrock(payload, context):\n", + " \"\"\"\n", + " Invoke the agent with a payload\n", + " \"\"\"\n", + " print(f'Payload: {payload}')\n", + " try:\n", + " user_input = payload.get(\"prompt\", \"What are some interesting places to visit?\")\n", + " print(f\"Processing request: {user_input}\")\n", + " \n", + " llm = get_llm()\n", + "\n", + " travel_agent = Agent(\n", + " role='Travel Destination Researcher',\n", + " goal='Find dream destinations matching user preferences using web search for current information',\n", + " backstory=\"You are an experienced travel agent specializing in personalized travel recommendations with access to real-time web information.\",\n", + " verbose=True,\n", + " allow_delegation=False,\n", + " llm=llm,\n", + " max_iter=3,\n", + " tools=[web_search]\n", + " )\n", + "\n", + " task = Task(\n", + " description=f\"Research and provide travel recommendations based on this request: {user_input}. Use web search to find current information about venues, events, and attractions.\",\n", + " expected_output=\"A comprehensive list of recommended destinations with current information, brief descriptions, and practical travel details.\",\n", + " agent=travel_agent\n", + " )\n", + "\n", + " crew = Crew(\n", + " agents=[travel_agent],\n", + " tasks=[task],\n", + " verbose=True\n", + " )\n", + "\n", + " result = crew.kickoff()\n", + " \n", + " print(\"Context:\\n-------\\n\", context)\n", + " print(\"Result Raw:\\n*******\\n\", result.raw)\n", + " \n", + " return {\"result\": result.raw}\n", + " \n", + " except Exception as e:\n", + " print(f'Exception occurred: {e}')\n", + " return {\"error\": f\"An error occurred: {str(e)}\"}\n", + "\n", + "if __name__ == \"__main__\":\n", + " app.run()" + ] + }, + { + "cell_type": "markdown", + "id": "c64db7b5-0f1b-475f-9bf2-467b4449d46a", + "metadata": {}, + "source": [ + "## What happens behind the scenes?\n", + "\n", + "When you use `BedrockAgentCoreApp`, it automatically:\n", + "\n", + "* Creates an HTTP server that listens on the port 8080\n", + "* Implements the required `/invocations` endpoint for processing the agent's requirements\n", + "* Implements the `/ping` endpoint for health checks (very important for asynchronous agents)\n", + "* Handles proper content types and response formats\n", + "* Manages error handling according to the AWS standards" + ] + }, + { + "cell_type": "markdown", + "id": "6820ca8f-a8a8-4f34-b4ef-b6dad3776261", + "metadata": {}, + "source": [ + "## Deploying the agent to AgentCore Runtime\n", + "\n", + "The `CreateAgentRuntime` operation supports comprehensive configuration options, letting you specify container images, environment variables and encryption settings. You can also configure protocol settings (HTTP, MCP) and authorization mechanisms to control how your clients communicate with the agent. \n", + "\n", + "**Note:** Operations best practice is to package code as container and push to ECR using CI/CD pipelines and IaC\n", + "\n", + "In this tutorial we will use the Amazon Bedrock AgentCore Python SDK to easily package your artifacts and deploy them to AgentCore runtime." + ] + }, + { + "cell_type": "markdown", + "id": "8855aceb-b79f-4aaa-b16f-8577c059816a", + "metadata": {}, + "source": [ + "### Configure AgentCore Runtime deployment\n", + "\n", + "Next we will use our starter toolkit to configure the AgentCore Runtime deployment with an entrypoint, the execution role we just created and a requirements file. We will also configure the starter kit to auto create the Amazon ECR repository on launch.\n", + "\n", + "During the configure step, your docker file will be generated based on your application code. \n", + "\n", + "
\n", + " \n", + "
\n", + "\n", + "Please note that when using the `bedrock_agentcore_starter_toolkit` to configure your agent, it takes care of the opentelemetry instrumentation. \n", + "\n", + "When configuring for containerized environment (such as docker) add the following command, an example is given below:\n", + "\n", + "`CMD [\"opentelemetry-instrument\", \"python\", \"runtime_agent_main.py\"]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e79eba2-ca59-463f-9ebf-56e362d7ae66", + "metadata": {}, + "outputs": [], + "source": [ + "from bedrock_agentcore_starter_toolkit import Runtime\n", + "from boto3.session import Session\n", + "boto_session = Session()\n", + "region = boto_session.region_name\n", + "\n", + "agentcore_runtime = Runtime()\n", + "agent_name = \"simple_crewai_travel_agent\"\n", + "response = agentcore_runtime.configure(\n", + " entrypoint=\"crewai_runtime_agent.py\",\n", + " auto_create_execution_role=True,\n", + " auto_create_ecr=True,\n", + " requirements_file=\"requirements.txt\",\n", + " region=region,\n", + " agent_name=agent_name,\n", + " memory_mode='NO_MEMORY'\n", + ")\n", + "response" + ] + }, + { + "cell_type": "markdown", + "id": "9e1b84cc-798e-472c-ac0b-2c315f4b704d", + "metadata": {}, + "source": [ + "### Launching agent to AgentCore Runtime\n", + "\n", + "Now that we've got a docker file, let's launch the agent to the AgentCore Runtime. This will create the Amazon ECR repository and the AgentCore Runtime\n", + "\n", + "
\n", + " \n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17a32ab8-7701-4900-8055-e24364bdf35c", + "metadata": {}, + "outputs": [], + "source": [ + "# Disable CrewAI's built-in telemetry to avoid conflicts\n", + "launch_result = agentcore_runtime.launch(env_vars={\n", + " \"CREWAI_DISABLE_TELEMETRY\": \"true\",\n", + " \"OTEL_PYTHON_EXCLUDED_URLS\": \"https://api.scarf.sh/\"\n", + " })\n", + "launch_result" + ] + }, + { + "cell_type": "markdown", + "id": "a0ae9c09-09db-4a76-871a-92eacd96b9c3", + "metadata": {}, + "source": [ + "### Checking for the AgentCore Runtime Status\n", + "Now that we've deployed the AgentCore Runtime, let's check for it's deployment status" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "afa6ac09-9adb-4846-9fc1-4d12aeb74853", + "metadata": {}, + "outputs": [], + "source": [ + "import time\n", + "status_response = agentcore_runtime.status()\n", + "status = status_response.endpoint['status']\n", + "end_status = ['READY', 'CREATE_FAILED', 'DELETE_FAILED', 'UPDATE_FAILED']\n", + "while status not in end_status:\n", + " time.sleep(10)\n", + " status_response = agentcore_runtime.status()\n", + " status = status_response.endpoint['status']\n", + " print(status)\n", + "status" + ] + }, + { + "cell_type": "markdown", + "id": "b7f89c56-918a-4cab-beaa-c7ac43a2ba29", + "metadata": {}, + "source": [ + "### Invoking AgentCore Runtime\n", + "\n", + "Finally, we can invoke our AgentCore Runtime with a payload\n", + "\n", + "
\n", + " \n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3d909e42-e1a0-407f-84c2-3d16cc889cd3", + "metadata": {}, + "outputs": [], + "source": [ + "invoke_response = agentcore_runtime.invoke({\"prompt\": \"What are some cowboy-themed attractions and museums in Texas?\"})\n", + "invoke_response" + ] + }, + { + "cell_type": "markdown", + "id": "fefa09f2-d25a-483f-aedb-11690bb8923a", + "metadata": {}, + "source": [ + "### Processing invocation results\n", + "\n", + "We can now process our invocation results to include it in an application" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "11249103-cfb3-47b5-970d-981a977a225a", + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import Markdown, display\n", + "import json\n", + "response_text = invoke_response['response'][0]\n", + "display(Markdown(response_text))" + ] + }, + { + "cell_type": "markdown", + "id": "2c1d2bce-be41-478c-8bed-b4037c385795", + "metadata": {}, + "source": [ + "### Invoking AgentCore Runtime with boto3\n", + "\n", + "Now that your AgentCore Runtime was created you can invoke it with any AWS SDK. For instance, you can use the boto3 `invoke_agent_runtime` method for it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7f84e68d-6c04-41b9-bf5b-60edc3fa0985", + "metadata": {}, + "outputs": [], + "source": [ + "import boto3\n", + "agent_arn = launch_result.agent_arn\n", + "agentcore_client = boto3.client(\n", + " 'bedrock-agentcore',\n", + " region_name=region\n", + ")\n", + "\n", + "boto3_response = agentcore_client.invoke_agent_runtime(\n", + " agentRuntimeArn=agent_arn,\n", + " qualifier=\"DEFAULT\",\n", + " payload=json.dumps({\"prompt\": \"What are some rodeo events happening in Oklahoma?\"})\n", + ")\n", + "\n", + "response_body = boto3_response['response'].read()\n", + "response_data = json.loads(response_body)\n", + "display(Markdown(response_data.get('result', 'No result found')))" + ] + }, + { + "cell_type": "markdown", + "id": "05efe60e", + "metadata": {}, + "source": [ + "### AgentCore Observability on Amazon CloudWatch \n", + "\n", + "To summarize, please follow the below steps to enable observability from AgentCore runtime hosted agents : \n", + "\n", + "- Enable Transaction Search on Amazon CloudWatch \n", + "- The agent emits traces and is instrumented using opentelemtry command : `opentelemetry-instrument python any_runtime_agent.py`\n", + "- The requirements.txt file contains `aws-opentelemetry-distro` listed while deploying the agent on Bedrock Agentcore Runtime.\n", + "\n", + "## Bedrock AgentCore Overview on GenAI Observability dashboard \n", + "\n", + "You are able to view all your Agents that have observability in them and filter the data based on time frames.\n", + "\n", + "In the main dashboard you are able to view runtime metrics across all agents.\n", + "\n", + "Now, if you click on the agent you just deployed you will be taken to a dashboard for the runtime metrics specific to this agent, you can also filter the data by a custom time frame.\n", + "\n", + "In the Sessions View tab, you can navigate to all the sessions associated with this agent.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "32099fd2", + "metadata": {}, + "source": [ + "In the Trace View tab, you can look into the traces and span information for this agent on runtime.\n", + "\n", + "\n", + "
\n", + " \n", + "
\n", + "\n", + "\n", + "Please click through the various features of GenAI observability dashboard to get more detailed information on traces.\n", + "\n", + "
\n", + " \n", + "
\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "7d3fdfe404469632", + "metadata": {}, + "source": [ + "## Cleanup (Optional)\n", + "\n", + "Let's now clean up the AgentCore Runtime created" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7c243e86-a214-483c-aef1-d5243f28ca9e", + "metadata": {}, + "outputs": [], + "source": [ + "launch_result.ecr_uri, launch_result.agent_id, launch_result.ecr_uri.split('/')[1]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76a6cf1416830a54", + "metadata": {}, + "outputs": [], + "source": [ + "agentcore_control_client = boto3.client(\n", + " 'bedrock-agentcore-control',\n", + " region_name=region\n", + ")\n", + "ecr_client = boto3.client(\n", + " 'ecr',\n", + " region_name=region\n", + " \n", + ")\n", + "\n", + "runtime_delete_response = agentcore_control_client.delete_agent_runtime(\n", + " agentRuntimeId=launch_result.agent_id,\n", + " \n", + ")\n", + "\n", + "response = ecr_client.delete_repository(\n", + " repositoryName=launch_result.ecr_uri.split('/')[1],\n", + " force=True\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "b118ad38-feeb-4d1d-9d57-e5c845becc56", + "metadata": {}, + "source": [ + "# Congratulations!\n", + "\n", + "You have successfully created and deployed a simple CrewAI agent to Amazon Bedrock AgentCore Runtime with observability enabled. This example demonstrates how to:\n", + "\n", + "- Create a simple CrewAI travel agent with web search capabilities\n", + "- Enable observability through Amazon CloudWatch\n", + "- Invoke the agent using both the SDK and boto3\n", + "\n", + "The agent can now be used with full observability and monitoring capabilities." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/README.md b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/README.md index 80547b72..2becd4b4 100644 --- a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/README.md +++ b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/README.md @@ -1,4 +1,4 @@ -# AgentCore Observability on Amazon CloudWatch for Bedrock AgentCore Agents +# AgentCore Observability on Amazon CloudWatch for Bedrock AgentCore Runtime Agents This repository contains examples to showcase AgentCore Observability for Strands Agent hosted on Amazon Bedrock AgentCore Runtime using Amazon OpenTelemetry Python Instrumentation and Amazon CloudWatch. Observability helps developers trace, debug, and monitor agent performance in production through unified operational dashboards. With support for OpenTelemetry compatible telemetry and detailed visualizations of each step of the agent workflow, Amazon CloudWatch GenAI Observability enables developers to easily gain visibility into agent behavior and maintain quality standards at scale. @@ -21,8 +21,11 @@ The Project folder has the following: ### Strands Agents -[ Strands](https://strandsagents.com/latest/) provides a framework for building LLM applications with complex workflows, focusing on model driven agentic developement. +[Strands](https://strandsagents.com/latest/) provides a framework for building LLM applications with complex workflows, focusing on model-driven agentic development. -## Clean Up +## Cleanup -Please delete the Amazon Cloudwatch Log groups and associated resources created on Amazon CloudWatch for Observability. +After completing the examples: + +1. Remove AgentCore Runtime deployments +2. Clean up any created ECR repositories diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/architecture_local.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/architecture_local.png similarity index 100% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/architecture_local.png rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/architecture_local.png diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/architecture_runtime.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/architecture_runtime.png similarity index 100% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/architecture_runtime.png rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/architecture_runtime.png diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/configure.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/configure.png similarity index 100% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/configure.png rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/configure.png diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/invoke.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/invoke.png similarity index 100% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/invoke.png rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/invoke.png diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/launch.png b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/launch.png similarity index 100% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/images/launch.png rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/images/launch.png diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/requirements.txt b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/requirements.txt similarity index 100% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/requirements.txt rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/requirements.txt diff --git a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/runtime_with_strands_and_bedrock_models.ipynb b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/runtime_with_strands_and_bedrock_models.ipynb similarity index 99% rename from 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/runtime_with_strands_and_bedrock_models.ipynb rename to 01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/runtime_with_strands_and_bedrock_models.ipynb index 8840b2ea..495f124a 100644 --- a/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/runtime_with_strands_and_bedrock_models.ipynb +++ b/01-tutorials/06-AgentCore-observability/01-Agentcore-runtime-hosted/Strands Agents/runtime_with_strands_and_bedrock_models.ipynb @@ -295,7 +295,8 @@ " auto_create_ecr=True,\n", " requirements_file=\"requirements.txt\",\n", " region=region,\n", - " agent_name=agent_name\n", + " agent_name=agent_name,\n", + " memory_mode='NO_MEMORY'\n", ")\n", "response" ] @@ -566,7 +567,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "3.11.11", "language": "python", "name": "python3" }, @@ -580,9 +581,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.18" + "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/CrewAI_Observability.ipynb b/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/CrewAI_Observability.ipynb index ada213d4..609f0e4a 100644 --- a/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/CrewAI_Observability.ipynb +++ b/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/CrewAI_Observability.ipynb @@ -44,7 +44,7 @@ "metadata": {}, "outputs": [], "source": [ - "#!pip install -r requirements.txt" + "!pip install -r requirements.txt" ] }, { @@ -73,6 +73,58 @@ "| `AWS_REGION` | `` | AWS Region |" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import boto3\n", + "cloudwatch_client = boto3.client(\"logs\")\n", + "response = cloudwatch_client.create_log_group(\n", + " logGroupName='agents/crewai-agent-logs'\n", + ")\n", + "response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "response = cloudwatch_client.create_log_stream(\n", + " logGroupName='agents/crewai-agent-logs',\n", + " logStreamName='default'\n", + ")\n", + "response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%%writefile .env\n", + "# AWS OpenTelemetry Distribution\n", + "OTEL_PYTHON_DISTRO=aws_distro\n", + "OTEL_PYTHON_CONFIGURATOR=aws_configurator\n", + "\n", + "# Export Protocol\n", + "OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf\n", + "OTEL_TRACES_EXPORTER=otlp\n", + "\n", + "# CloudWatch Integration (uncomment and configure as needed)\n", + "OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=agents/crewai-agent-logs,x-aws-log-stream=default,x-aws-metric-namespace=bedrock-agentcore\n", + "\n", + "# Service Identification\n", + "OTEL_RESOURCE_ATTRIBUTES=service.name=agentic-travel-crewai\n", + "\n", + "# Enable Agent Observability\n", + "AGENT_OBSERVABILITY_ENABLED=true" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -159,6 +211,15 @@ "from crewai.tools import tool\n", "from ddgs import DDGS\n", "import logging\n", + "from opentelemetry.instrumentation.crewai import CrewAIInstrumentor\n", + "\n", + "# Instrument CrewAI with Opentelemetry\n", + "CrewAIInstrumentor().instrument()\n", + "\n", + "# Set up logging\n", + "logging.basicConfig(level=logging.INFO)\n", + "logger = logging.getLogger(__name__)\n", + "\n", "\n", "# Set up logging\n", "logging.basicConfig(level=logging.INFO)\n", @@ -349,6 +410,12 @@ "from crewai import Agent, Task, Crew, LLM\n", "from crewai.tools import tool\n", "from ddgs import DDGS\n", + "\n", + "from opentelemetry.instrumentation.crewai import CrewAIInstrumentor\n", + "\n", + "# Instrument CrewAI with Opentelemetry\n", + "CrewAIInstrumentor().instrument()\n", + "\n", "# Set up logging\n", "logging.basicConfig(level=logging.INFO)\n", "logger = logging.getLogger(__name__)\n", @@ -538,7 +605,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.12.11" + "version": "3.13.5" } }, "nbformat": 4, diff --git a/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/requirements.txt b/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/requirements.txt index c2863c32..32d6f495 100644 --- a/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/requirements.txt +++ b/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/CrewAI/requirements.txt @@ -2,6 +2,7 @@ crewai crewai-tools aws-opentelemetry-distro~=0.12.1 +opentelemetry-instrumentation-crewai ddgs boto3 botocore diff --git a/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/README.md b/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/README.md index 0965c067..52ef07f1 100644 --- a/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/README.md +++ b/01-tutorials/06-AgentCore-observability/02-Agent-not-hosted-on-runtime/README.md @@ -1,66 +1,50 @@ -# AgentCore Observability for Open Source Agents not on AgentCore Runtime +# Self-Hosted Agent Observability using AgentCore -This repository contains examples to showcase AgentCore Observability for popular AI Open Source frameworks **not** hosted on Amazon Bedrock AgentCore Runtime using Amazon OpenTelemetry Python Instrumentation and Amazon CloudWatch. Observability helps developers trace, debug, and monitor agent performance in production through unified operational dashboards. With support for OpenTelemetry compatible telemetry and detailed visualizations of each step of the agent workflow, Amazon CloudWatch GenAI Observability enables developers to easily gain visibility into agent behavior and maintain quality standards at scale. +This section demonstrates AgentCore Observability for popular open-source agent frameworks **not** hosted on Amazon Bedrock AgentCore Runtime. Learn to add comprehensive observability to your existing agents using OpenTelemetry and Amazon CloudWatch. -We will create an agent with the following opensource agent frameworks: +## Available Frameworks -- **CrewAI** -- **LangGraph** -- **LlamaIndex** -- **Strands Agents** - -## Project Structure - -``` -02-open-source-agents-3p/ -├── CrewAI/ -│ ├── .env.example -│ ├── CrewAI_Observability.ipynb -│ └── requirements.txt -├── Langgraph/ -│ ├── .env.example -│ ├── Langgraph_Observability.ipynb -│ └── requirements.txt -├── LlamaIndex/ -│ ├── .env.example -│ ├── LlamaIndex_Observability.ipynb -│ ├── README.md -│ └── requirements.txt -└── Strands/ - ├── .env.example - ├── requirements.txt - └── Strands_Observability.ipynb -``` - -## Getting Started +### CrewAI +- **Notebook**: `CrewAI_Observability.ipynb` +- **Description**: Autonomous AI agents working in teams +- **Features**: Multi-agent collaboration with custom instrumentation -Each framework has its own directory with: -- A Jupyter notebook demonstrating the framework's capabilities -- A requirements.txt file listing necessary dependencies -- An .env.example file showing required environment variables +### LangGraph +- **Notebook**: `Langgraph_Observability.ipynb` +- **Description**: Stateful, multi-actor LLM applications +- **Features**: Complex reasoning systems with trace visualization -## Usage +### LlamaIndex +- **Notebook**: `LlamaIndex_Observability.ipynb` +- **Description**: LLM-powered agents over data +- **Features**: Function agents with session tracking +- **Additional**: Detailed README with architecture diagrams -1. Navigate to the directory of the framework you want to explore -2. Install the requirements: `pip install -r requirements.txt` -3. Configure your AWS credentials -3. Copy the .env.example file to .env and update the variables -4. Open and run the Jupyter notebook +### Strands Agents +- **Notebook**: `Strands_Observability.ipynb` +- **Description**: Model-driven agentic development +- **Features**: Complex workflow agents with custom spans -## Frameworks Overview +## Getting Started -### CrewAI -[CrewAI](https://www.crewai.com/) enables the creation of autonomous AI agents that can work together in roles to accomplish tasks. +1. Choose your framework directory +2. Install requirements: `pip install -r requirements.txt` +3. Configure AWS credentials +4. Copy `.env.example` to `.env` and update variables +5. Enable CloudWatch Transaction Search +6. Run the Jupyter notebook -### LangGraph -[LangGraph](https://www.langchain.com/langgraph) extends LangChain with stateful, multi-actor applications. It's particularly useful for creating complex reasoning systems with LLMs. -### LlamaIndex -[LlamaIndex](https://www.llamaindex.ai/) allows for the creation of LLM-powered agents over data with LLMs and workflows. +## Prerequisites -### Strands Agents -[ Strands](https://strandsagents.com/latest/) provides a framework for building LLM applications with complex workflows, focusing on model driven agentic developement. +- AWS account with Bedrock and CloudWatch access with the right permissions +- Python 3.10+ +- AWS CloudWatch Transaction Search enabled +- Framework-specific dependencies -## Clean Up +## Cleanup -Please delete the Log groups and associated resources created on Amazon CloudWatch. +After completing examples: +1. Delete CloudWatch log groups +2. Remove any created AWS resources +3. Clean up local environment files diff --git a/01-tutorials/06-AgentCore-observability/03-advanced-concepts/README.md b/01-tutorials/06-AgentCore-observability/03-advanced-concepts/README.md index 79c603d4..fadeb71d 100644 --- a/01-tutorials/06-AgentCore-observability/03-advanced-concepts/README.md +++ b/01-tutorials/06-AgentCore-observability/03-advanced-concepts/README.md @@ -1,23 +1,49 @@ -# Advanced Concepts - Observability +# Advanced Observability Concepts -This folder contains advanced observability concepts and techniques for Amazon Bedrock AgentCore. +This section covers advanced observability patterns and techniques for Amazon Bedrock AgentCore, helping you implement sophisticated customized monitoring and debugging capabilities. -## 📁 Contents +## Available Tutorials ### 01-custom-span-creation/ -Learn how to create custom spans for enhanced tracing and monitoring of your agent workflows. +- **Notebook**: `Custom_Span_Creation.ipynb` +- **Description**: Learn to create custom spans for detailed operation tracing +- **Features**: Manual span creation, custom attributes +- **Use Cases**: Fine-grained monitoring, debugging +## What You'll Learn -## 🚀 Getting Started +- **Custom Span Creation**: Add detailed tracing to specific operations +- **Span Attributes**: Enrich traces with custom metadata +- **Nested Spans**: Create hierarchical trace structures +- **Performance Monitoring**: Identify bottlenecks in agent workflows +- **Error Tracking**: Capture and trace exceptions and failures -1. Copy `.env.example` to `.env` and configure your AWS credentials and Cloudwatch variables. -2. Enable Transaction Search in your region in AWS CloudWatch. -2. Install dependencies: `pip install -r requirements.txt` -3. Open the Jupyter notebook to follow the tutorial +## Getting Started +1. Navigate to the tutorial directory +2. Copy `.env.example` to `.env` and configure: + - AWS credentials + - CloudWatch log group settings + - OpenTelemetry configuration +3. Enable CloudWatch Transaction Search in your AWS region +4. Install dependencies: `pip install -r requirements.txt` +5. Open and run the Jupyter notebook -## 🧹 Cleanup +## Prerequisites -After completing the tutorials, clean up the CloudWatch log groups created during the examples. +- Understanding of basic OpenTelemetry concepts +- Familiarity with Amazon CloudWatch +- Experience with agent frameworks (recommended) +- AWS account with appropriate permissions +## Advanced Patterns Covered +- **Manual Instrumentation**: When and how to add custom spans +- **Custom Metrics**: Creating domain-specific measurements + +## Cleanup + +After completing tutorials: +1. Delete CloudWatch log groups created during examples +2. Remove any test resources +3. Clean up environment configuration files \ No newline at end of file diff --git a/01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/README.md b/01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/README.md new file mode 100644 index 00000000..09702727 --- /dev/null +++ b/01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/README.md @@ -0,0 +1,62 @@ +# Third-Party Observability Integration + +This section demonstrates how to integrate Amazon Bedrock AgentCore Runtime hosted agents with third-party observability platforms. Learn to leverage specialized monitoring tools while maintaining the benefits of AgentCore Runtime. + +## Available Integrations + +The publish folder contains: +- A Jupyter notebook demonstrating AgentCore runtime with various observability solutions +- A requirements.txt file listing necessary dependencies + +## Getting Started + +1. Choose your observability platform +2. Create an account on the respective platform +3. Obtain API keys and configuration details +4. Install requirements: `pip install -r requirements.txt` +5. Configure environment variables in the notebook +6. Deploy your agent to AgentCore Runtime +7. Run the notebook to see integrated observability + + +## Framework Support + +Amazon Bedrock AgentCore supports any agentic framework and model of your choice: +- CrewAI +- LangGraph +- LlamaIndex +- Strands Agents + +### Strands Agents +[Strands](https://strandsagents.com/latest/) provides built-in telemetry support, making it ideal for demonstrating third-party integrations. + +## Configuration Requirements + +Each platform requires specific configuration: + +### Braintrust +- API key from Braintrust dashboard +- Project configuration + +### Langfuse +- Public and secret keys +- Project configuration + +## Cleanup + +After completing examples: +1. Delete AgentCore Runtime deployments +2. Remove ECR repositories +3. Clean up platform-specific resources +4. Revoke API keys if no longer needed + +## Additional Resources + +- [Braintrust Documentation](https://www.braintrust.dev/docs) +- [Langfuse Documentation](https://langfuse.com/docs) +- [AgentCore Runtime Guide](https://docs.aws.amazon.com/bedrock-agentcore/latest/userguide/runtime.html) + +# Third-Party Observability for Amazon Bedrock AgentCore Agents + +This repository contains examples of using agents hosted on Amazon Bedrock AgentCore Runtime with third-party observability tools like Braintrust, Langfuse, and others. These examples demonstrate OpenTelemetry integration for monitoring agent performance, tracing LLM interactions, and debugging workflows. + diff --git a/01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/README.txt b/01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/README.txt deleted file mode 100644 index 79e49515..00000000 --- a/01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/README.txt +++ /dev/null @@ -1,27 +0,0 @@ -# Third-Party Observability for Amazon Bedrock AgentCore Agents - -This repository contains examples of using agents hosted on Amazon Bedrock AgentCore Runtime with third-party observability tools like Braintrust, Langfuse, and others. These examples demonstrate OpenTelemetry integration for monitoring agent performance, tracing LLM interactions, and debugging workflows. - - -## Getting Started - -The publish folder contains: -- A Jupyter notebook demonstrating AgentCore runtime with various observability solutions -- A requirements.txt file listing necessary dependencies - -## Usage - -1. Install the requirements: `pip install -r requirements.txt` -2. Configure your AWS credentials -3. Set up your observability platform account and obtain API keys -4. Update environment variables in the notebook with your credentials -5. Open and run the Jupyter notebook - -## Frameworks -While the examples demonstrate using Strands Agent SDK, Amazon bedrock AgentCore enables developers to use any agentic framework and any model of their choice. - -### Strands Agents -[Strands](https://strandsagents.com/latest/) provides a framework for building LLM applications with complex workflows, with built-in telemetry support. - -## Clean Up -Remember to delete the AgentCore Runtime and ECR repository using the cleanup section in the notebook to avoid ongoing charges. \ No newline at end of file diff --git a/01-tutorials/06-AgentCore-observability/README.md b/01-tutorials/06-AgentCore-observability/README.md index 6622f0b6..1b06e080 100644 --- a/01-tutorials/06-AgentCore-observability/README.md +++ b/01-tutorials/06-AgentCore-observability/README.md @@ -1,17 +1,24 @@ -# AgentCore Observability on Amazon CloudWatch +# AgentCore Observability -This repository demonstrates how to implement AgentCore observability for Agents using Amazon CloudWatch and OpenTelemetry. It provides examples for both Amazon Bedrock AgentCore Runtime hosted agents and popular open-source agent frameworks. +This repository demonstrates how to implement AgentCore observability for Agents using Amazon CloudWatch and other providers. It provides examples for both Amazon Bedrock AgentCore Runtime hosted agents and agents hosted non on runtime with popular open-source agent frameworks. + + +For more details on AgentCore Observability, please refer to [this](https://aws.amazon.com/blogs/machine-learning/build-trustworthy-ai-agents-with-amazon-bedrock-agentcore-observability/) blog post. ## Project Structure ``` 06-AgentCore-observability/ ├── 01-Agentcore-runtime-hosted/ -│ ├── images/ -│ ├── .env.example -│ ├── README.md -│ ├── requirements.txt -│ └── runtime_with_strands_and_bedrock_models.ipynb +│ ├── CrewAI/ +│ │ ├── images/ +│ │ ├── requirements.txt +│ │ └── runtime-with-crewai-and-bedrock-models.ipynb +│ ├── Strands Agents/ +│ │ ├── images/ +│ │ ├── requirements.txt +│ │ └── runtime_with_strands_and_bedrock_models.ipynb +│ └── README.md ├── 02-Agent-not-hosted-on-runtime/ │ ├── CrewAI/ │ │ ├── .env.example @@ -21,44 +28,71 @@ This repository demonstrates how to implement AgentCore observability for Agents │ │ ├── .env.example │ │ ├── Langgraph_Observability.ipynb │ │ └── requirements.txt +│ ├── LlamaIndex/ +│ │ ├── images/ +│ │ ├── .env.example +│ │ ├── LlamaIndex_Observability.ipynb +│ │ ├── README.md +│ │ └── requirements.txt │ ├── Strands/ +│ │ ├── images/ │ │ ├── .env.example │ │ ├── requirements.txt │ │ └── Strands_Observability.ipynb │ └── README.md ├── 03-advanced-concepts/ -│ └── 01-custom-span-creation/ -│ ├── .env.example -│ ├── Custom_Span_Creation.ipynb -│ └── requirements.txt -├── README.md -└── utils.py +│ ├── 01-custom-span-creation/ +│ │ ├── .env.example +│ │ ├── Custom_Span_Creation.ipynb +│ │ └── requirements.txt +│ └── README.md +├── 04-Agentcore-runtime-partner-observability/ +│ ├── Braintrust/ +│ │ ├── requirements.txt +│ │ └── runtime_with_strands_and_braintrust.ipynb +│ ├── Langfuse/ +│ │ ├── requirements.txt +│ │ └── runtime_with_strands_and_langfuse.ipynb +│ ├── images/ +│ └── README.md +└── README.md ``` ## Overview -This repository provides examples and tools to help developers implement observability for GenAI applications. AgentCore Observability helps developers trace, debug, and monitor agent performance in production through unified operational dashboards. With support for OpenTelemetry compatible telemetry and detailed visualizations of each step of the agent workflow, Amazon CloudWatch GenAI Observability enables developers to easily gain visibility into agent behavior and maintain quality standards at scale. +This repository provides examples and tools to help developers implement observability for GenAI applications. AgentCore Observability helps developers trace, debug, and monitor agent performance in production through unified operational dashboards. With support for OpenTelemetry compatible telemetry and detailed visualizations of each step of the agent workflow, Amazon CloudWatch GenAI Observability enables developers to easily gain visibility into agent behavior and maintain standards at scale. ## Contents +Demonstrates examples using the popular Agent dveelopment fraemworks: + +- **Strands Agents**: Build LLM applications with complex workflows using model-driven agentic development +- **CrewAI**: Create autonomous AI agents that work together in roles to accomplish tasks +- **LangGraph**: Extend LangChain with stateful, multi-actor applications for complex reasoning systems +- **LlamaIndex**: LLM-powered agents over data with workflows + + ### 1. Bedrock AgentCore Runtime Hosted (01-Agentcore-runtime-hosted) -Examples demonstrating observability for Strands Agent hosted on Amazon Bedrock AgentCore Runtime using Amazon OpenTelemetry Python Instrumentation and Amazon CloudWatch. +Examples demonstrating observability for Agents hosted on Amazon Bedrock AgentCore Runtime using Amazon OpenTelemetry Python Instrumentation and Amazon CloudWatch. -### 2. Open Source Agent Frameworks (02-open-source-agents-3p) +### 2. Agent Not Hosted on Runtime (02-Agent-not-hosted-on-runtime) Examples showcasing observability for popular open-source agent frameworks not hosted on Amazon Bedrock AgentCore Runtime: -- **CrewAI**: Create autonomous AI agents that work together in roles to accomplish tasks -- **LangGraph**: Extend LangChain with stateful, multi-actor applications for complex reasoning systems -- **Strands Agents**: Build LLM applications with complex workflows using model-driven agentic development - ### 3. Advanced Concepts (03-advanced-concepts) Advanced observability patterns and techniques: - **Custom Span Creation**: Learn how to create custom spans for detailed tracing and monitoring of specific operations within your agent workflows +### 4. Partner Observability (04-Agentcore-runtime-partner-observability) + +Examples of using agents hosted on Amazon Bedrock AgentCore Runtime with third-party observability tools: + +- **Braintrust**: AI evaluation and monitoring platform +- **Langfuse**: LLM observability and analytics + ## Getting Started 1. Navigate to the directory of the framework you want to explore