Skip to content

Commit f69a4fb

Browse files
authored
Fix RedTeam sample typos (azure-ai-foundry#230)
* update promptflow-eval dependencies to azure-ai-evaluation * clear local variables * fix errors and remove 'question' col from data * small fix in evaluator config * Fix typos in RedTeam sample * Docs update
1 parent 8cd0202 commit f69a4fb

File tree

1 file changed

+13
-11
lines changed

1 file changed

+13
-11
lines changed

scenarios/evaluate/AI_RedTeaming/AI_RedTeaming.ipynb

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,11 @@
1515
"## Before you begin\n",
1616
"\n",
1717
"### Prerequisite\n",
18-
"The AI Red Teaming Agent requires an Azure AI Foundry project configuration and Azure credentials. Your project configuration will be used to log red teaming scan results after the run is finished.\n",
18+
"First, if you have an Azure subscription, create an [Azure AI hub](https://learn.microsoft.com/en-us/azure/ai-studio/concepts/ai-resources) then [create an Azure AI project](https://learn.microsoft.com/en-us/azure/ai-studio/concepts/ai-resources). AI projects and Hubs can be served within a private network and are compatible with private endpoints. You **do not** need to provide your own LLM deployment as the AI Red Teaming Agent hosts adversarial models for both simulation and evaluation of harmful content and connects to it via your Azure AI project.\n",
1919
"\n",
20-
"**Important**: Make sure to authenticate to Azure using `az login` in your terminal before running this notebook.\n",
20+
"**Note**: In order to upload your results to Azure AI Foundry, you must have the `Storage Blob Data Contributor` role\n",
21+
"\n",
22+
"**Important**: First, ensure that you've installed the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) and then make sure to authenticate to Azure using `az login` in your terminal before running this notebook.\n",
2123
"\n",
2224
"### Installation\n",
2325
"From a terminal window, navigate to your working directory which contains this sample notebook, and execute the following.\n",
@@ -95,7 +97,7 @@
9597
"AZURE_OPENAI_API_KEY=\"your-api-key-here\"\n",
9698
"AZURE_OPENAI_ENDPOINT=\"https://endpoint-name.openai.azure.com/openai/deployments/deployment-name/chat/completions\"\n",
9799
"AZURE_OPENAI_DEPLOYMENT_NAME=\"gpt-4\"\n",
98-
"AZURE_OPENAI_API_VERSION=\"2023-12-01-preview\"\n",
100+
"AZURE_OPENAI_API_VERSION=\"2024-12-01-preview\"\n",
99101
"\n",
100102
"# Azure AI Project\n",
101103
"AZURE_SUBSCRIPTION_ID=\"12345678-1234-1234-1234-123456789012\"\n",
@@ -123,7 +125,7 @@
123125
" \"AZURE_OPENAI_ENDPOINT\"\n",
124126
") # e.g., \"https://endpoint-name.openai.azure.com/openai/deployments/deployment-name/chat/completions\"\n",
125127
"azure_openai_api_key = os.environ.get(\"AZURE_OPENAI_API_KEY\") # e.g., \"your-api-key\"\n",
126-
"azure_openai_api_version = \"2023-12-01-preview\" # Use the latest API version"
128+
"azure_openai_api_version = os.environ.get(\"AZURE_OPENAI_API_VERSION\") # Use the latest API version"
127129
]
128130
},
129131
{
@@ -236,7 +238,7 @@
236238
" target=financial_advisor_callback,\n",
237239
" scan_name=\"Basic-Callback-Scan\",\n",
238240
" attack_strategies=[AttackStrategy.Flip],\n",
239-
" output_file=\"red_team_output.json\",\n",
241+
" output_path=\"red_team_output.json\",\n",
240242
")"
241243
]
242244
},
@@ -304,15 +306,15 @@
304306
" session_state: Optional[str] = None, # noqa: ARG001\n",
305307
" context: Optional[Dict[str, Any]] = None, # noqa: ARG001\n",
306308
") -> dict[str, list[dict[str, str]]]:\n",
307-
" deployment = os.environ.get(\"AZURE_DEPLOYMENT_NAME\")\n",
308-
" endpoint = os.environ.get(\"AZURE_ENDPOINT\")\n",
309-
" api_version = os.environ.get(\"AZURE_API_VERSION\")\n",
310-
"\n",
311309
" # Get token provider for Azure AD authentication\n",
312310
" token_provider = get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\")\n",
313311
"\n",
314312
" # Initialize Azure OpenAI client\n",
315-
" client = AzureOpenAI(azure_endpoint=endpoint, api_version=api_version, azure_ad_token_provider=token_provider)\n",
313+
" client = AzureOpenAI(\n",
314+
" azure_endpoint=azure_openai_endpoint,\n",
315+
" api_version=azure_openai_api_version,\n",
316+
" azure_ad_token_provider=token_provider,\n",
317+
" )\n",
316318
"\n",
317319
" ## Extract the latest message from the conversation history\n",
318320
" messages_list = [{\"role\": message.role, \"content\": message.content} for message in messages]\n",
@@ -321,7 +323,7 @@
321323
" try:\n",
322324
" # Call the model\n",
323325
" response = client.chat.completions.create(\n",
324-
" model=deployment,\n",
326+
" model=azure_openai_deployment,\n",
325327
" messages=[\n",
326328
" {\"role\": \"user\", \"content\": latest_message},\n",
327329
" ],\n",

0 commit comments

Comments
 (0)