diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 0000000..49be42c --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,46 @@ +{ + "name": "RAG with Azure SQL Database, LangChain, LangGraph, and Chainlit", + "image": "mcr.microsoft.com/devcontainers/python:0-3.11-bullseye", + "features": { + "ghcr.io/azure/azure-dev/azd:latest": {}, + "ghcr.io/devcontainers/features/azure-cli:1": { + "installBicep": true + }, + "ghcr.io/devcontainers/features/common-utils": {}, + "ghcr.io/devcontainers/features/docker-in-docker:2": {}, + "ghcr.io/devcontainers/features/dotnet:2": { + "version": "8.0" + }, + "ghcr.io/devcontainers/features/git:1": {}, + "ghcr.io/devcontainers/features/github-cli:1": {}, + "ghcr.io/devcontainers/features/node:1": { + "version": "lts" + }, + "ghcr.io/devcontainers/features/powershell:1": {} + }, + "customizations": { + "vscode": { + "extensions": [ + "ms-azuretools.vscode-azurefunctions", + "ms-azuretools.vscode-azureresourcegroups", + "ms-azuretools.vscode-bicep", + "ms-azuretools.vscode-docker", + "ms-dotnettools.csdevkit", + "ms-dotnettools.csharp", + "ms-dotnettools.vscode-dotnet-runtime", + "ms-python.python", + "ms-python.debugpy", + "ms-mssql.mssql" + ], + "settings": { + "python.defaultInterpreterPath": "./chainlit/.venv/bin/python" + } + } + }, + "postCreateCommand": "npm install -g azure-functions-core-tools@4 --unsafe-perm true && chmod +x ./infra/azd-hooks/preprovision.sh", + "remoteUser": "vscode", + "forwardPorts": [ + 7071, + 8000 + ] +} diff --git a/.vscode/launch.json b/.vscode/launch.json index 894cbe6..2bf7d00 100644 --- a/.vscode/launch.json +++ b/.vscode/launch.json @@ -6,6 +6,21 @@ "type": "coreclr", "request": "attach", "processId": "${command:azureFunctions.pickProcess}" + }, + { + "name": "Chainlit Debugger (Windows)", + "type": "debugpy", + "request": "launch", + "program": "${workspaceFolder}/chainlit/.venv/Scripts/chainlit.exe", + "args": [ + "run", + "${workspaceFolder}/chainlit/app.py" + ], + "jinja": true, + "env": { + "PYTHONPATH": "${workspaceFolder}/chainlit" + }, + "python": "${workspaceFolder}/chainlit/.venv/Scripts/python.exe", } ] } \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..9f47941 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,81 @@ +# Contributing to [project-title] + +This project welcomes contributions and suggestions. Most contributions require you to agree to a +Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us +the rights to use your contribution. For details, visit . + +When you submit a pull request, a CLA bot will automatically determine whether you need to provide +a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions +provided by the bot. You will only need to do this once across all repos using our CLA. + +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). +For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or +contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. + +- [Code of Conduct](#coc) +- [Issues and Bugs](#issue) +- [Feature Requests](#feature) +- [Submission Guidelines](#submit) + +## Code of Conduct + +Help us keep this project open and inclusive. Please read and follow our [Code of Conduct](https://opensource.microsoft.com/codeofconduct/). + +## Found an Issue? + +If you find a bug in the source code or a mistake in the documentation, you can help us by +[submitting an issue](#submit-issue) to the GitHub Repository. Even better, you can +[submit a Pull Request](#submit-pr) with a fix. + +## Want a Feature? + +You can *request* a new feature by [submitting an issue](#submit-issue) to the GitHub +Repository. If you would like to *implement* a new feature, please submit an issue with +a proposal for your work first, to be sure that we can use it. + +* **Small Features** can be crafted and directly [submitted as a Pull Request](#submit-pr). + +## Submission Guidelines + +### Submitting an Issue + +Before you submit an issue, search the archive, maybe your question was already answered. + +If your issue appears to be a bug, and hasn't been reported, open a new issue. +Help us to maximize the effort we can spend fixing issues and adding new +features, by not reporting duplicate issues. Providing the following information will increase the +chances of your issue being dealt with quickly: + +* **Overview of the Issue** - if an error is being thrown a non-minified stack trace helps +* **Version** - what version is affected (e.g. 0.1.2) +* **Motivation for or Use Case** - explain what are you trying to do and why the current behavior is a bug for you +* **Browsers and Operating System** - is this a problem with all browsers? +* **Reproduce the Error** - provide a live example or a unambiguous set of steps +* **Related Issues** - has a similar issue been reported before? +* **Suggest a Fix** - if you can't fix the bug yourself, perhaps you can point to what might be + causing the problem (line of code or commit) + +You can file new issues by providing the above information at the corresponding repository's issues link: https://github.com/[organization-name]/[repository-name]/issues/new]. + +### Submitting a Pull Request (PR) + +Before you submit your Pull Request (PR) consider the following guidelines: + +* Search the repository (https://github.com/[organization-name]/[repository-name]/pulls) for an open or closed PR + that relates to your submission. You don't want to duplicate effort. + +* Make your changes in a new git fork: + +* Commit your changes using a descriptive commit message +* Push your fork to GitHub: +* In GitHub, create a pull request +* If we suggest changes then: + * Make the required updates. + * Rebase your fork and force push to your GitHub repository (this will update your Pull Request): + + ```shell + git rebase master -i + git push -f + ``` + +That's it! Thank you for your contribution! \ No newline at end of file diff --git a/README.md b/README.md index 3953b99..f272246 100644 --- a/README.md +++ b/README.md @@ -1,103 +1,395 @@ # Azure SQL DB, Langchain, LangGraph and Chainlit -Sample RAG pattern using Azure SQL DB, Langchain and Chainlit as demonstrated in the [#RAGHack](https://github.com/microsoft/RAG_Hack) conference. Full details and video recording available here: [RAG on Azure SQL Server](https://github.com/microsoft/RAG_Hack/discussions/53). +This repository showcases a simple AI-powered chat application built using Python and leveraging the power of [Chainlit](https://docs.chainlit.io/get-started/overview) and [LangChain](https://www.langchain.com/) to implement retrieval-augmented generation (RAG). The project, which stems from the innovative demonstrations presented at the [#RAGHack](https://github.com/microsoft/RAG_Hack) event, allows users to submit natural language queries about event sessions and speakers using [vector similarity search with Azure SQL and Azure OpenAI](https://learn.microsoft.com/en-us/samples/azure-samples/azure-sql-db-openai/azure-sql-db-openai/). For additional insights and to access the recording, visit the discussion: [RAG on Azure SQL Server](https://github.com/microsoft/RAG_Hack/discussions/53). -The sample is build using plain LangChain (`app.py`) or using LangGraph (`app-langgraph.py`) to define the RAG process. +The application provides two implementations of the RAG process: + +1. A straightforward version using LangChain (`app.py`) +2. A dynamic approach using LangGraph (`app-langgraph.py`) + +The app demonstrates how Azure SQL Database and the RAG pattern can enhance data retrieval and utilization in a copilot chat application. ## Architecture -![Architecture](./_assets/architecture.png) +![Application architecture](./_assets/architecture.png) + +The Chainlit application implements the RAG pattern, utilizing LangChain and function calling to perform vector similarity searches over data stored in an Azure SQL database. Within the database, a stored procedure is used to call an Azure OpenAI endpoint to generate vector embeddings representing the search query using an embedding model. These embeddings are then compared against previously stored vectors using Azure SQL's built-in [VECTOR_DISTANCE function](https://learn.microsoft.com/sql/t-sql/functions/vector-distance-transact-sql?view=azuresqldb-current) to identify sessions and speakers that are semantically similar to the query. Vector representations of speaker names and session titles and abstracts are stored in the database using Azure SQL Database's native [Vector data type](https://learn.microsoft.com/sql/t-sql/data-types/vector-data-type?view=azuresqldb-current&tabs=csharp-sample). + +> [!NOTE] +> Vector Functions are in Public Preview. Learn the details about vectors in Azure SQL here: -## Solution +Embeddings stored in the database can be kept up-to-date using an Azure Function App and [Azure SQL Trigger Binding](https://learn.microsoft.com/azure/azure-functions/functions-bindings-azure-sql-trigger). Whenever changes occur in designated tables in the database, the function is triggered, directly calling Azure OpenAI to generate embeddings for new or updated records. The vectors are then written into the database. -The solution works locally and in Azure. The solution is composed of three main Azure components: +The application is composed of three main Azure components: -- [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/sql-database-paas-overview?view=azuresql): The database that stores the data. -- [Azure Open AI](https://learn.microsoft.com/azure/ai-services/openai/): The language model that generates the text and the embeddings. +- [Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/database/sql-database-paas-overview?view=azuresql): The Azure SQL database that stores application data. +- [Azure Open AI](https://learn.microsoft.com/azure/ai-services/openai/): Hosts the language models for generating embeddings and completions. - [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-csharp): The serverless function to automate the process of generating the embeddings (this is optional for this sample) +The solution can be run locally or in Azure. However, running the complete solution in Azure requires the Chainlit application to be containerized and deployed to Azure using one of the available options for hosting containerized applications, such as Azure Container Apps or Azure Kubernetes Service (AKS). The steps for that are not covered in the instructions for this solution. -### Azure Open AI +## Try It Out -Make sure to have two models deployed, one for generating embeddings (*text-embedding-3-small* model recommended) and one for handling the chat (*gpt-4 turbo* recommended). You can use the Azure OpenAI service to deploy the models. Make sure to have the endpoint and the API key ready. The two models are assumed to be deployed with the following names: +The fastest way to try out the sample application is to follow the instructions below for creating a GitHub Codespace and deploying the required Azure resources by running the [Azure Developer CLI](https://learn.microsoft.com/azure/developer/azure-developer-cli/overview?tabs=windows) template and Bicep scripts included in the repo. -- Embedding model: `text-embedding-3-small` -- Chat model: `gpt-4` +You can also deploy resources manually or use existing ones in your Azure subscription. +## Set Up Your Development Environment -### Database +The recommended approach to setting up a development environment for this solution is utilizing **GitHub Codespaces**, which offers a streamlined, ready-to-use workspace requiring minimal configuration. If you can't access codespaces or would rather work locally, this section includes instructions to help you configure **VS Code** locally for a smooth development experience. -> [!NOTE] -> Vector Functions are in Public Preview. Learn the details about vectors in Azure SQL here: https://aka.ms/azure-sql-vector-public-preview +### Create a Codespace + +GitHub Codespaces provides a secure, customizable workspace built on a development container specification, enabling a seamless coding experience. These development containers, called dev containers, are Docker-based environments tailored for software development. When working within a codespace, your code runs inside a dev container hosted on a virtual machine, ensuring consistency and efficiency in your workflow. + +This solution includes a preconfigured dev container that you can use to set up a codespace on GitHub. You can view the dev container specification for this project in the repo file `.devcontainer/devcontainer.json`. + +To set up a codespace: + +1. Navigate to the [Azure SQL DB LangChain RAG with Chainlit repo](https://aka.ms/azuresqldb-rag-langchain-chainlit). +2. Fork the repo. +3. On your fork, create a codespace by selecting **Code --> Codespaces --> Create codespace...**. +4. The codespace will take about 10 minutes to be created and fully configured. + +Once your codespace has been created, you will **complete the remaining tasks from the VS Code instance within your codespace** environment. You can skip to the [Provision Azure Resources](#provision-azure-resources) section below. + +### Use Your Local Machine + +If you cannot access GitHub Codespaces or prefer to use your local machine, follow the instructions below to configure a development environment on your local machine using Visual Studio Code and Python. + +Install the following: + +1. [Visual Studio Code](https://code.visualstudio.com/download) with the following extensions: + + - [Azure Functions](https://marketplace.visualstudio.com/items/?itemName=ms-azuretools.vscode-azurefunctions) + - [Azure Account](https://marketplace.visualstudio.com/items/?itemName=ms-vscode.azure-account) + - [Azure Resources](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-azureresourcegroups) + - [Bicep](https://marketplace.visualstudio.com/items/?itemName=ms-azuretools.vscode-bicep) + - [Docker](https://marketplace.visualstudio.com/items/?itemName=ms-azuretools.vscode-docker) + - [C#](https://marketplace.visualstudio.com/items/?itemName=ms-dotnettools.csharp) + - [C# Dev Kit](https://marketplace.visualstudio.com/items/?itemName=ms-dotnettools.csdevkit) + - [.NET Install Tool](https://marketplace.visualstudio.com/items/?itemName=ms-dotnettools.vscode-dotnet-runtime) + - [Python](https://marketplace.visualstudio.com/items/?itemName=ms-python.python) + - [Python Debugger](https://marketplace.visualstudio.com/items/?itemName=ms-python.debugpy) + - [MSSQL](https://marketplace.visualstudio.com/items/?itemName=ms-mssql.mssql) + +2. [Azure Function Core Tools](https://learn.microsoft.com/azure/azure-functions/functions-run-local?tabs=windows%2Cisolated-process%2Cnode-v4%2Cpython-v2%2Chttp-trigger%2Ccontainer-apps&pivots=programming-language-csharp) + +3. [Python 3.11](https://www.python.org/downloads/release/python-31112/) + +4. [Microsoft ODBC Driver for SQL Server](https://learn.microsoft.com/sql/connect/odbc/microsoft-odbc-driver-for-sql-server?view=sql-server-ver16) + +After installing the above tools, clone the [Azure SQL DB LangChain RAG with Chainlit repo](https://aka.ms/azuresqldb-rag-langchain-chainlit) and open it in VS Code. + +## Provision Azure Resources + +The solution requires an Azure SQL database and an Azure OpenAI service. You can optionally deploy an Azure Function App to automate the generation of vector embeddings for new and updated data. + +Using the provided Azure Developer CLI template to deploy the required resources into your Azure subscription is the easiest way to get started with this solution. However, you can also opt to manually deploy the resources or use existing services in your subscription. + +### Using Azure Developer CLI Template + +The template provided in this repo will deploy a small Azure SQL database and an Azure OpenAI service with the required `gpt-4o` and `text-embedding-ada-002` models. It can also be used to deploy an Azure Function App. + +Azure Developer CLI templates are run using the `azd up` command. Follow the steps below to provision the necessary Azure resources using the template: + +1. Open a new terminal windows in VS Code. + +2. Sign into Azure using the Azure CLI: + + ```bash + az login + ``` + +3. Sign in to the Azure Developer CLI: + + ```bash + azd auth login + ``` + +4. Run the deployment command to execute the included Bicep scripts: + + ```bash + azd up + ``` + +5. After issuing the `azd up` command, you will be prompted for several values. Provide the requested information to provision and deploy the Azure resources: + + - Enter an environment name, such as "dev" or "test." + - Select the subscription you want to use for the resources for this solution. + - Indicate if you want to deploy an Azure Function App. + + > Deploying an Azure Function app is an optional component of the solution. You can run and test the function in your codespace or locally if you prefer not to deploy the function to Azure. + + - Select the Azure region where you would like to deploy the resources for this solution. + + > [!NOTE] + > The Bicep script will deploy an Azure OpenAI service and create deployments for the `gpt-4o` and `text-embedding-ada-002` models. To ensure the deployment succeeds, review the [regional availability for Azure OpenAI models](https://learn.microsoft.com/azure/ai-services/openai/concepts/models?tabs=standard%2Cstandard-chat-completions#models-by-deployment-type) before selecting a region. You should also verify you have at least 10,000 TPMs of available capacity for each model. + + - Enter a resource group name, such as `rg-azure-sql-rag-app`. + - Enter a strong password for the Azure SQL Admin account. Note that this is needed to deploy the database, but the Bicep script will configure the SQL Server and database to use Microsoft Entra ID authentication only. + +6. Skip to [Deploy Sample Database](#deploy-sample-database) below. + +### Manually Provision Resources + +You can manually deploy the required Azure resources using the Azure CLI or Azure portal. You must create an Azure SQL database and Azure OpenAI service. Optionally, you can provision an Azure Function App to host the sample function. + +For the database, ensure you add the IP address of your local machine to the SQL Server's firewall. + +To run the solution, you must deploy two models to your Azure Open AI service: + +1. A text embedding model for generating embeddings (*text-embedding-ada-002* model recommended) +2. A chat model for handling the chat (*gpt-4o* recommended) + +You can use the Azure AI Foundry portal to deploy the required models into your Azure OpenAI service. The two models are assumed to be deployed with the following names: + +- Chat model: `gpt-4o` +- Embedding model: `text-embedding-ada-002` + +Once your resources are deployed, skip to [Deploy Sample Database](#deploy-sample-database) below. + +### Use Existing Azure Resources + +Leveraging an existing Azure SQL database and Azure OpenAI service in your subscription to run the solution is also possible. Using that approach, you must deploy the sample database onto your Azure SQL server and ensure the correct models are deployed into your Azure OpenAI service. + +Ensure your Azure Open AI service has two models deployed, one for generating embeddings (*text-embedding-ada-002* model recommended) and one for handling the chat (*gpt-4o* recommended). You can use the Azure AI Foundry portal to deploy the models into your Azure OpenAI service. The two models are assumed to be deployed with the following names: + +- Chat model: `gpt-4o` +- Embedding model: `text-embedding-ada-002` + +## Deploy Sample Database + +To deploy the database, you can either use the provided .NET 8 Core console application or do it manually by running the scripts in the solution's `/database/sql` folder. + +> [!NOTE] +> You should wait at least 5 minutes after completing your Azure resource deployment to run the database scripts to ensure the deployed embedding model endpoint in Azure OpenAI is available and ready to generate embeddings. + +### Use Provided .NET 8 Core Console App + +To use the .NET 8 Core console application to deploy the database: + +1. Navigate to the `/database` folder in the repo. +2. Create a `.env` file in the `/database` folder, using the `.env.example` file as a starter, and populate the following environment variables with values from your environment: + + - `MSSQL`: The connection string to the Azure SQL database where you want to deploy the database objects and sample data. + + - If you used the Azure Developer CLI template, select the `ADO.NET` (Microsoft Entra passwordless authentication) connection string from your database's connection strings page. Otherwise, select the connection string appropriate for the authentication method you set up on your database. -To deploy the database, you can either the provided .NET 8 Core console application or do it manually. + - `OPENAI_URL`: Provide the URL of your Azure OpenAI endpoint, e.g., ''. **Ensure the URL ends with "/"**. -To use .NET 8 Core console application move into the `/database` and then make sure to create a `.env` file in the `/database` folder starting from the `.env.example` file: + - `OPENAI_KEY`: Specify the API key of your Azure OpenAI endpoint. + - `OPENAI_MODEL`: Provide the deployment name of your Azure OpenAI embedding endpoint (e.g., 'text-embedding-ada-002'). -- `MSSQL`: the connection string to the Azure SQL database where you want to deploy the database objects and sample data -- `OPENAI_URL`: specify the URL of your Azure OpenAI endpoint, eg: 'https://my-open-ai.openai.azure.com/' -- `OPENAI_KEY`: specify the API key of your Azure OpenAI endpoint -- `OPENAI_MODEL`: specify the deployment name of your Azure OpenAI embedding endpoint, eg: 'text-embedding-3-small' +To run the .NET 8 Core console application: -If you want to deploy the database manually, make sure to execute the script in the `/database/sql` folder in the order specifed by the number in the file name. Some files (`020-security.sql` and `060-get_embedding.sql`) with have placeholders that you have to replace with your own values: +1. Open a new integrated terminal in VS Code. -- `$OPENAI_URL$`: replace with the URL of your Azure OpenAI endpoint, eg: 'https://my-open-ai.openai.azure.com/' +2. Sign into Azure using the Azure CLI: + + ```bash + az login + ``` + +3. At the terminal prompt, change directories to the `/database` folder: + + ```bash + cd database + ``` + +4. Build the database project: + + ```bash + dotnet build + ``` + +5. Run the database project: + + ```bash + dotnet run + ``` + +### Manual Database Deployment + +If you prefer to deploy the database manually, you must run each of the scripts in the solution's `/database/sql` folder against your database. Ensure you execute the scripts in the `/database/sql` folder in the order specified by the number in the file name. Some files (`020-security.sql` and `060-get_embedding.sql`) have placeholders that you must replace with your values: + +- `$OPENAI_URL$`: replace with the URL of your Azure OpenAI endpoint, e.g., '' - `$OPENAI_KEY$`: replace with the API key of your Azure OpenAI endpoint -- `$OPENAI_MODEL$`: replace with the deployment name of your Azure OpenAI embedding endpoint, eg: 'text-embedding-3-small' +- `$OPENAI_MODEL$`: replace with the deployment name of your Azure OpenAI embedding model, e.g., 'text-embedding-ada-002' + +## Verify Embeddings In Database + +Vector embeddings must be inserted into the `[web].[sessions]` and `[web].[speakers]` tables in the database prior to running the Chainlit application. The database deployment scripts executed in the previous step should have accomplished this, but the embeddings should be verified before proceeding. + +1. Connect to your database using your preferred database management tool, such as VS Code and the MSSQL extension, SSMS, Query Editor in the Azure Portal, or Azure Data Studio. + +2. Select the top 5 rows from the `[web].[sessions]` table and verify the `embeddings` column contains vector arrays for each row. + +3. Repeat the above step against the `[web].[speakers]` table. + +If your tables do not contain embeddings, you must manually rerun the `040-tables.sql` and `100-sample-data.sql` scripts in the solution's `/database/sql` folder against your database. These scripts will drop and recreate the tables and then attempt to repopulate them with sample data and embeddings. If there is an error generating embeddings, you will see it when executing the `100-sample-data.sql` script. Typical errors include not waiting long enough after deploying your embedding model to Azure OpenAI before running the script and not having an embedding model deployment with the expected name of `text-embedding-ada-002` in your Azure OpenAI service. + +## Run the Chainlit App + +The Chainlit application provides a simple Python-based chat interface for interacting with data in your database. The Chainlit solution is located in the `chainlit` folder. To get started with the application: + +1. Open a new integrated terminal in VS Code. + +2. Sign into Azure using the Azure CLI: + + ```bash + az login + ``` + +3. Change directories to the `/chainlit` folder. + + ```bash + cd chainlit + ``` + +4. Create a Python virtual environment named `.venv`: + + ```bash + python -m venv .venv + ``` + +5. Activate the virtual environment: + + On Windows: + + ```powershell + .venv\Script\activate + ``` + + On Linux or Mac: + + ```bash + source .venv/bin/activate + ``` -### Chainlit +6. Install the required Python libraries from the `requirements.txt` file: -Chainlit solution is in `chainlit` folder. Move into the folder, create a virtual environment and install the requirements: + ```bash + pip install -r requirements.txt + ``` -```bash -python -m venv .venv -source .venv/bin/activate -pip install -r requirements.txt -``` +7. In the VS Code Solution Explorer, navigate to the `/chainlit` folder, create a `.env` file, starting from the `.env.example` file, and populate it with the values for your environment. -or, on Windows: + Microsoft Entra ID Auth is required if you used the Azure Developer CLI template to deploy your database. Therefore, your `AZURE_SQL_CONNECTION_STRING` variable should look like the following, with the `[YOUR_SQL_SERVER_NAME]` token replaced with the name of your SQL server in Azure. If you named your database differently, you must also update the `Database` value. -```PowerShell -python -m venv .venv -.venv/Script/activate -pip install -r requirements.txt -``` + ```ini + AZURE_SQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=tcp:[YOUR_SQL_SERVER_NAME].database.windows.net,1433;Database=sessiondb;Encrypt=yes;Connection Timeout=30;' + ``` -Then make sure to create a `.env` file in the `/chainlit` folder starting from the `.env.example` file and it with your own values, then run the chainlit solution: +8. If you are using codespaces, you must install the ODBC Driver for SQL Server on the container: -```bash -chainlit run app.py -``` + ```bash + sudo su + ``` -or + ```bash + curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - + ``` -```bash -chainlit run app-langgraph.py -``` + ```bash + curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list + ``` -if you want to use the LangGraph solution. + ```bash + exit + ``` -Once the application is running, you'll be able to ask question about your data and get the answer from the Azure OpenAI model. For example you can ask question on the data you have in the database: + ```bash + sudo apt-get update + ``` -``` -Is there any session on Retrieval Augmented Generation? -``` + ```bash + sudo ACCEPT_EULA=Y apt-get install -y msodbcsql18 unixodbc unixodbc-dev + ``` -You'll see that Langchain will call the function `get_similar_sessions` that behind the scenes connects to the database and excute the stored procedure `web.find_sessions` which perform vector search on database data. +9. Then, run the chainlit solution: + + ```bash + chainlit run app.py + ``` + + Or, if you want to use the LangGraph solution: + + ```bash + chainlit run app-langgraph.py + ``` + +10. Once the application is running, you can ask questions about your data and get the answer from the Azure OpenAI model. For example, you can ask questions about the session topics you have in the database: + + - Are there any sessions on retrieval-augmented generation? + - Show me sessions featuring Azure SQL. + +You'll see that Langchain will call the function `get_similar_sessions` that, behind the scenes, connects to the database and executes the stored procedure `web.find_sessions`, which performs vector search on database data. The RAG process is defined using Langchain's LCEL [Langchain Expression Language](https://python.langchain.com/v0.1/docs/expression_language/) that can be easily extended to include more complex logic, even including complex agent actions with the aid of [LangGraph](https://langchain-ai.github.io/langgraph/), where the function calling the stored procedure will be a [tool](https://langchain-ai.github.io/langgraph/how-tos/tool-calling/?h=tool) available to the agent. +## Run the Azure Function App (optional) + +To automate the process of generating the embeddings for new and updated session and speaker data, you can use Azure Functions. Thanks to [Azure SQL Trigger Binding](https://learn.microsoft.com/azure/azure-functions/functions-bindings-azure-sql-trigger), it is possible to have tables monitored for changes and then react to those changes by executing code in the Azure Function itself. As a result, it is possible to automate the process of generating the embeddings and storing them in the database. + +In this solution, the Azure Function is written in C#, but you can easily create the same solution using Python, Node.js or any other supported language. The Azure Functions solution is in the `azure-functions` folder. + +1. Navigate to the `azure-functions` folder in the solution, then create a `local.settings.json` starting from the provided `local.settings.json.example` file, and populate it with your values. + +2. Run the Azure Function locally (ensure you have the [Azure Function core tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local) installed): + + ```bash + func start + ``` + + The Azure Function will monitor the configured tables for changes and automatically call the Azure OpenAI endpoint to generate the embeddings for the new or updated data. + +3. To trigger the function, run the following query against your database to update the `require_embeddings_update` flag in the `web.speakers` table: + + ```sql + UPDATE [web].[speakers] SET [require_embeddings_update] = 1; + ``` + +4. Observe the function output in the VS Code integrated terminal window. You should see update changes being processed against the `web.speakers` table. + +You can also choose to deploy the function to your Function App instance in Azure if you opted to deploy that service. + +## Cleanup + +Once you have completed this workshop, delete the Azure resources you created. You are charged for the configured capacity, not how much the resources are used. Follow these instructions to delete your resource group and all resources you created for this solution accelerator. + +### Persist changes to GitHub + +If you want to save any changes you have made to files, use the Source Control tool in VS Code to commit and push your changes to your fork of the GitHub repo. + +### Azure Developer CLI Deployed Resources + +1. In VS Code, open a new integrated terminal prompt. + +2. At the terminal prompt, execute the following command to delete the resources created by the deployment script: + + ```bash + azd down --purge + ``` + + The `--purge` flag purges the resources that provide soft-delete functionality in Azure, including Azure KeyVault and Azure OpenAI. This flag is required to remove all resources completely. + +3. In the terminal window, you will be shown a list of the resources that will be deleted and prompted about continuing. Enter "y" at the prompt to being the resource deletion. + +### Manually Provisioned Resources + +1. In the Azure portal, select the resource group to which you deployed resources. + +2. Delete the resource group. -### Azure Functions (optional) +## Code walkthoughh -In order to automate the process of generating the embeddings, you can use the Azure Functions. Thanks to [Azure SQL Trigger Binding](https://learn.microsoft.com/azure/azure-functions/functions-bindings-azure-sql-trigger), it is possible to have tables monitored for changes and then react to those changes by executing some code in the Azure Function itself. As a result it is possible to automate the process of generating the embeddings and storing them in the database. +Refer to the [code walkthrough document](walkthrough.md) for a details explanation of the code structure and how the different compontents work together. -In a perfect microservices architecture, the Azure Functions are written in C#, but you can easily create the same solutoin using Python, Node.js or any other supported language. +## Contributing -The Azure Functions solution is in the `azure-functions` folder. Move into the folder, then create a `local.settings.json` starting from the provided `local.settings.json.example` file and fill it with your own values. Then run the Azure Functions locally (make sure to have the [Azure Function core tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local) installed): +Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests. -```bash -func start -``` +## License -the Azure Function will monitor the configured tables for changes and automatically call the Azure OpenAI endpoint to generate the embeddings for the new or updated data. +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. diff --git a/azure-function/local.settings.json.example b/azure-function/local.settings.json.example index 2014697..6a204b1 100644 --- a/azure-function/local.settings.json.example +++ b/azure-function/local.settings.json.example @@ -3,9 +3,9 @@ "Values": { "AzureWebJobsStorage": "UseDevelopmentStorage=true", "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated", - "AZURE_SQL_CONNECTION_STRING": "Server=.database.windows.net;Initial Catalog=;Persist Security Info=False;Authentication=Active Directory Default;MultipleActiveResultSets=False;Encrypt=True;Connection Timeout=30;", + "AZURE_SQL_CONNECTION_STRING": "Server=.database.windows.net;Initial Catalog=sessiondb;Persist Security Info=False;Authentication=Active Directory Default;MultipleActiveResultSets=False;Encrypt=True;Connection Timeout=30;", "AZURE_OPENAI_ENDPOINT": "https://.openai.azure.com/", "AZURE_OPENAI_KEY": "", - "AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME": "text-embedding-3-small" + "AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME": "text-embedding-ada-002" } } \ No newline at end of file diff --git a/azure-sql-db-rag-langchain-chainlit.sln b/azure-sql-db-rag-langchain-chainlit.sln new file mode 100644 index 0000000..220bad3 --- /dev/null +++ b/azure-sql-db-rag-langchain-chainlit.sln @@ -0,0 +1,30 @@ +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio Version 17 +VisualStudioVersion = 17.5.2.0 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FunctionTrigger", "azure-function\FunctionTrigger.csproj", "{9BA6CC7F-CA88-D4E9-C133-141055E35754}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Database.Deploy", "database\Database.Deploy.csproj", "{F0C8EE28-EDD0-6EE4-F337-02AE517F7A66}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Release|Any CPU = Release|Any CPU + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {9BA6CC7F-CA88-D4E9-C133-141055E35754}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {9BA6CC7F-CA88-D4E9-C133-141055E35754}.Debug|Any CPU.Build.0 = Debug|Any CPU + {9BA6CC7F-CA88-D4E9-C133-141055E35754}.Release|Any CPU.ActiveCfg = Release|Any CPU + {9BA6CC7F-CA88-D4E9-C133-141055E35754}.Release|Any CPU.Build.0 = Release|Any CPU + {F0C8EE28-EDD0-6EE4-F337-02AE517F7A66}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {F0C8EE28-EDD0-6EE4-F337-02AE517F7A66}.Debug|Any CPU.Build.0 = Debug|Any CPU + {F0C8EE28-EDD0-6EE4-F337-02AE517F7A66}.Release|Any CPU.ActiveCfg = Release|Any CPU + {F0C8EE28-EDD0-6EE4-F337-02AE517F7A66}.Release|Any CPU.Build.0 = Release|Any CPU + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection + GlobalSection(ExtensibilityGlobals) = postSolution + SolutionGuid = {CA9365A7-2CAE-4950-9BE7-D2B30E6623EA} + EndGlobalSection +EndGlobal diff --git a/azure.yaml b/azure.yaml new file mode 100644 index 0000000..e6e07b6 --- /dev/null +++ b/azure.yaml @@ -0,0 +1,26 @@ +name: azure-sql-rag-app +metadata: + location: ${AZURE_REGION} + environment: ${AZURE_ENVIRONMENT} + subscription: ${AZURE_SUBSCRIPTION_ID} + clientIpAddress: ${CLIENT_IP_ADDRESS} + principalName: ${PRINCIPAL_NAME} +workflows: + up: + steps: + - azd: provision + - azd: deploy +infra: + bicep: ./infra/main.bicep +hooks: + preprovision: + posix: + shell: sh + continueOnError: false + interactive: false + run: ./infra/azd-hooks/preprovision.sh + windows: + shell: pwsh + continueOnError: false + interactive: false + run: ./infra/azd-hooks/preprovision.ps1 diff --git a/chainlit.md b/chainlit.md new file mode 100644 index 0000000..4507ac4 --- /dev/null +++ b/chainlit.md @@ -0,0 +1,14 @@ +# Welcome to Chainlit! 🚀🤖 + +Hi there, Developer! 👋 We're excited to have you on board. Chainlit is a powerful tool designed to help you prototype, debug and share applications built on top of LLMs. + +## Useful Links 🔗 + +- **Documentation:** Get started with our comprehensive [Chainlit Documentation](https://docs.chainlit.io) 📚 +- **Discord Community:** Join our friendly [Chainlit Discord](https://discord.gg/k73SQ3FyUh) to ask questions, share your projects, and connect with other developers! 💬 + +We can't wait to see what you create with Chainlit! Happy coding! 💻😊 + +## Welcome screen + +To modify the welcome screen, edit the `chainlit.md` file at the root of your project. If you do not want a welcome screen, just leave this file empty. diff --git a/chainlit/.env.example b/chainlit/.env.example index 1a807ed..8a7d975 100644 --- a/chainlit/.env.example +++ b/chainlit/.env.example @@ -1,6 +1,6 @@ AZURE_OPENAI_ENDPOINT="https://.openai.azure.com/" AZURE_OPENAI_API_KEY="" -AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="gpt-4" -AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME="text-embedding-3-small" -AZURE_OPENAI_API_VERSION="2024-02-01" -AZURE_SQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=.database.windows.net,1433;Database=;Encrypt=yes;Connection Timeout=30' \ No newline at end of file +AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="gpt-4o" +AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME="text-embedding-ada-002" +AZURE_OPENAI_API_VERSION="2024-10-21" +AZURE_SQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=.database.windows.net,1433;Database=sessiondb;Encrypt=yes;Connection Timeout=30' \ No newline at end of file diff --git a/chainlit/requirements.txt b/chainlit/requirements.txt index bd27947..0955968 100644 --- a/chainlit/requirements.txt +++ b/chainlit/requirements.txt @@ -1,14 +1,9 @@ python-dotenv - pyodbc azure-identity - langchain langchain-openai langchain-community langchain-sqlserver langgraph - chainlit - - diff --git a/chainlit/utilities.py b/chainlit/utilities.py index 5824a16..2ff7ad6 100644 --- a/chainlit/utilities.py +++ b/chainlit/utilities.py @@ -7,36 +7,34 @@ def get_mssql_connection(source_variable_name: str) -> pyodbc.Connection: logging.info('Getting MSSQL connection') - mssql_connection_string = os.environ[source_variable_name] - if any(s in mssql_connection_string.lower() for s in ["uid"]): + mssql_connection_string = os.environ[source_variable_name] + if any(s in mssql_connection_string.lower() for s in ["uid"]) and any(s in mssql_connection_string.lower() for s in ["pwd"]): logging.info('Using SQL Server authentication') attrs_before = None else: - logging.info('Getting EntraID credentials...') - credential = identity.DefaultAzureCredential(exclude_interactive_browser_credential=False) - token_bytes = credential.get_token("https://database.windows.net/.default").token.encode("UTF-16-LE") + logging.info('Getting EntraID credentials...') + credential = identity.DefaultAzureCredential(exclude_interactive_browser_credential=False) + token_bytes = credential.get_token("https://database.windows.net/.default").token.encode("UTF-16-LE") token_struct = struct.pack(f' str: """ - Use this function to get a list of sessions that are potentially relevant for the specified topic. - The sessions are provided in the format of `id|title|abstract|speakers|start-time|end-time`. - + The sessions are provided in the format of `id|title|abstract|speakers|start-time|end-time`. """ conn = get_mssql_connection("AZURE_SQL_CONNECTION_STRING") logging.info("Querying MSSQL...") logging.info(f"Topic: '{topic}'") - try: - cursor = conn.cursor() + try: + cursor = conn.cursor() results = cursor.execute("SET NOCOUNT ON; EXEC web.find_sessions @text=?", (topic)).fetchall() logging.info(f"Found {len(results)} similar sessions.") @@ -44,13 +42,13 @@ def get_similar_sessions(topic:str) -> str: payload = "" for row in results: description = str(row[2]).replace("\n", " ") - speakers = ", ".join(json.loads(row[7])) - payload += f'{row[0]}|{row[1]}|{description}|{speakers}|{row[4]}|{row[5]}' + speakers = ", ".join(json.loads(row[7])) + payload += f'{row[0]}|{row[1]}|{description}|{speakers}|{row[4]}|{row[5]}' payload += "\n" - - return payload + + return payload finally: - cursor.close() + cursor.close() if __name__ == "__main__": from dotenv import load_dotenv diff --git a/database/.env.example b/database/.env.example index 83dca07..862289d 100644 --- a/database/.env.example +++ b/database/.env.example @@ -1,4 +1,4 @@ -MSSQL='Server=.database.windows.net;Initial Catalog=;Persist Security Info=False;User ID=sampledb;Authentication=Active Directory Default;Connection Timeout=30;' -OPENAI_URL='https://.openai.azure.com' +MSSQL='Server=.database.windows.net;Initial Catalog=;Persist Security Info=False;User ID=sessiondb;Authentication=Active Directory Default;Connection Timeout=30;' +OPENAI_URL='https://.openai.azure.com/' OPENAI_KEY='' -OPENAI_MODEL='text-embedding-3-small' \ No newline at end of file +OPENAI_MODEL='text-embedding-ada-002' \ No newline at end of file diff --git a/database/sql/040-tables.sql b/database/sql/040-tables.sql index d9a8118..bf09ff0 100644 --- a/database/sql/040-tables.sql +++ b/database/sql/040-tables.sql @@ -1,3 +1,23 @@ +/* + This script creates the tables for the web application, including the searched_text, sessions, + speakers, and sessions_speakers tables. +*/ + +/* + Table name: searched_text + Description: + This table stores the searched text, the date and time of the search, and the performance + metrics for the search. +*/ +DROP TABLE IF EXISTS [web].[searched_text]; +GO +DROP TABLE IF EXISTS [web].[sessions_speakers]; +GO +DROP TABLE IF EXISTS [web].[speakers]; +GO +DROP TABLE IF EXISTS [web].[sessions]; +GO + CREATE TABLE [web].[searched_text] ( [id] INT IDENTITY (1, 1) NOT NULL, @@ -11,6 +31,12 @@ CREATE TABLE [web].[searched_text] ); GO +/* + Table name: sessions + Description: + This table stores the session information, including the title, abstract, external ID, + last fetched date and time, start and end times, tags, recording URL, and embeddings. +*/ CREATE TABLE [web].[sessions] ( [id] INT DEFAULT (NEXT VALUE FOR [web].[global_id]) NOT NULL, @@ -31,6 +57,12 @@ CREATE TABLE [web].[sessions] ); GO +/* + Table name: speakers + Description: + This table stores the speaker information, including the external ID, full name, + require_embeddings_update flag, and embeddings. +*/ CREATE TABLE [web].[speakers] ( [id] INT DEFAULT (NEXT VALUE FOR [web].[global_id]) NOT NULL, @@ -44,6 +76,12 @@ CREATE TABLE [web].[speakers] ); GO +/* + Table name: sessions_speakers + Description: + This table establishes a many-to-many relationship between sessions and speakers. + It contains the session ID and speaker ID as foreign keys. +*/ CREATE TABLE [web].[sessions_speakers] ( [session_id] INT NOT NULL, [speaker_id] INT NOT NULL, @@ -54,6 +92,9 @@ CREATE TABLE [web].[sessions_speakers] ( ); GO +/* + Create non-clustered indexes on the sessions_speakers tables to improve query performance. +*/ CREATE NONCLUSTERED INDEX [ix2] ON [web].[sessions_speakers]([speaker_id] ASC); GO diff --git a/database/sql/060-get_embedding.sql b/database/sql/060-get_embedding.sql index a4ccab6..636d7ca 100644 --- a/database/sql/060-get_embedding.sql +++ b/database/sql/060-get_embedding.sql @@ -7,7 +7,7 @@ begin try declare @payload nvarchar(max) = json_object('input': @inputText); declare @response nvarchar(max) exec @retval = sp_invoke_external_rest_endpoint - @url = '$OPENAI_URL$/openai/deployments/$OPENAI_MODEL$/embeddings?api-version=2023-03-15-preview', + @url = '$OPENAI_URL$openai/deployments/$OPENAI_MODEL$/embeddings?api-version=2023-05-15', @method = 'POST', @credential = [$OPENAI_URL$], @payload = @payload, diff --git a/database/sql/100-sample-data.sql b/database/sql/100-sample-data.sql index b2ebe4e..b88625e 100644 --- a/database/sql/100-sample-data.sql +++ b/database/sql/100-sample-data.sql @@ -1,18 +1,42 @@ -insert into web.speakers - (id, full_name, require_embeddings_update) -values - (5000, 'John Doe', 0) -go +/* Script for populating the database with sample speaker and session data. */ -declare @t as nvarchar(max), @e as vector(1536) -select @t = full_name from web.speakers where id = 5000 -exec web.get_embedding @t, @e output -update web.speakers set embeddings = @e where id = 5000 -go +/* SPEAKERS */ +-- Insert sample speaker data into the web.speakers table. +-- The require_embeddings_update column is set to 1 to indicate that embeddings need to be generated for these speakers. +INSERT INTO web.speakers (id, full_name, require_embeddings_update) +VALUES + (5000, 'John Doe', 1), + (5001, 'Jane Smith', 1); +GO -insert into web.sessions - (id, title, abstract, external_id, start_time, end_time, require_embeddings_update) -values +-- Generate embeddings for the speakers' full name. +DECLARE @id int, @full_name NVARCHAR(max), @embeddings vector(1536); + +DECLARE speaker_cursor CURSOR FOR +SELECT id, full_name FROM web.speakers WHERE require_embeddings_update = 1; + +OPEN speaker_cursor; +FETCH NEXT FROM speaker_cursor INTO @id, @full_name; +WHILE @@FETCH_STATUS = 0 +BEGIN + EXEC web.get_embedding @full_name, @embeddings OUTPUT; + UPDATE web.speakers + SET + embeddings = @embeddings, + require_embeddings_update = 0 + WHERE id = @id; + + FETCH NEXT FROM speaker_cursor INTO @id, @full_name; +END; +CLOSE speaker_cursor; +DEALLOCATE speaker_cursor; +GO + +/* SESSIONS */ +-- Insert sample session data into the web.sessions table. +-- The require_embeddings_update column is set to 1 to indicate that embeddings need to be generated for these sessions. +INSERT INTO web.sessions (id, title, abstract, external_id, start_time, end_time, require_embeddings_update) +VALUES ( 1000, 'Building a session recommender using OpenAI and Azure SQL', @@ -20,25 +44,8 @@ values 'S1', '2024-06-01 10:00:00', '2024-06-01 11:00:00', - 0 - ) -go - -declare @t as nvarchar(max), @e as vector(1536) -select @t = title + ':' + abstract from web.sessions where id = 1000 -exec web.get_embedding @t, @e output -update web.sessions set embeddings = @e where id = 1000 -go - -insert into web.sessions_speakers - (session_id, speaker_id) -values - (1000, 5000) -go - -insert into web.sessions - (id, title, abstract, external_id, start_time, end_time, require_embeddings_update) -values + 1 + ), ( 1001, 'Unlock the Art of Pizza Making with John Doe!', @@ -46,26 +53,8 @@ values 'S2', '2024-06-01 11:00:00', '2024-06-01 12:00:00', - 0 - ) -go - -declare @t as nvarchar(max), @e as vector(1536) -select @t = title + ':' + abstract from web.sessions where id = 1001 -exec web.get_embedding @t, @e output -update web.sessions set embeddings = @e where id = 1001 -go - -insert into web.sessions_speakers - (session_id, speaker_id) -values - (1001, 5000) -go - - -insert into web.sessions - (id, title, abstract, external_id, start_time, end_time, require_embeddings_update) -values + 1 + ), ( 1002, 'RAG on Azure SQL', @@ -73,18 +62,53 @@ values 'R1', '2024-09-05 16:00:00', '2024-09-05 17:00:00', - 0 - ) -go + 1 + ), + ( + 1003, + 'Bring your own data with OpenAI', + 'In this session you''ll learn how to use your own private data with OpenAI. We''ll cover the basics of how to retrieve data from an Azure SQL database, how to use it with OpenAI using the RAG pattern, and some best practices improving performance when performing retrieval-augmented generation (RAG).', + 'R2', + '2024-09-05 14:00:00', + '2024-09-05 15:00:00', + 1 + ); +GO + +-- Generate embeddings for the sessions' title and abstract. +-- The embeddings are stored in the embeddings column, and the require_embeddings_update column is set to 0 to indicate that embeddings have been generated. +DECLARE @id int, @title NVARCHAR(max), @abstract NVARCHAR(max), @text_to_embed NVARCHAR(max), @embeddings vector(1536); + +DECLARE session_cursor CURSOR FOR +SELECT id, title, abstract FROM web.sessions WHERE require_embeddings_update = 1; + +OPEN session_cursor; +FETCH NEXT FROM session_cursor INTO @id, @title, @abstract; +WHILE @@FETCH_STATUS = 0 +BEGIN + SET @text_to_embed = @title + ':' + @abstract; -- Concatenate title and abstract for embedding generation + EXEC web.get_embedding @text_to_embed , @embeddings OUTPUT; + UPDATE web.sessions + SET + embeddings = @embeddings, + require_embeddings_update = 0 + WHERE id = @id; -declare @t as nvarchar(max), @e as vector(1536) -select @t = title + ':' + abstract from web.sessions where id = 1002 -exec web.get_embedding @t, @e output -update web.sessions set embeddings = @e where id = 1002 -go + FETCH NEXT FROM session_cursor INTO @id, @title, @abstract; +END; +CLOSE session_cursor; +DEALLOCATE session_cursor; +GO -insert into web.sessions_speakers - (session_id, speaker_id) -values - (1002, 5000) -go \ No newline at end of file +/* SPEAKERS SESSIONS */ +-- Insert sample data into the web.sessions_speakers table to establish a many-to-many relationship between sessions and speakers. +TRUNCATE TABLE web.sessions_speakers; -- Clear existing data in the sessions_speakers table +GO +INSERT INTO web.sessions_speakers (session_id, speaker_id) +VALUES + (1000, 5000), -- John Doe and Jane Smith are speakers for session 1000 + (1001, 5000), -- John Doe is a speaker for session 1001 + (1002, 5000), -- John Doe is a speaker for session 1002 + (1002, 5001), -- Jane Smith is a speaker for session 1002 + (1003, 5001); -- Jane Smith is a speaker for session 1003 +GO diff --git a/infra/abbreviations.json b/infra/abbreviations.json new file mode 100644 index 0000000..5b6771b --- /dev/null +++ b/infra/abbreviations.json @@ -0,0 +1,12 @@ +{ + "insightsComponents": "appi-", + "managedIdentityUserAssignedIdentities": "id-", + "openAiAccounts": "openai-", + "operationalInsightsWorkspaces": "log-", + "resourcesResourceGroups": "rg-", + "sqlServers": "sql-", + "sqlServersDatabases": "sqldb-", + "storageStorageAccounts": "st", + "webServerFarms": "plan-", + "webSitesFunctions": "func-" +} diff --git a/infra/azd-hooks/preprovision.ps1 b/infra/azd-hooks/preprovision.ps1 new file mode 100644 index 0000000..2082969 --- /dev/null +++ b/infra/azd-hooks/preprovision.ps1 @@ -0,0 +1,13 @@ +Write-Host "Running preprovision.ps1..." + +# Get the user principal name of the signed-in user and write it to the .env file for the azd environment +Write-Host "Fetching User Principal Name and setting PRINCIPAL_NAME environment variable..." +$env:PRINCIPAL_NAME = $(az ad signed-in-user show --query 'userPrincipalName' -o tsv) +azd env set "PRINCIPAL_NAME" "$env:PRINCIPAL_NAME" +Write-Host "PRINCIPAL_NAME: $env:PRINCIPAL_NAME" + +# Get the client IP address and write it to the .env file for the azd environment +Write-Host "Fetching Client IP address and setting CLIENT_IP_ADDRESS environment variable..." +$env:CLIENT_IP_ADDRESS = Invoke-RestMethod -Uri "https://api.ipify.org" +azd env set "CLIENT_IP_ADDRESS" "$env:CLIENT_IP_ADDRESS" +Write-Host "CLIENT_IP_ADDRESS: $env:CLIENT_IP_ADDRESS" \ No newline at end of file diff --git a/infra/azd-hooks/preprovision.sh b/infra/azd-hooks/preprovision.sh new file mode 100644 index 0000000..7c288a5 --- /dev/null +++ b/infra/azd-hooks/preprovision.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +echo "Running preprovision.sh..." + +# Get the user principal name of the signed-in user and write it to the .env file for the azd environment +echo "Fetching User Principal Name and setting PRINCIPAL_NAME environment variable..." +PRINCIPAL_NAME=$(az ad signed-in-user show --query 'userPrincipalName' -o tsv) +azd env set "PRINCIPAL_NAME" "$PRINCIPAL_NAME" +echo "PRINCIPAL_NAME: $PRINCIPAL_NAME" + +# Get the client IP address and write it to the .env file for the azd environment +echo "Fetching Client IP address and setting CLIENT_IP_ADDRESS environment variable..." +CLIENT_IP_ADDRESS=$(curl -s https://api.ipify.org) +azd env set "CLIENT_IP_ADDRESS" "$CLIENT_IP_ADDRESS" +echo "CLIENT_IP_ADDRESS: $CLIENT_IP_ADDRESS" diff --git a/infra/function-app.bicep b/infra/function-app.bicep new file mode 100644 index 0000000..010ac81 --- /dev/null +++ b/infra/function-app.bicep @@ -0,0 +1,121 @@ +// Creates an Azure Function App with a Storage Account, Log Analytics workspace, and Application Insights. + +@description('The Azure region to deploy the resources.') +param location string = resourceGroup().location + +@description('The name of the Function App Service Plan.') +param functionAppServicePlanName string + +@description('The name of the Azure Function App.') +param functionAppName string + +@description('The name of the Azure Log Analytics workspace.') +param logAnalyticsName string + +@description('The name of the Azure Application Insights resource.') +param appInsightsName string + +@description('The name of the Azure Storage account.') +param storageAccountName string + +@description('Tags to assign to the Azure OpenAI resource.') +param tags object = {} + +@description('Creates an Azure Storage account.') +resource storageAccount 'Microsoft.Storage/storageAccounts@2023-01-01' = { + name: storageAccountName + location: location + sku: { + name: 'Standard_LRS' + } + kind: 'StorageV2' + properties: { + accessTier: 'Hot' + } + tags: tags +} + +@description('Creates an Azure Log Analytics workspace.') +resource logAnalyticsWorkspace 'Microsoft.OperationalInsights/workspaces@2023-09-01' = { + name: logAnalyticsName + location: location + properties: { + sku: { + name: 'PerGB2018' + } + retentionInDays: 90 + workspaceCapping: { + dailyQuotaGb: 1 + } + } + tags: tags +} + +@description('Creates an Azure Application Insights resource.') +resource appInsights 'Microsoft.Insights/components@2020-02-02-preview' = { + name: appInsightsName + location: location + kind: 'web' + properties: { + Application_Type: 'web' + WorkspaceResourceId: logAnalyticsWorkspace.id + } + tags: tags +} + +@description('Creates an Azure Function App Service Plan.') +resource functionAppServicePlan 'Microsoft.Web/serverFarms@2022-09-01' = { + name: functionAppServicePlanName + location: location + sku: { + name: 'Y1' + tier: 'Dynamic' + } + properties: {} + tags: tags +} + +@description('Creates an Azure Function App.') +resource functionApp 'Microsoft.Web/sites@2022-03-01' = { + name: functionAppName + location: location + identity: { + type: 'SystemAssigned' + } + kind: 'functionapp' + properties: { + serverFarmId: functionAppServicePlan.id + siteConfig: { + appSettings: [ + { + name: 'AzureWebJobsStorage' + value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccountName};EndpointSuffix=${environment().suffixes.storage};AccountKey=${storageAccount.listKeys().keys[0].value}' + } + { + name: 'WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED' + value: '1' + } + { + name: 'FUNCTIONS_EXTENSION_VERSION' + value: '~4' + } + { + name: 'APPLICATIONINSIGHTS_CONNECTION_STRING' + value: appInsights.properties.ConnectionString + } + { + name: 'FUNCTIONS_WORKER_RUNTIME' + value: 'dotnet-isolated' + } + ] + ftpsState: 'FtpsOnly' + minTlsVersion: '1.2' + netFrameworkVersion: 'v8.0' + } + httpsOnly: true + virtualNetworkSubnetId: null + publicNetworkAccess: 'Enabled' + clientAffinityEnabled: false + } + tags: tags +} diff --git a/infra/main.bicep b/infra/main.bicep new file mode 100644 index 0000000..30a352e --- /dev/null +++ b/infra/main.bicep @@ -0,0 +1,119 @@ +@minLength(1) +@maxLength(64) +@description('Name of the environment that can be used as part of naming resource convention') +param environmentName string + +@description('Primary Azure region for all resources') +param location string + +@description('Name of the resource group') +param resourceGroupName string + +@description('The name of the SQL Server admin user') +param sqlAdminUser string = 'sqlAdmin' + +@description('SQL Admin Password') +@secure() +param sqlAdminPassword string + +@description('The name of the SQL database') +param databaseName string = 'sessiondb' + +@description('The Azure OpenAI completions model to deploy') +param openAiCompletionsModel string = 'gpt-4o' +@description('The Azure OpenAI embeddings model to deploy') +param openAiEmbeddingsModel string = 'text-embedding-ada-002' + +var abbrs = loadJsonContent('./abbreviations.json') +var resourceToken = toLower(uniqueString(subscription().id, environmentName, location, resourceGroupName)) + +@description('The Client IP address for the SQL Server firewall.') +param clientIpAddress string = '0.0.0.0' // This is overwritten by the preprovision script. + +@description('The Azure User Principal Name of the user deploying the template.') +param principalName string // This is set by the preprovision script. + +@description('The Azure Principal ID of the user deploying the template.') +var principalId = deployer().objectId + +@description('Determines whether to deploy a Function App and associated resources.') +param deployFunctionApp bool + +// Tags that should be applied to all resources. +var tags = { + 'azd-env-name': environmentName +} + +targetScope = 'subscription' +resource rg 'Microsoft.Resources/resourceGroups@2022-09-01' = { + name: resourceGroupName + location: location + tags: tags +} + +@description('Creates a SQL Server and a SQL Database') +module sqlDatabase 'sql-db.bicep' = { + name: 'sqlDatabase' + params: { + serverName: '${abbrs.sqlServers}${resourceToken}' + location: location + principalId: principalId + administratorLogin: sqlAdminUser + administratorLoginPassword: sqlAdminPassword + userPrincipalName: principalName + databaseName: databaseName + clientIpAddress: clientIpAddress + } + scope: rg +} + +module openAi 'openai.bicep' = { + name: 'openai' + params: { + deployments: [ + { + name: openAiCompletionsModel + sku: { + name: 'Standard' + capacity: 10 + } + model: { + name: openAiCompletionsModel + version: '2024-05-13' + } + } + { + name: openAiEmbeddingsModel + sku: { + name: 'Standard' + capacity: 10 + } + model: { + name: openAiEmbeddingsModel + version: '2' + } + } + ] + principalId: principalId + location: location + name: '${abbrs.openAiAccounts}${resourceToken}' + sku: 'S0' + tags: tags + } + scope: rg +} + +@description('Creates an Azure Function App and associated resources.') +module functionApp 'function-app.bicep' = if (deployFunctionApp) { + name: 'functionApp' + params: { + functionAppServicePlanName: '${abbrs.webServerFarms}${resourceToken}' + functionAppName: '${abbrs.webSitesFunctions}${resourceToken}' + logAnalyticsName: '${abbrs.operationalInsightsWorkspaces}${resourceToken}' + appInsightsName: '${abbrs.insightsComponents}${resourceToken}' + storageAccountName: '${abbrs.storageStorageAccounts}${resourceToken}' + location: location + tags: tags + } + scope: rg +} diff --git a/infra/main.parameters.json b/infra/main.parameters.json new file mode 100644 index 0000000..8030e12 --- /dev/null +++ b/infra/main.parameters.json @@ -0,0 +1,18 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "environmentName": { + "value": "${AZURE_ENV_NAME}" + }, + "location": { + "value": "${AZURE_LOCATION}" + }, + "principalName": { + "value": "${PRINCIPAL_NAME=NotYetSet}" + }, + "clientIpAddress": { + "value": "${CLIENT_IP_ADDRESS}" + } + } + } \ No newline at end of file diff --git a/infra/openai.bicep b/infra/openai.bicep new file mode 100644 index 0000000..c58be25 --- /dev/null +++ b/infra/openai.bicep @@ -0,0 +1,76 @@ +// Description: Bicep template to deploy Azure OpenAI Service with specified deployments and role assignments. +// This template creates an Azure OpenAI account, configures deployments, and assigns the Cognitive Services OpenAI Contributor role to a specified principal. +// It also outputs the endpoint of the OpenAI account. + +@description('The name of the Azure OpenAI resource.') +param name string + +@description('The Azure region into Azure OpenAI will be deployed.') +param location string = resourceGroup().location + +@description('The SKU of the Azure OpenAI resource.') +param sku string = 'S0' + +@description('Tags to assign to the Azure OpenAI resource.') +param tags object = {} + +@description('Array of model deployments to be created.') +param deployments array + +@description('The Azure Principal ID of the user deploying the template.') +param principalId string + +@description('The Azure Principal Type of the user deploying the template.') +param principalType string = 'User' + +@description('Creates an Azure OpenAI resource.') +resource openAi 'Microsoft.CognitiveServices/accounts@2023-05-01' = { + name: name + location: location + identity: { + type: 'SystemAssigned' + } + kind: 'OpenAI' + sku: { + name: sku + } + + properties: { + customSubDomainName: name + publicNetworkAccess: 'Enabled' + } + tags: tags +} + +@description('Creates model deployments for the Azure OpenAI resource.') +@batchSize(1) +resource openAiDeployments 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = [ + for deployment in deployments: { + parent: openAi + name: deployment.name + sku: { + capacity: deployment.sku.capacity + name: deployment.sku.name + } + properties: { + model: { + format: 'OpenAI' + name: deployment.model.name + version: deployment.model.version + } + } + } +] + +@description('Assigns the Cognitive Services OpenAI Contributor role to the user deploying the template.') +resource openAIContributorRoleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = { + scope: openAi + name: guid(subscription().id, resourceGroup().id, principalId, 'Cognitive Services OpenAI Contributor') + properties: { + roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'a001fd3d-188f-4b5d-821b-7da978bf7442') // Cognitive Services OpenAI Contributor role ID + principalId: principalId + principalType: principalType + } +} + +output endpoint string = openAi.properties.endpoint diff --git a/infra/sql-db.bicep b/infra/sql-db.bicep new file mode 100644 index 0000000..9870d67 --- /dev/null +++ b/infra/sql-db.bicep @@ -0,0 +1,82 @@ +// Bicep template to create an Azure SQL Database with a firewall rule. +// This template creates an Azure SQL Server and a SQL Database, and configures a firewall rule to allow access from a specified IP address. +// It also assigns the Azure Principal ID of the user deploying the template as an administrator of the SQL Server. + +@description('The Azure region into which the SQL Server will be deployed.') +param location string = resourceGroup().location + +@description('The name of the SQL Server.') +param serverName string + +@description('The name of the SQL Server administrator.') +param administratorLogin string = 'sqlAdmin' + +@description('The password for the SQL Server administrator.') +@secure() +param administratorLoginPassword string + +@description('Name of the SQL database') +param databaseName string = 'sessiondb' + +@description('The client IP address for the SQL Server firewall.') +param clientIpAddress string + +@description('The principal ID of the user deploying the template.') +param principalId string = deployer().objectId + +@description('The Azure Principal Name of the user deploying the template.') +param userPrincipalName string // Set to the name of the user deploying the template + +@description('Tags to assign to the Azure OpenAI resource.') +param tags object = {} + +@description('Creates an Azure SQL Server and a SQL Database with a firewall rule.') +resource server 'Microsoft.Sql/servers@2024-05-01-preview' = { + name: serverName + location: location + identity: { + type: 'SystemAssigned' + } + properties: { + administratorLogin: administratorLogin + administratorLoginPassword: administratorLoginPassword + administrators: { + administratorType: 'ActiveDirectory' + azureADOnlyAuthentication: true + login: userPrincipalName + principalType: 'User' + sid: principalId + tenantId: subscription().tenantId + } + publicNetworkAccess: 'Enabled' + } + tags: tags + + resource database 'databases@2024-05-01-preview' = { + name: databaseName + location: location + sku: { + name: 'Basic' + tier: 'Basic' + capacity: 5 + } + properties: {} + } + + resource firewallRule 'firewallRules@2024-05-01-preview' = { + name: 'AllowAllAzureServices' + properties: { + startIpAddress: '0.0.0.0' + endIpAddress: '0.0.0.0' + } + } +} + +resource firewallRule 'Microsoft.Sql/servers/firewallRules@2024-05-01-preview' = { + name: 'AllowClientIP' + parent: server + properties: { + startIpAddress: clientIpAddress + endIpAddress: clientIpAddress + } +} diff --git a/walkthrough.md b/walkthrough.md new file mode 100644 index 0000000..a850f4d --- /dev/null +++ b/walkthrough.md @@ -0,0 +1,498 @@ +# Code walkthrough + +This document highlights the key components of the application's codebase and provides a brief overview of how they work together to create the solution. + +- [Code walkthrough](#code-walkthrough) + - [Solution](#solution) + - [Database](#database) + - [Enable Calling Azure OpenAI Directly from the Database](#enable-calling-azure-openai-directly-from-the-database) + - [Vector Storage](#vector-storage) + - [Generate Embeddings](#generate-embeddings) + - [Perform Vector Similarity Searches](#perform-vector-similarity-searches) + - [Chainlit App](#chainlit-app) + - [`utilities.py`](#utilitiespy) + - [`app.py`](#apppy) + - [`app-langgraph.py`](#app-langgraphpy) + - [Compare LangChain Approaches](#compare-langchain-approaches) + - [Azure Function](#azure-function) + - [`Program.cs`](#programcs) + - [`SessionProcessor.cs`](#sessionprocessorcs) + +## Solution + +The solution is comprised of three core projects: + +- Database project +- Chainlit + LangChain app +- Azure Function + +It relies on three main Azure components: + +- [Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/database/sql-database-paas-overview?view=azuresql): The Azure SQL database that stores application data. +- [Azure Open AI](https://learn.microsoft.com/azure/ai-services/openai/): Hosts the language models for generating embeddings and completions. +- [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-overview?pivots=programming-language-csharp): The serverless function to automate the process of generating the embeddings (this is optional for this sample) + +## Database + +The database project, contained in the `/database` folder, contains a [SQL database project](https://learn.microsoft.com/sql/tools/sql-database-projects/get-started?view=sql-server-ver16&pivots=sq1-visual-studio) representing the SQL objects that comprise the schema of the solution database. The project defines the sample database's tables, stored procedures, and security-related objects. + +In this solution, Azure SQL functions as a Vector store. The database structures rely on the native vector search capabilities of Azure SQL, including the [Vector data type](https://learn.microsoft.com/sql/t-sql/data-types/vector-data-type?view=azuresqldb-current&tabs=csharp-sample) and the built-in [VECTOR_DISTANCE function](https://learn.microsoft.com/sql/t-sql/functions/vector-distance-transact-sql?view=azuresqldb-current). + +### Enable Calling Azure OpenAI Directly from the Database + +A database scoped credential is used to ensure calls from the database to the Azure OpenAI embedding endpoint can be authenticated correctly. Two database objects are required to enable this approach. The first is a master key to protect the private key associated with database scoped credentials. + +```sql +IF NOT EXISTS(SELECT * FROM sys.symmetric_keys WHERE [name] = '##MS_DatabaseMasterKey##') +BEGIN + CREATE MASTER KEY ENCRYPTION BY PASSWORD = N'V3RYStr0NGP@ssw0rd!'; +END +GO +``` + +The second object is a database scoped credential. This object relies on your Azure OpenAI URL and key to create the credentials necessary to connect to the service's embedding endpoint from T-SQL code in the database. + +```sql +IF EXISTS(SELECT * FROM sys.[database_scoped_credentials] WHERE [name] = '$OPENAI_URL$') +BEGIN + DROP DATABASE SCOPED CREDENTIAL [$OPENAI_URL$]; +END +GO + +CREATE DATABASE SCOPED CREDENTIAL [$OPENAI_URL$] +WITH IDENTITY = 'HTTPEndpointHeaders', SECRET = '{"api-key":"$OPENAI_KEY$"}'; +GO +``` + +### Vector Storage + +The `sessions` and `speakers` tables store vector embedding representations of data using Azure SQL Database's native [Vector data type](https://learn.microsoft.com/sql/t-sql/data-types/vector-data-type?view=azuresqldb-current&tabs=csharp-sample). Each table defines a column with a `vector` data type for storing embeddings. The vector data type requires specification of the number of dimensions in the vector. When using Azure OpenAI's `text-embedding-ada-002` model, that number is 1536. + +```sql +[embeddings] VECTOR(1536) +``` + +### Generate Embeddings + +The `get_embedding` stored procedure handles calling Azure OpenAI to generate embeddings directly from the database. It relies on the `sp_invoke_external_rest_endpoint` procedure to call Azure OpenAI's REST endpoint. It passes the input text as a `json_object` to the endpoint. + +```sql +BEGIN TRY + DECLARE @retval int; + DECLARE @payload nvarchar(max) = json_object('input': @inputText); + DECLARE @response nvarchar(max) + EXEC @retval = sp_invoke_external_rest_endpoint + @url = '$OPENAI_URL$openai/deployments/$OPENAI_MODEL$/embeddings?api-version=2023-05-15', + @method = 'POST', + @credential = [$OPENAI_URL$], + @payload = @payload, + @response = @response OUTPUT; +END TRY +``` + +The embeddings generated by Azure OpenAI are then returned as a `vector` data type: + +```sql +DECLARE @re nvarchar(max) = json_query(@response, '$.result.data[0].embedding') +SET @embedding = cast(@re AS vector(1536)); +``` + +### Perform Vector Similarity Searches + +RAG functionality in the Chainlit + LangChain app is handled by passing user queries to the database's `find_sessions` stored procedure. The query text must first be vectorized to be correctly compared to stored embeddings. This process is accomplished by calling the `get_embedding` stored procedure and passing in the user query text. + +```sql +EXEC @retval = web.get_embedding @text, @qv OUTPUT; +``` + +Once the query embeddings have been generated, they are compared to the vectors stored in the `sessions` and `speakers` tables using Azure SQL's built-in [VECTOR_DISTANCE function](https://learn.microsoft.com/sql/t-sql/functions/vector-distance-transact-sql?view=azuresqldb-current) to identify sessions and speakers that are semantically similar to the query. + +```sql +vector_distance('cosine', se.[embeddings], @qv) as distance +``` + +SQL Common Table Expressions (CTEs) are used to combine the search results from the `sessions` and `speakers` tables, and the final result selects and orders the results, so the most relevant based on `cosine_similarity` are returned. + +```sql +SELECT TOP(@top) + a.id, + a.title, + a.abstract, + a.external_id, + a.start_time, + a.end_time, + a.recording_url, + ISNULL((SELECT TOP (1) speakers FROM cteSpeakers WHERE session_id = a.id), '[]') AS speakers, + 1-distance AS cosine_similarity +FROM cteSimilar2 r +INNER JOIN web.sessions a on r.session_id = a.id +WHERE (1-distance) > @min_similarity + AND rn = 1 +ORDER BY distance ASC, a.title ASC; +``` + +## Chainlit App + +Code for the Chainlit application is contained in three files, `app.py`, `app-langgraph.py`, and `utilities.py`. + +### `utilities.py` + +We'll focus first on `utilities.py`, which is called from the other two. The code in `utilities.py` handles creating a connection to the Azure SQL database, and it defines the `get_similar_sessions` function used by `app.py` and `app-langgraph.py` to perform RAG. This function acts as a wrapper around executing the `web.find_sessions` stored procedure in the database, which performs a vector similarity search against the `sessions` and `speakers` tables. It returns a list of semantically similar sessions formatted for consumption by an LLM completion model. + +```python +cursor = conn.cursor() +results = cursor.execute("SET NOCOUNT ON; EXEC web.find_sessions @text=?", (topic)).fetchall() + +payload = "" +for row in results: + description = str(row[2]).replace("\n", " ") + speakers = ", ".join(json.loads(row[7])) + payload += f'{row[0]}|{row[1]}|{description}|{speakers}|{row[4]}|{row[5]}' + payload += "\n" + +return payload +``` + +### `app.py` + +The code in `app.py` showcases a straightforward approach to performing RAG operations against an Azure SQL database. The `app.py` approach utilizes LangChain's Runnable-based execution and follows a **predefined, linear execution model**, moving sequentially from retrieval to response generation. This linear approach is predictable and easy to debug but lacks the conditional execution provided by using LangGraph, as you will see in the `app-langgraph.py` example. + +**Azure OpenAI API Initialization**: + +The `AzureChatOpenAI` object creates a connection to the Azure OpenAI API. + +```python +openai = AzureChatOpenAI( + openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"], + azure_deployment=os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"], + streaming=True +) +``` + +**Prompt Construction**: + +LangChain's `ChatPromptTemplate` is used to provide a **structured prompt** with predefined placeholders (`{sessions}`, `{question}`), establishing a predefined structure for LLM prompts. + +```python +prompt = ChatPromptTemplate.from_messages( + [ + ("ai", "You are a system assistant who helps users find the right session..."), + ("human", "The sessions available at the conference are the following: {sessions}"), + ("human", "{question}"), + ] +) +``` + +AI instructions are **static** in this approach, meaning the assistant responds **based only on this template**. This structure ensures that the AI assistant stays **focused** on retrieving conference session information, but it **limits dynamic adaptability** since the conversation flow is fixed. + +**Data Retrieval**: + +A `RunnableLambda` wraps `get_similar_sessions`, allowing **dynamic data retrieval**. This runs **before** passing the query to the AI model, ensuring only relevant data is provided to the LLM, improving AI accuracy. + +```python +retriever = RunnableLambda(get_similar_sessions, name="GetSimilarSessions").bind() +``` + +**Pipeline Execution**: + +A **data flow pipeline** is constructed using LangChain's `Runnable` framework. `app.py` relies on a Linear Execution (`Runnable | Prompt | Model | Parser`) model in which data moves step-by-step, always executing sequentially. + +```python +runnable = {"sessions": retriever, "question": RunnablePassthrough()} | prompt | openai | StrOutputParser() +``` + +- `retriever`: Fetches similar sessions from the database. +- `RunnablePassthrough()`: Passes user query without modification. +- `prompt`: Formats the final AI request. +- `openai`: Runs the request through Azure OpenAI. +- `StrOutputParser()`: Parses AI output into a readable format. + +**Session Storage**: + +Session memory is managed using the `cl.user_session`, which stores the runnable object in Chainlit's session memory. This method retains memory across multiple user interactions in the same session. However, `user_session` storage is temporary, so the conversation context does not persist beyond the session. + +```python +cl.user_session.set("runnable", runnable) +``` + +**Sending the Final AI Response**: + +The fully generated response is returned to the user, ending the interaction cycle and ensuring Chainlit updates the chat window. + +```python +@cl.on_message +async def on_message(message: cl.Message): + runnable = cl.user_session.get("runnable") # type: Runnable + + response_message = cl.Message(content="") + + for chunk in await cl.make_async(runnable.stream)( + input=message.content, + config=RunnableConfig(callbacks=[cl.LangchainCallbackHandler()]), + ): + await response_message.stream_token(chunk) + + await response_message.send() +``` + +### `app-langgraph.py` + +The code in `app-langgraph.py` showcases an approach to performing RAG operations against an Azure SQL database using LangChain, LangGraph, and Chainlit. Unlike `app.py`, this implementation leverages a **graph-based execution model** to provide adaptive control over tool execution. This approach implements a graph-driven execution model and introduces dynamic state transitions, allowing conditional execution depending on whether tool calls are required. The **LangGraph approach** is more flexible for complex workflows, where branching logic or memory checkpoints are essential. + +**Azure OpenAI API Initialization**: + +Similar to `app.py`, the `AzureChatOpenAI` object creates a connection to the Azure OpenAI API. + +```python +model = AzureChatOpenAI( + openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"], + azure_deployment=os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"] +) +``` + +**Prompt Construction**: + +Using LangGraph, the prompt is constructed using message-based state transitions (`HumanMessage`, `SystemMessage`). This message-based approach is useful where input dynamically evolves. It isn't bound by a rigid template, allowing the assistant to create contextually. + +```python +def call_model(state: MessagesState): + messages = state["messages"] + response = model.invoke(messages) + return {"messages": [response]} +``` + +**Data Retrieval**: + +Data retrieval is handled via tool binding and a `ToolNode`. External tools like `get_similar_sessions` are bound directly to the model, enabling dynamic function calls. + +```python +tools = [get_similar_sessions] +model = model.bind_tools(tools) +``` + +Tools are added to a node, allowing data retrieval to be encapsulated inside a `ToolNode`. This node is only executed when the AI determines that retrieval is necessary. + +```python +tool_node = ToolNode(tools=tools) +``` + +**Pipeline Execution**: + +The workflow when using LangGraph follows a graph-based execution pattern. + +`app-langgraph.py` – Graph Execution (`StateGraph`) + +```python +workflow = StateGraph(MessagesState) + +workflow.add_node("agent", call_model) +workflow.add_node("tools", tool_node) + +workflow.add_edge(START, "agent") + +workflow.add_conditional_edges("agent", should_continue) +``` + +State is managed using a state graph, which enables decision-based execution. The conditional edge calling `should_continue()` determines whether session retrieval should happen. This allows branching, making the assistant adaptive and ensuring tools only execute when necessary. + +**Session Storage**: + +The workflow can call the model with stateful input using the `call_model` function. + +```python +def call_model(state: MessagesState): + messages = state["messages"] + response = model.invoke(messages) + return {"messages": [response]} +``` + +**Sending the Final AI Response**: + +The fully generated response is returned to the user, ending the interaction cycle and ensuring Chainlit updates the chat window. + +```python +@cl.on_message +async def on_message(msg: cl.Message): + config = {"configurable": {"thread_id": cl.context.session.id}} + final_answer = cl.Message(content="") + + for msg, metadata in await cl.make_async(graph.stream)( + {"messages": [HumanMessage(content=msg.content)]}, + stream_mode="messages", + config=RunnableConfig(callbacks=[cl.LangchainCallbackHandler()], **config) + ): + if metadata["langgraph_node"] == "agent": + await final_answer.stream_token(msg.content) + + await final_answer.send() +``` + +### Compare LangChain Approaches + +Both `app.py` and `app-langgraph.py` define a chat UI structure with Chainlit and utilize LangChain to implement retrieval-augmented generation (RAG) for querying the Azure SQL database. However, these scripts differ in their approach to structuring the retrieval and response workflow with LangChain. + +**Similarities**: + +1. **Shared Libraries & Frameworks** + - Both scripts import **LangChain**, **Chainlit**, and **AzureChatOpenAI** to handle language model interactions. + - They rely on `get_similar_sessions()` from the `utilities` module to retrieve relevant sessions. + +2. **Azure OpenAI Integration** + - Each script initializes `AzureChatOpenAI` using **environment variables** for API configuration. + - The model is set to stream responses asynchronously. + +3. **Chainlit Event Handling** + - Both use the `@cl.on_message` decorator to process user input dynamically. + - They execute LLM calls and return results in a streaming format. + +**Key Differences**: + +| Aspect | `app.py` (LangChain) | `app-langgraph.py` (LangGraph) | +|---|---|---| +| **Workflow Structure** | Uses **Runnable Components** (`RunnableLambda`, `RunnablePassthrough`) | Implements a **stateful execution graph** with LangGraph (`StateGraph`) | +| **Prompt Handling** | Uses `ChatPromptTemplate` to format structured prompts | Relies on **message-based state transitions** (`HumanMessage`, `SystemMessage`) | +| **Data Retrieval** | Direct **retriever invocation** via `RunnableLambda` | Calls a **ToolNode** that executes session retrieval | +| **Tool Execution** | Manually executes session retrieval (`RunnableLambda(get_similar_sessions)`) | Conditionally invokes tool execution within a **ToolNode** | +| **Execution Strategy** | Sequential pipeline (`retriever → prompt → model → parser`) | Graph-based execution (`agent → tools → conditional routing`) | +| **Response Flow** | Defines a simple chain using `RunnableConfig` | Implements a **conditional node** (`should_continue()`) to control execution | +| **State Management** | Maintains temporary state via `cl.user_session` | Uses `MemorySaver` for **checkpointing** messages and preserving state | + +In summary: + +LangChain's Runnable-based execution (as in `app.py`) is: + +- Predictable and **easy to debug** +- Well-suited for **straightforward tasks** +- **Linear**, meaning steps always execute in a set order + +On the other hand, LangGraph takes inspiration from **agent-based architectures** and **state machines**, meaning: + +- Workflows **can adapt** dynamically +- Execution **is conditional**, rather than strictly sequential +- **Memory tracking** improves long-term interaction + +## Azure Function + +This optional component of the solution provides a mechanism for automating the generation of embeddings to ensure efficient filtering and batch processing. The Azure Function monitors SQL changes, generates vector embeddings for new and updated rows, and writes the embeddings back into SQL. The code for the function is split into two separate files, `Program.cs` and `SessionProcessor.cs`. + +### `Program.cs` + +`Program.cs` is used to configure the Azure Function, including its connections to Azure OpenAI and Azure SQL. + +```csharp +.ConfigureServices(services => +{ + Uri openaiEndPoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") is string value && + Uri.TryCreate(value, UriKind.Absolute, out Uri? uri) && + uri is not null + ? uri + : throw new ArgumentException( + $"Unable to parse endpoint URI"); + + string? apiKey; + var keyVaultEndpoint = Environment.GetEnvironmentVariable("AZURE_KEY_VAULT_ENDPOINT"); + if (!string.IsNullOrEmpty(keyVaultEndpoint)) + { + var openAIKeyName = Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY"); + var keyVaultClient = new SecretClient(vaultUri: new Uri(keyVaultEndpoint), credential: new DefaultAzureCredential()); + apiKey = keyVaultClient.GetSecret(openAIKeyName).Value.Value; + } + else + { + apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY"); + } + + OpenAIClient openAIClient = apiKey != null ? + new(openaiEndPoint, new AzureKeyCredential(apiKey)) : + new(openaiEndPoint, new DefaultAzureCredential()); + + services.AddSingleton(openAIClient); + + services.AddTransient((_) => new SqlConnection(Environment.GetEnvironmentVariable("AZURE_SQL_CONNECTION_STRING"))); +}) +.ConfigureFunctionsWebApplication() +.Build(); +``` + +### `SessionProcessor.cs` + +The `SessionProcessor.cs` file defines Azure Functions for processing changes in SQL tables and updates embeddings using Azure OpenAI. Let's break it down using code snippets to highlight key areas. + +**1. Defining Data Models**: + +The base `Item` class contains common properties such as `Id` and `RequireEmbeddingsUpdate`, enabling inheritance for `Session`, `Speaker`, and `ChangedItem`. + +```csharp +public class Item +{ + public required int Id { get; set; } + + [JsonPropertyName("require_embeddings_update")] + public bool RequireEmbeddingsUpdate { get; set; } +} +``` + +Each specialized class defined in the file represents different entities (`Session`, `Speaker`) that will be processed in the function. + +**2. Handling SQL Table Changes**: + +The `SessionProcessor` class registers two Azure Functions (`SessionTrigger` and `SpeakerTrigger`) to listen for changes in SQL tables. + +```csharp +[Function(nameof(SessionTrigger))] +public async Task SessionTrigger( + [SqlTrigger("[web].[sessions]", "AZURE_SQL_CONNECTION_STRING")] + IReadOnlyList> changes +) +``` + +```csharp +[Function(nameof(SpeakerTrigger))] +public async Task SpeakerTrigger( + [SqlTrigger("[web].[speakers]", "AZURE_SQL_CONNECTION_STRING")] + IReadOnlyList> changes +) +``` + +Each trigger monitors the respective SQL table (`web.sessions` and `web.speakers`) and filters for changes requiring creating or updating embeddings. + +**3. Transforming Data for Processing**: + +Filtered changes are mapped into `ChangedItem` instances, carrying necessary metadata such as the operation type and relevant payload (e.g., `Title + Abstract` for a session). + +```csharp +var ci = from c in changes + where c.Operation != SqlChangeOperation.Delete + where c.Item.RequireEmbeddingsUpdate == true + select new ChangedItem() { + Id = c.Item.Id, + Operation = c.Operation, + Payload = c.Item.Title + ':' + c.Item.Abstract + }; +``` + +This filtering ensures that only meaningful updates are processed. The functions rely on the `require_embeddings_update` field in the `sessions` and `speakers` tables being set to true whenever updated embeddings are necessary. + +**4. Processing Data with OpenAI**: + +A helper method, `ProcessChanges`, manages updates by calling Azure OpenAI to generate embeddings for the changed items. + +```csharp +var response = await openAIClient.GetEmbeddingsAsync( + new EmbeddingsOptions(_openAIDeploymentName, [change.Payload]) +); +``` + +The embeddings are then serialized and written to the database using a stored procedure. + +```csharp +await conn.ExecuteAsync( + upsertStoredProcedure, + commandType: CommandType.StoredProcedure, + param: new + { + @id = change.Id, + @embeddings = JsonSerializer.Serialize(e) + } +); +```