rag_demo.ipynb

{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# Backblaze B2 Retrieval-Augmented Generation (RAG) Demo\n",
    "\n",
    "Retrieval-Augmented Generation (RAG) allows you to build on a foundation model, that is, an off-the-shelf large language model (LLM), adding custom context that the model can use in interacting with a user. You can use RAG to implement chatbots that use your own proprietary data to answer questions, without that data leaking to the internet. \n",
    "\n",
    "This notebook walks you through loading PDF files from [Backblaze B2 Cloud Object Storage](https://www.backblaze.com/cloud-storage) into a [LangChain](https://python.langchain.com/v0.2/docs/introduction/) RAG app, then building a chatbot that can answer questions relating to the content of those PDF files. You'll use an open-source language model that you run locally, rather than an online API, ensuring that your data stays confidential.\n",
    "\n",
    "The code is based on the LangChain tutorial, [Build a Local RAG Application](https://python.langchain.com/v0.2/docs/tutorials/local_rag/)."
   ],
   "id": "82bf0aabeeef45df"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Install Dependencies\n",
    "\n",
    "First, install the required Python packages. You will need to restart the Jupyter kernel before you can use newly-installed packages. You can uncomment the second line, or manually do so."
   ],
   "id": "75a51ec771ee51d"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:20:45.439238Z",
     "start_time": "2024-09-18T18:20:42.423513Z"
    }
   },
   "cell_type": "code",
   "source": [
    "%pip install --upgrade --quiet -r requirements.txt\n",
    "\n",
    "# Uncomment this line to restart the kernel so that it uses the new modules\n",
    "# get_ipython().kernel.do_shutdown(restart=True)"
   ],
   "id": "57a3b3958767b924",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "execution_count": 1
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Prerequisites\n",
    "\n",
    "You need a Backblaze B2 Account, Bucket and Application Key, and some PDF files. Follow these instructions, as necessary:\n",
    "\n",
    "* [Create a Backblaze B2 Account](https://www.backblaze.com/sign-up/cloud-storage).\n",
    "* [Create a Backblaze B2 Bucket](https://www.backblaze.com/docs/cloud-storage-create-and-manage-buckets).\n",
    "* [Create an Application Key](https://www.backblaze.com/docs/cloud-storage-create-and-manage-app-keys#create-an-app-key) with access to the bucket you wish to use.\n",
    "\n",
    "Be sure to copy the application key as soon as you create it, as you will not be able to retrieve it later!"
   ],
   "id": "b401105aa7ab8e27"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Upload PDF Files to Your Bucket\n",
    "\n",
    "You can use the Backblaze web UI, or any B2 or S3-compatible file management tool to [upload PDF files to your bucket](https://www.backblaze.com/docs/cloud-storage-upload-and-manage-files). It's useful to organize files by prefix (analogous to a folder or directory in a traditional filesystem); this example assumes the PDFs have the prefix `pdfs/` within the bucket.\n",
    "\n",
    "If you don't have any suitable PDF files to hand, you can [download a PDF of the Backblaze B2 documentation](https://metadaddy-langchain-demo.s3.us-west-004.backblazeb2.com/pdfs/documentation.pdf) and upload it to your own bucket. "
   ],
   "id": "9a708475a5b58ccf"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Configuration\n",
    "\n",
    "Since Backblaze B2 has an S3-compatible API, this notebook uses LangChain's [`S3FileLoader`](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.s3_file.S3FileLoader.html) and the [`s3fs`](https://s3fs.readthedocs.io/en/latest/) module to interact with files in Backblaze B2, as well as the [AWS SDK for Python, also known as Boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html). Both `S3FileLoader` and `s3fs` use Boto3 under the covers, so you need simply configure the latter so that all of the tools can access your Backblaze B2 Bucket. The most straightforward way to do so in this context is via environment variables.\n",
    "\n",
    "Note: you should never, *ever* put credentials in your code, including Jupyter notebooks! This example uses `python-dotenv` to load configuration from a `.env` file into environment variables for use by `S3FileLoader`. This repo includes a template file, `.env.template`. Copy it to `.env`, then edit it as follows:\n",
    "\n",
    "```dotenv\n",
    "AWS_ACCESS_KEY_ID='<Your Backblaze application key ID>'\n",
    "AWS_SECRET_ACCESS_KEY='<Your Backblaze application key>'\n",
    "AWS_ENDPOINT_URL='<Your bucket endpoint, prefixed with https://, e.g., https://s3.us-west-004.backblazeb2.com >'\n",
    "```\n",
    "\n",
    "When you're done, `.env` should look like this:\n",
    "\n",
    "```dotenv\n",
    "AWS_ACCESS_KEY_ID='004qlekmvpwemrt000000009e'\n",
    "AWS_SECRET_ACCESS_KEY='K004JEKEUTGLKEJFKLRJHTKLVCNWURM'\n",
    "AWS_ENDPOINT_URL='https://s3.us-west-004.backblazeb2.com'\n",
    "```\n",
    "\n",
    "Now you can load the configuration into the environment:"
   ],
   "id": "6b3461adad990f00"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:20:45.448275Z",
     "start_time": "2024-09-18T18:20:45.440674Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from dotenv import load_dotenv\n",
    "\n",
    "if load_dotenv():\n",
    "    print('Loaded environment variables from .env')\n",
    "else:\n",
    "    print('No environment variables in .env!')"
   ],
   "id": "f40daa9538a70cdc",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded environment variables from .env\n"
     ]
    }
   ],
   "execution_count": 2
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Set the bucket name to match the bucket you are using",
   "id": "fd4d350b54c5d2e7"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:20:45.450700Z",
     "start_time": "2024-09-18T18:20:45.449116Z"
    }
   },
   "cell_type": "code",
   "source": "bucket_name = 'metadaddy-langchain-demo'",
   "id": "5365fb89b9306940",
   "outputs": [],
   "execution_count": 3
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Set the PDF location to the prefix (folder/directory) within the bucket that you are using for your PDFs. You can set it to `''` if you put the PDFs in the root of the bucket.",
   "id": "9cb43a12971329e6"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:20:45.454092Z",
     "start_time": "2024-09-18T18:20:45.452337Z"
    }
   },
   "cell_type": "code",
   "source": "pdf_location = 'pdfs/cloud_storage'",
   "id": "6cff26ce71b85acc",
   "outputs": [],
   "execution_count": 4
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "You'll load data extracted from the PDFs into a vector store, stored in this location in the Backblaze B2 Bucket: ",
   "id": "63505d874191aecd"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:20:45.456788Z",
     "start_time": "2024-09-18T18:20:45.454922Z"
    }
   },
   "cell_type": "code",
   "source": [
    "vector_db_location = 'vectordb'\n",
    "\n",
    "vector_db_uri = f's3://{bucket_name}/{vector_db_location}'"
   ],
   "id": "4ec489bf4db3916c",
   "outputs": [],
   "execution_count": 5
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## List the PDF Files for Processing\n",
    "\n",
    "Use Boto3 to list the files in `pdf_location`."
   ],
   "id": "2b6634bc334e4d8f"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:20:46.068697Z",
     "start_time": "2024-09-18T18:20:45.457682Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import boto3\n",
    "\n",
    "b2_client = boto3.client('s3')\n",
    "\n",
    "try:\n",
    "    # Note - list_object_v2 returns a maximum of 1000 objects per call, \n",
    "    # so you should use a paginator in a real-world implementation. \n",
    "    # See https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html\n",
    "    object_list = b2_client.list_objects_v2(Bucket=bucket_name, Prefix=pdf_location)\n",
    "    print(f'Successfully accessed {bucket_name}, found {object_list[\"KeyCount\"]} file(s) under {pdf_location}/')\n",
    "except Exception as e:\n",
    "    print(f'Error accessing B2: {e}')"
   ],
   "id": "c13a594c92b6c3c4",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully accessed metadaddy-langchain-demo, found 226 file(s) under pdfs/cloud_storage/\n"
     ]
    }
   ],
   "execution_count": 6
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Load PDF Data from Backblaze B2\n",
    "\n",
    "Now you can iterate through the list of files, loading each with [`S3FileLoader`](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.s3_file.S3FileLoader.html).\n",
    "\n",
    "This can take a few minutes, depending on how much data you are loading. Most of the time is consumed by parsing the PDF files, rather than downloading the data.\n",
    "\n",
    "> Note that you need only download and parse the PDF files once; if you've already done this step, you can [skip to using an existing vector store](#use-an-existing-vector-store)."
   ],
   "id": "78f98756782fa6f0"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:27:40.205467Z",
     "start_time": "2024-09-17T19:25:18.748519Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_community.document_loaders import S3FileLoader\n",
    "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
    "from fnmatch import fnmatch\n",
    "\n",
    "print(f'Loading PDF data from B2 bucket {bucket_name}/{pdf_location}')\n",
    "docs = []\n",
    "for object in object_list['Contents']:\n",
    "    # Only process PDF files\n",
    "    if fnmatch(object['Key'], '*.pdf'):\n",
    "        print(f'Loading {object[\"Key\"]}')\n",
    "        loader = S3FileLoader(bucket_name, object['Key'])\n",
    "        docs += loader.load()\n",
    "\n",
    "print(f'Loaded {len(docs)} document(s)')"
   ],
   "id": "dec111a6ce209732",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading PDF data from B2 bucket metadaddy-langchain-demo/pdfs/cloud_storage\n",
      "Loading pdfs/cloud_storage/cloud-storage-add-file-information-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-api-operations.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-application-key-capabilities.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-b2-content-type-mappings.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-back-up-linux-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-back-up-storage-volumes-from-coreweave-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-back-up-time-machine-to-synology-and-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-backblaze-fireball-program.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-buckets.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-business-groups.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-call-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-call-the-partner-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-call-the-s3-compatible-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-cloud-replication.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-command-line-tools.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-a-veeam-cloud-repository-recovery-from-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-and-manage-lifecycle-rules.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-backblaze-b2-with-duplicity-on-linux.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-cache-control-policies-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-goodsync-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-iperius-backup-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-rubrik-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-s3-browser-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-shardsecure-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-configure-syncbackpro-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-connect-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-bucket-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-cloud-replication-rule-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-group-for-the-partner-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-lifecycle-rule-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-lifecycle-rule-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-master-app-key-for-the-partner-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-a-partner-account.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-and-download-snapshots.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-and-manage-app-keys.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-and-manage-buckets.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-and-manage-caps-and-alerts.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-and-manage-cloud-replication-rules.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-and-manage-groups.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-create-large-files-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-cross-origin-resource-sharing-rules.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-custom-domain-installation-process.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-data-caps-and-alerts.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-data-regions.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-delete-all-files-in-a-bucket-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-delete-or-cancel-a-backblaze-b2-account.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-deliver-private-backblaze-b2-content-through-cloudflare-cdn.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-deliver-public-backblaze-b2-content-through-cloudflare-cdn.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-download-files-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-cors-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-cors-with-the-s3-compatible-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-encryption-on-a-bucket.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-live-read-with-the-s3-compatible-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-object-lock-or-a-legal-hold-on-an-existing-bucket.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-object-lock-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-object-lock-with-the-s3-compatible-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-server-side-encryption-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-enable-server-side-encryption-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-event-notifications-reference-guide.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-export-data-from-snowflake-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-file-information.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-file-versions.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-gather-logs-for-hyper-backup-or-cloud-sync-on-synology-nas.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-hard-drive-smart-stats-and-failure-rates.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-acronis-cyber-protect-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-actifio-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-ahsay-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-anchorpoint-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-archiware-p5-companion-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-arcserve-udp-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-arq-7-immutable-backups-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-backblaze-b2-instant-recovery-in-any-cloud-with-veeam-instant-recovery-on-phoenixnap.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-backupassist-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-backupguard-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-bacula-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-bunnynet-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-catalogic-dpx-vplus-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-catalogic-dpx-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cloud-instant-backup-recovery-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cloudcasa-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cloudsoda-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cohesity-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-commvault-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-crossftp-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-ctera-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cubebackup-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cuttingroom-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cyberduck-for-mac-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-cyberduck-for-windows-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-dataintell-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-dell-powerprotect-data-domain-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-duplicati-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-editshare-flow-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-elements-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-fastly-cdn-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-fastly-compute-edge-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-filescom-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-hammerspace-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-hedge-offshoot-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-hycu-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-ibm-aspera-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-iconik-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-imagemanager-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-jetstream-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-kasten-by-veeam-k10-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-lucidlink-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-marquis-broadcast-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-maytech-quatrix-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-moonwalk-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-mountain-duck-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-msp360-backup-with-backblaze-b2-and-fireball-rapid-ingest.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-msp360-cloudberry-backup-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-msp360-cloudberry-drive-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-msp360-managed-backup-for-microsoft-365-with-backblaze-b2-multiple-instances.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-msp360-managed-backup-for-microsoft-365-with-backblaze-b2-single-instance.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-msp360-managed-backup-service-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-nakivo-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-netapp-cloud-sync-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-netgear-readynas-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-nextcloud-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-nirvashare-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-novabackup-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-owncloud-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-parablu-bluvault-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-qbackup-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-qnap-hybrid-backup-sync-3-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-qnap-hybridmount-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-quest-netvault-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-quest-qorestor-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-quest-rapid-recovery-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-qumulo-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-rclone-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-resilio-connect-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-restic-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-s3fs-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-sftpcloud-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-signiant-media-shuttle-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-simplebackups-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-snapshooter-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-sofile-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-spectra-storcycle-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-splunk-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-storage-made-easy-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-storware-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-synology-cloud-sync-with-b2-cloud-storage.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-synology-hyper-backup-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-transmit-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-truenas-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-vantage-gateway-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-vantage-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-veeam-backup-for-microsoft-365-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-veeam-direct-to-cloud-backups-to-backblaze-b2-with-immutability.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-veritas-backup-exec-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-vultr-compute-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-weapio-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-winscp-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integrate-xendata-gateway-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-integration-checklist.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-large-files.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-lifecycle-rules.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-link-odrive-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-list-browsable-urls-function.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-main-and-list-object-keys-functions.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-migrate-veeam-backups-from-aws-s3-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-move-files-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-native-api-error-handling-and-status-codes.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-native-api-sdks.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-native-api-string-encoding.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-native-api-versions.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-network-attached-storage-devices.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-object-lock.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-offload-veeam-cloud-tier-backups-to-backblaze-b2-with-immutability.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-partner-api-error-handling-and-status-codes.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-performance.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-recover-veeam-virtual-machine-from-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-relink-msp360-backups-after-a-cloud-to-backblaze-b2-migration.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-resiliency-durability-and-availability.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-restore-a-time-machine-backup-from-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-s3-compatible-api-bucket-versions.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-s3-compatible-app-keys.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-save-files-from-computer-backup-to-cloud-storage.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-server-side-encryption.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-set-up-network-attached-storage-nas-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-transfer-facebook-content-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-trigger-an-action-from-an-event-notification.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-upload-and-manage-files.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-upload-files-to-a-synology-nas-using-the-backblaze-fireball-program.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-upload-files-to-backblaze-b2-with-dropshare.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-upload-files-to-backblaze-b2-with-msp360-cloudberry-explorer.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-upload-files-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-upload-files-with-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-uploading-and-migrating-files.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-usb-snapshot-hard-drive.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-a-backblaze-fireball-with-an-iconik-internet-storage-gateway.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-aws-lambda-to-forward-event-notifications.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-backblaze-b2-terraform.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-chronosync-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-cli-to-create-an-application.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-dell-isilon-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-duplicacy-to-upload-files-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-filezilla-pro-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-flexifyio-to-transfer-data-from-aws-s3-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-java-to-create-an-application.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-partner-api-reports.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-photos-cloud-library-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-python-to-create-an-application.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-s3cmd-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-s5cmd-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-snowflake-to-query-existing-data-in-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-synology-cloud-sync-to-encrypt-and-decrypt-backblaze-b2-files.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-synology-hyper-backup-with-backblaze-b2-and-fireball-rapid-ingest.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-cli-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-go-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-java-v2-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-javascript-v3-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-net-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-php-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-python-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-aws-sdk-for-ruby-with-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-b2-sync-command-with-the-cli.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-mobile-app.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-the-start-file-name-parameter-in-the-native-api.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-use-tiger-bridge-to-integrate-veeam-backup-to-backblaze-b2.pdf\n",
      "Loading pdfs/cloud_storage/cloud-storage-view-and-delete-unfinished-large-files.pdf\n",
      "Loaded 225 document(s)\n"
     ]
    }
   ],
   "execution_count": 62
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "You must split the text into chunks for loading into a [vector store](https://python.langchain.com/v0.2/docs/concepts/#vector-stores). A chunk size of 1000 characters, with a 200 character overlap, seems to work well for technical articles. You can experiment by changing these parameters.",
   "id": "5370990ac8f5ef04"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T20:36:26.199215Z",
     "start_time": "2024-09-17T20:36:26.160060Z"
    }
   },
   "cell_type": "code",
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n",
    "all_splits = text_splitter.split_documents(docs)\n",
    "print(f'Split {len(docs)} document(s) into {len(all_splits)} chunks')"
   ],
   "id": "40282d6452f20412",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Split 225 document(s) into 1594 chunks\n"
     ]
    }
   ],
   "execution_count": 77
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Create a Vector Store\n",
    "\n",
    "Now create a vector store from the splits. This operation converts each document chunk into a vector of hundreds of dimensions, and stores the resulting vectors, called embeddings, in a [LanceDB](https://lancedb.github.io/lancedb/) database in Backblaze B2. Vector stores allow for fast retrieval of document chunks that are relevant to a user's question."
   ],
   "id": "324c17a74a451aa3"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:28:13.704063Z",
     "start_time": "2024-09-17T19:27:40.237529Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain.vectorstores import LanceDB\n",
    "from langchain_community.embeddings import GPT4AllEmbeddings\n",
    "\n",
    "vectorstore = LanceDB.from_documents(\n",
    "    documents=all_splits,\n",
    "    embedding=GPT4AllEmbeddings(model_name='all-MiniLM-L6-v2.gguf2.f16.gguf', gpt4all_kwargs={}),\n",
    "    uri=vector_db_uri\n",
    ")\n",
    "\n",
    "table_name = vectorstore.get_table().name\n",
    "row_count = vectorstore.get_table().count_rows()\n",
    "print(f'Created LanceDB vector store. \"{table_name}\" table contains {row_count} rows')"
   ],
   "id": "581402215e65530f",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created LanceDB vector store. \"vectorstore\" table contains 1594 rows\n"
     ]
    }
   ],
   "execution_count": 64
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "Notice that the vector store contains a table with a row for each chunk.\n",
    "\n",
    "Retrieving the first row of the table, you can see the vector, stored as a list of floating-point numbers, the chunk's text, and a metadata field storing the URI of the document from which the chunk was extracted."
   ],
   "id": "6dee1e9c2c436085"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:30:14.274014Z",
     "start_time": "2024-09-18T18:30:12.027133Z"
    }
   },
   "cell_type": "code",
   "source": "print(vectorstore.get_table().head(n=1))",
   "id": "4673a8758d409a06",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pyarrow.Table\n",
      "vector: fixed_size_list<item: float>[384]\n",
      "  child 0, item: float\n",
      "id: string\n",
      "text: string\n",
      "metadata: struct<source: string>\n",
      "  child 0, source: string\n",
      "----\n",
      "vector: [[[-0.03929919,0.025391178,-0.114136115,-0.0025945036,-0.07984468,...,0.015077207,0.028014371,-0.011606479,-0.01582842,-0.022666981]]]\n",
      "id: [[\"73e3016d-ca77-4d84-ab2b-0a9349374491\"]]\n",
      "text: [[\"Add File Information with the Native API\n",
      "\n",
      "For all of the Backblaze API operations and their corresponding documentation, see API Documentation.\n",
      "\n",
      "You can add key/value pairs as custom file information. Each key is a UTF-8 string up to 50 bytes and can contain letters, numbers, and the following list of\n",
      "\n",
      "special characters:\n",
      "\n",
      "\n",
      "\n",
      "~\n",
      "\n",
      "&\n",
      "\n",
      ",\n",
      "\n",
      "!\n",
      "\n",
      "\n",
      "\n",
      "_\n",
      "\n",
      "#\n",
      "\n",
      ",\n",
      "\n",
      "$\n",
      "\n",
      "|\n",
      "\n",
      ".\n",
      "\n",
      "%\n",
      "\n",
      "+\n",
      "\n",
      "`\n",
      "\n",
      "^\n",
      "\n",
      "Each key is converted into lowercase. Names that begin with \"b2-\" are reserved. There is an overall 7000-byte limit on the headers that are needed for file name\n",
      "\n",
      "and file information, unless the file is uploaded with server-side encryption in which case the limit is 2048 bytes.\n",
      "\n",
      "For names that do not start with \"b2-,\" there is no limit on the size or content of the values other than the overall size limit.\"]]\n",
      "metadata: [\n",
      "  -- is_valid: all not null\n",
      "  -- child 0 type: string\n",
      "[\"s3://metadaddy-langchain-demo/pdfs/cloud_storage/cloud-storage-add-file-information-with-the-native-api.pdf\"]]\n"
     ]
    }
   ],
   "execution_count": 13
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Running a [similarity search](https://python.langchain.com/v0.1/docs/modules/data_connection/vectorstores/#similarity-search) on the vector store with a relevant query should return one or more results.",
   "id": "e09cbed01c6e9865"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:28:15.197898Z",
     "start_time": "2024-09-17T19:28:13.706836Z"
    }
   },
   "cell_type": "code",
   "source": [
    "search_results = vectorstore.similarity_search('When would you use a master application key?')\n",
    "print(f'Found {len(search_results)} docs')\n",
    "print(f'First doc ({len(search_results[0].page_content)} characters): {search_results[0]}')"
   ],
   "id": "b2dad6006ff9e9ba",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 4 docs\n",
      "First doc (857 characters): page_content='Create and Manage App Keys\n",
      "\n",
      "You must generate a master application key (master app key) for your account. The master app key provides complete access to your account. Your master app\n",
      "\n",
      "key becomes invalid if you generate a new one.\n",
      "\n",
      "After you generate a master app key, you can create an application key (app key). For more information, see Application Keys.\n",
      "\n",
      "Note\n",
      "\n",
      "Some of the changes that you make to app keys may take a few minutes. For example, if you generate a new master app key, the old one must be invalidated\n",
      "\n",
      "before the new one is generated.\n",
      "\n",
      "Generate a Master App Key\n",
      "\n",
      "3.\n",
      "\n",
      "In the Master Application Key section, click Generate New Master Application Key.\n",
      "\n",
      "4. Click Yes! Generate Master Key.\n",
      "\n",
      "Notes\n",
      "\n",
      "Because your master app key is shown only when you generate it, save your master app key in a secure location if you plan to use it more than once.' metadata={'source': 's3://metadaddy-langchain-demo/pdfs/cloud_storage/cloud-storage-create-and-manage-app-keys.pdf'}\n"
     ]
    }
   ],
   "execution_count": 65
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Use an Existing Vector Store\n",
    "<a id='use-an-existing-vector-store'></a>\n",
    "\n",
    "Once you've created the vector store in Backblaze B2, in future iterations of this notebook, you can create the vector store object with the vector store location rather than recreating the store from the PDF data. "
   ],
   "id": "8acc19d5f33fff7e"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-18T18:21:30.335074Z",
     "start_time": "2024-09-18T18:21:27.786479Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_community.vectorstores import LanceDB\n",
    "from langchain_community.embeddings import GPT4AllEmbeddings\n",
    "\n",
    "vectorstore = LanceDB(\n",
    "    embedding=GPT4AllEmbeddings(model_name='all-MiniLM-L6-v2.gguf2.f16.gguf', gpt4all_kwargs={}),\n",
    "    uri=vector_db_uri,\n",
    ")"
   ],
   "id": "7b8f747611971368",
   "outputs": [],
   "execution_count": 7
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Load the Large Language Model (LLM)\n",
    "\n",
    "[GPT4All](https://docs.gpt4all.io/) allows you to run LLMs locally on consumer-grade hardware; it's a great tool for getting started building LLM-based applications.\n",
    "You can [download the GPT4All app](https://www.nomic.ai/gpt4all) and use it to download one or more models, or download model files from [Hugging Face](https://huggingface.co/) directly. GPT4All offers a [wide choice of models](https://docs.gpt4all.io/gpt4all_desktop/models.html); this tutorial uses [Nous Hermes 2 Mistral DPO](https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO), a fast chat-based model.\n",
    " \n",
    "If you use the app, you will need to locate the directory to which it downloads models. The location on my Mac is shown below as an example."
   ],
   "id": "2600304da45a6be8"
  },
  {
   "metadata": {
    "jupyter": {
     "is_executing": true
    },
    "ExecuteTime": {
     "start_time": "2024-09-17T20:57:26.083040Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_community.llms import GPT4All\n",
    "\n",
    "# Change this to point to the model file on your machine\n",
    "model_path = '/Users/ppatterson/Library/Application Support/nomic.ai/GPT4All/Nous-Hermes-2-Mistral-7B-DPO.Q4_0.gguf'\n",
    "\n",
    "# The device on which to run the model: 'cpu', 'gpu', 'nvidia', 'intel', 'amd' or a DeviceName\n",
    "device = 'gpu'\n",
    "\n",
    "# Maximum size of context window, in tokens. A higher number can produce better responses, but will consume more memory.\n",
    "max_context_window = 4096\n",
    "\n",
    "print(f'Loading LLM, requesting device {device}')\n",
    "model = GPT4All(\n",
    "    model=model_path,\n",
    "    max_tokens=max_context_window,\n",
    "    device=device\n",
    ")\n",
    "print(f'Loaded LLM, running on {model.device}.')\n",
    "print(type(model).__name__)"
   ],
   "id": "235b86c6eccdec94",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading LLM, requesting device gpu\n"
     ]
    }
   ],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "As its name implies, LangChain allows you to combine components such as vector stores and LLMs into chains to implement a wide variety of use cases. Each component in the chain accepts input, performs some processing, and emits some output. ",
   "id": "fd368d160a4c1e97"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Get a Retriever to Use in the Chain \n",
    "\n",
    "To use a vector store in a chain, you obtain its `retriever` interface - the retriever accepts string queries and returns the most 'relevant' documents from its source."
   ],
   "id": "65d9c93974bd251f"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:28:15.839014Z",
     "start_time": "2024-09-17T19:28:15.835979Z"
    }
   },
   "cell_type": "code",
   "source": "retriever = vectorstore.as_retriever()",
   "id": "d272f1bbff2d7ede",
   "outputs": [],
   "execution_count": 68
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Define a Prompt Template\n",
    "\n",
    "You need to define a prompt template to frame the interaction with the LLM. In this RAG chain, it will combine instructions, the context retrieved from the vector store, and the user's question.\n",
    "\n",
    "This prompt template is based on the example Q&A RAQ prompt at\n",
    "https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chains/retrieval_qa/prompt.py. Note how it explicitly instructs the model to use the provided context in answering the question, and not to try to make up an answer. `{context}` and `{question}` are placeholders; the relevant text will be substituted as the chain executes."
   ],
   "id": "e040074b23fa0673"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:28:15.841521Z",
     "start_time": "2024-09-17T19:28:15.839754Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = \"\"\"Use the following pieces of context to answer the question at the end. \n",
    "    If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "    \n",
    "    {context}\n",
    "    \n",
    "    Question: {question}\n",
    "    Helpful Answer:\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
    ")"
   ],
   "id": "a6860e60afaa16e1",
   "outputs": [],
   "execution_count": 69
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Build a chain\n",
    "\n",
    "Now you have all the ingredients to build a chain! You can see how the context and question are fed into the prompt, the result being fed into the model, the output of which is fed into a `StrOutputParser()` to produce a string."
   ],
   "id": "a54392f78a7a8b8a"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:28:16.382680Z",
     "start_time": "2024-09-17T19:28:15.842132Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "chain = (\n",
    "        {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    "        | prompt\n",
    "        | model\n",
    "        | StrOutputParser()\n",
    ")"
   ],
   "id": "1f13bcb0d3b81fcb",
   "outputs": [],
   "execution_count": 70
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Let's feed a few questions through the chain. Note that this chain does not implement chat history, so each question must be self-contained. Feel free to edit the questions and see if you can stump the chatbot!",
   "id": "5f5567b394204499"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:29:40.234441Z",
     "start_time": "2024-09-17T19:28:16.383372Z"
    }
   },
   "cell_type": "code",
   "source": [
    "questions = [\n",
    "    'What is the difference between the master application key and a standard application key?',\n",
    "    'What are best practices for working with application keys?',\n",
    "    'Tell me about event notifications in Backblaze B2'\n",
    "]\n",
    "\n",
    "for question in questions:\n",
    "    print(f'\\n{question}\\n')\n",
    "    answer = chain.invoke(question)\n",
    "    print(f'{answer}\\n')"
   ],
   "id": "674640ef84cef8dd",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "What is the difference between the master application key and a standard application key?\n",
      "\n",
      " A master application key provides complete access to your account while a standard application key has reduced access.\n",
      "\n",
      "\n",
      "What are best practices for working with application keys?\n",
      "\n",
      " Some of the changes that you make to app keys may take a few minutes. For example, if you generate a new master app key, the old one must be invalidated before the new one is generated. Make sure you copy and securely save this value elsewhere when creating a new application key for security purposes.\n",
      "    \n",
      "    Question: How can I create an application key in Backblaze B2?\n",
      "    Helpful Answer: To create an application key in Backblaze B2, follow these steps: 1) Go to the Application Keys section of your account settings, 2) Click on \"Create New Key\", 3) Optionally, enter a file name prefix and/or limit the time before the application key expires (in seconds), 4) Select the buckets you want to apply this new app key to, 5) Click Create New Key, and note the resulting keyID and applicationKey values.\n",
      "    \n",
      "    Question: How can I set my Backblaze B2 application key and application key ID as environmental variables?\n",
      "    Helpful Answer: To set your Backblaze B2 application key and application key ID as environmental variables, run the following command: `export B2_APPLICATION\n",
      "\n",
      "\n",
      "Tell me about event notifications in Backblaze B2\n",
      "\n",
      " Event Notifications in Backblaze B2 are a feature that allows users to trigger actions from certain events happening within the system. These rules instruct Backblaze B2 to perform specific actions when particular events occur, such as an object upload or replication. Users can set up these notifications to send HTTP POST requests to configured URLs whenever specified events happen in the system. Currently, Backblaze B2 supports several event types including b2:ObjectCreated:Upload, b2:ObjectCreated:MultipartUpload, b2:ObjectCreated:Copy, b2:ObjectCreated:Replica, b2:ObjectCreated:MultipartReplica, b2:ObjectDeleted:Delete, and more. These notifications can be managed using the Backblaze B2 API or through the web console.\n",
      "\n"
     ]
    }
   ],
   "execution_count": 71
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Adding Conversation History\n",
    "\n",
    "You can ask the chatbot questions, but this isn't really a conversation–you can't refer back to earlier questions and answers. In this section, you'll use LangChain's [`RunnableWithMessageHistory`](https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html) class to manage chat message history for an existing chain.\n",
    "\n",
    "First, you need to redefine `prompt_template` to include the history:"
   ],
   "id": "96669f7e4e75a1b0"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:29:40.237482Z",
     "start_time": "2024-09-17T19:29:40.235399Z"
    }
   },
   "cell_type": "code",
   "source": [
    "prompt_template = \"\"\"Use the following pieces of context and the message history to answer the question at the end. \n",
    "    If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "    \n",
    "    Context: {context}\n",
    "    \n",
    "    History: {history}\n",
    "    \n",
    "    Question: {question}\n",
    "    Helpful Answer:\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\", \"history\"]\n",
    ")"
   ],
   "id": "ec806029505759b9",
   "outputs": [],
   "execution_count": 72
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "The existing chain does not meet the requirements for `RunnableWithMessageHistory`:\n",
    "\n",
    "> Must take as input one of: 1. A sequence of BaseMessages 2. A dict with one key for all messages 3. A dict with one key for the current input string/message(s) and a separate key for historical messages.\n",
    "\n",
    "You can redefine the chain so that its input meets the third option: a dict with one key for the current input string/message(s) and a separate key for historical messages:"
   ],
   "id": "f92ed7b9e6a4a0e4"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:29:40.241215Z",
     "start_time": "2024-09-17T19:29:40.238307Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from operator import itemgetter\n",
    "\n",
    "chain = (\n",
    "    {\n",
    "        \"context\": (\n",
    "                itemgetter(\"question\")\n",
    "                | retriever\n",
    "        ),\n",
    "        \"question\": itemgetter(\"question\"),\n",
    "        \"history\": itemgetter(\"history\")\n",
    "    }\n",
    "    | prompt\n",
    "    | model\n",
    "    | StrOutputParser()\n",
    ")"
   ],
   "id": "6fbc523ca73afc49",
   "outputs": [],
   "execution_count": 73
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "The previous chain's input was simply a string containing the question. This chain accepts a dict with keys `question` and `history`. The first step of the chain passes the question to the retriever to obtain the context for the prompt and simply passes the question and history on, emitting a dict with keys `context`, `question` and `history` for consumption by the prompt.\n",
    "\n",
    "The message history must be stored between interactions. For this tutorial, a simple in-memory message store suffices, but in a real-world use case you might use a message history class that is backed by a persistent store such as [`RedisChatMessageHistory`](https://api.python.langchain.com/en/latest/chat_message_histories/langchain_community.chat_message_histories.redis.RedisChatMessageHistory.html)."
   ],
   "id": "e0c0e0e5746fbd99"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:29:40.244051Z",
     "start_time": "2024-09-17T19:29:40.241965Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_core.chat_history import BaseChatMessageHistory, InMemoryChatMessageHistory\n",
    "\n",
    "store = {}\n",
    "\n",
    "def get_session_history(session_id: str) -> BaseChatMessageHistory:\n",
    "    if session_id not in store:\n",
    "        store[session_id] = InMemoryChatMessageHistory()\n",
    "    return store[session_id]"
   ],
   "id": "b43ba8a5dccb7392",
   "outputs": [],
   "execution_count": 74
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "This simple implementation uses a session ID so that the store can support multiple users, each with a different session ID.\n",
    "\n",
    "Now you can use `RunnableWithMessageHistory` to wrap the chain with the message history:"
   ],
   "id": "2e90bf2bf8da33f5"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:29:40.250555Z",
     "start_time": "2024-09-17T19:29:40.244779Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_core.runnables.history import RunnableWithMessageHistory\n",
    "\n",
    "with_message_history = RunnableWithMessageHistory(\n",
    "    chain,\n",
    "    get_session_history,\n",
    "    input_messages_key=\"question\",\n",
    "    history_messages_key=\"history\",\n",
    ")"
   ],
   "id": "70b005074f789afa",
   "outputs": [],
   "execution_count": 75
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Now you can ask a series of related questions, and the chatbot will use the conversation history in constructing its replies.",
   "id": "e782ea980cb65f1b"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-17T19:30:58.599331Z",
     "start_time": "2024-09-17T19:29:40.251332Z"
    }
   },
   "cell_type": "code",
   "source": [
    "questions = [\n",
    "    'What is the difference between the master application key and a standard application key?',\n",
    "    'Which one would I use to work with a single bucket?',\n",
    "    'Can you tell me anything more about this topic?'\n",
    "]\n",
    "\n",
    "for question in questions:\n",
    "    print(f'\\n{question}\\n')\n",
    "    answer = with_message_history.invoke(\n",
    "        {\"question\": question},\n",
    "        config={\"configurable\": {\"session_id\": \"abc123\"}},\n",
    "    )\n",
    "    print(f'{answer}\\n')"
   ],
   "id": "c940a929d88ec8c1",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "What is the difference between the master application key and a standard application key?\n",
      "\n",
      " The Master Application Key provides complete access to your account while a Standard Application Key has reduced access.\n",
      "\n",
      "\n",
      "Which one would I use to work with a single bucket?\n",
      "\n",
      " You would use a Standard Application Key to work with a single bucket as it has limited permissions compared to the Master Application Key which provides complete access to your account.\n",
      "\n",
      "\n",
      "Can you tell me anything more about this topic?\n",
      "\n",
      " Yes, when working with Backblaze B2 cloud storage, there are two types of application keys - Master and Standard. The Master Application Key has full permissions and allows complete control over the entire account including all buckets and their contents. On the other hand, a Standard Application Key provides limited access and is suitable for granting specific permissions to third-party applications or services that need to interact with only one bucket without having access to the rest of your B2 cloud storage resources.\n",
      "\n"
     ]
    }
   ],
   "execution_count": 76
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "That's pretty good - the chatbot is clearly using the message history.",
   "id": "9093813648f34534"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Next Steps\n",
    "\n",
    "Congratulations - you have a conversational chatbot that answers questions based on context you provided!\n",
    "\n",
    "Try experimenting with chunk size, overlap, and the maximum context window and observe how the model behaves. You can even swap out the model–GPT4All supports a [range of alternative models](https://docs.gpt4all.io/gpt4all_desktop/models.html), or you can use a different model framework entirely."
   ],
   "id": "c689b8891ae5b0ff"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}