fine_tune_ai_model.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DigitalPasts/LLM-fine-tuning/blob/main/fine_tune_ai_model.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "OMAQZiNND0sG"
      },
      "source": [
        "# Fine-Tune Large Language Models for Hittite Glossing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k9lGh_J0D0sH"
      },
      "source": [
        "\n",
        "In this notebook, we explore the fine-tuning of the T5 large language model (LLM) for the task of Hittite glossing, focusing on its adaptability to low-resource ancient languages. We employ the pre-trained T5 model to investigate its efficacy in addressing the unique challenges of Hittite morphology. The notebook outlines the process of fine-tuning T5 and evaluates its performance using metrics such as token-level accuracy."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d8qHP2YJD0sI"
      },
      "source": [
        "# Table of Contents"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "Xcn798btD0sI"
      },
      "source": [
        "- [ 1 - Set up Kernel, Load Required Dependencies, Dataset and LLM](#1)\n",
        "  - [ 1.1 - Set up Kernel and Required Dependencies](#1.1)\n",
        "  - [ 1.2 - Load Dataset and LLM](#1.2)\n",
        "  - [ 1.3 - Test the Model with Zero Shot Inferencing](#1.3)\n",
        "- [ 2 - Perform Full Fine-Tuning](#2)\n",
        "  - [ 2.1 - Preprocess the Dialog-Summary Dataset](#2.1)\n",
        "  - [ 2.2 - Fine-Tune the Model with the Preprocessed Dataset](#2.2)\n",
        "  - [ 2.3 - Evaluate the Model Qualitatively (Human Evaluation)](#2.3)\n",
        "  - [ 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)](#2.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vZXbV0hgD0sJ"
      },
      "source": [
        "<a name='1'></a>\n",
        "## 1 - Set up Kernel, Load Required Dependencies, Dataset and LLM"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eG6rUp0aD0sK"
      },
      "source": [
        "<a name='1.1'></a>\n",
        "### 1.1 - Set up Kernel and Required Dependencies"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "pKKSAMwRD0sL"
      },
      "source": [
        "Now install the required packages for the LLM and datasets.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tPEBI3qID0sL",
        "outputId": "a7389c3c-597b-4756-f2d9-9fefee34b031"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: pip in /usr/local/lib/python3.10/dist-packages (24.1.2)\n",
            "Collecting pip\n",
            "  Downloading pip-24.3.1-py3-none-any.whl.metadata (3.7 kB)\n",
            "Downloading pip-24.3.1-py3-none-any.whl (1.8 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m61.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: pip\n",
            "  Attempting uninstall: pip\n",
            "    Found existing installation: pip 24.1.2\n",
            "    Uninstalling pip-24.1.2:\n",
            "      Successfully uninstalled pip-24.1.2\n",
            "Successfully installed pip-24.3.1\n",
            "Collecting torch==1.13.1\n",
            "  Downloading torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl.metadata (24 kB)\n",
            "Collecting datasets==2.11.0\n",
            "  Downloading datasets-2.11.0-py3-none-any.whl.metadata (20 kB)\n",
            "Collecting evaluate==0.4.0\n",
            "  Downloading evaluate-0.4.0-py3-none-any.whl.metadata (9.4 kB)\n",
            "Collecting transformers==4.27.2\n",
            "  Downloading transformers-4.27.2-py3-none-any.whl.metadata (106 kB)\n",
            "Collecting rouge_score==0.1.2\n",
            "  Downloading rouge_score-0.1.2.tar.gz (17 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting loralib==0.1.1\n",
            "  Downloading loralib-0.1.1-py3-none-any.whl.metadata (14 kB)\n",
            "Collecting peft==0.3.0\n",
            "  Downloading peft-0.3.0-py3-none-any.whl.metadata (21 kB)\n",
            "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (0.2.0)\n",
            "Requirement already satisfied: openai in /usr/local/lib/python3.10/dist-packages (1.54.4)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (2.2.2)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (1.26.4)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (3.8.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (4.66.6)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==1.13.1) (4.12.2)\n",
            "Collecting nvidia-cuda-runtime-cu11==11.7.99 (from torch==1.13.1)\n",
            "  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cudnn-cu11==8.5.0.96 (from torch==1.13.1)\n",
            "  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cublas-cu11==11.10.3.66 (from torch==1.13.1)\n",
            "  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch==1.13.1)\n",
            "  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (17.0.0)\n",
            "Collecting dill<0.3.7,>=0.3.0 (from datasets==2.11.0)\n",
            "  Downloading dill-0.3.6-py3-none-any.whl.metadata (9.8 kB)\n",
            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (2.32.3)\n",
            "Collecting xxhash (from datasets==2.11.0)\n",
            "  Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
            "Collecting multiprocess (from datasets==2.11.0)\n",
            "  Downloading multiprocess-0.70.17-py310-none-any.whl.metadata (7.2 kB)\n",
            "Requirement already satisfied: fsspec>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]>=2021.11.1->datasets==2.11.0) (2024.10.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (3.11.2)\n",
            "Requirement already satisfied: huggingface-hub<1.0.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (0.26.2)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (24.2)\n",
            "Collecting responses<0.19 (from datasets==2.11.0)\n",
            "  Downloading responses-0.18.0-py3-none-any.whl.metadata (29 kB)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (6.0.2)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.27.2) (3.16.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.27.2) (2024.9.11)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.27.2)\n",
            "  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from rouge_score==0.1.2) (1.4.0)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from rouge_score==0.1.2) (3.9.1)\n",
            "Requirement already satisfied: six>=1.14.0 in /usr/local/lib/python3.10/dist-packages (from rouge_score==0.1.2) (1.16.0)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft==0.3.0) (5.9.5)\n",
            "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (from peft==0.3.0) (1.1.1)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==1.13.1) (75.1.0)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==1.13.1) (0.45.0)\n",
            "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from openai) (1.9.0)\n",
            "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai) (0.27.2)\n",
            "Requirement already satisfied: jiter<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from openai) (0.7.1)\n",
            "Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai) (2.9.2)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai) (1.3.1)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas) (2024.2)\n",
            "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas) (2024.2)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (1.3.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (4.55.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (1.4.7)\n",
            "Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (11.0.0)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (3.2.0)\n",
            "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (3.10)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (1.2.2)\n",
            "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (2.4.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (24.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (1.5.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (6.1.0)\n",
            "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (0.2.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (1.17.2)\n",
            "Requirement already satisfied: async-timeout<6.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (4.0.3)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (2024.8.30)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (1.0.7)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai) (0.14.0)\n",
            "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai) (2.23.4)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets==2.11.0) (3.4.0)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets==2.11.0) (2.2.3)\n",
            "Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.10/dist-packages (from accelerate->peft==0.3.0) (0.4.5)\n",
            "INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.\n",
            "Collecting multiprocess (from datasets==2.11.0)\n",
            "  Downloading multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)\n",
            "  Downloading multiprocess-0.70.15-py310-none-any.whl.metadata (7.2 kB)\n",
            "  Downloading multiprocess-0.70.14-py310-none-any.whl.metadata (6.6 kB)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->rouge_score==0.1.2) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->rouge_score==0.1.2) (1.4.2)\n",
            "Downloading torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl (887.5 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m887.5/887.5 MB\u001b[0m \u001b[31m32.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading datasets-2.11.0-py3-none-any.whl (468 kB)\n",
            "Downloading evaluate-0.4.0-py3-none-any.whl (81 kB)\n",
            "Downloading transformers-4.27.2-py3-none-any.whl (6.8 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.8/6.8 MB\u001b[0m \u001b[31m42.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading loralib-0.1.1-py3-none-any.whl (8.8 kB)\n",
            "Downloading peft-0.3.0-py3-none-any.whl (56 kB)\n",
            "Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m317.1/317.1 MB\u001b[0m \u001b[31m50.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.0/21.0 MB\u001b[0m \u001b[31m44.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m849.3/849.3 kB\u001b[0m \u001b[31m41.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m557.1/557.1 MB\u001b[0m \u001b[31m40.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading dill-0.3.6-py3-none-any.whl (110 kB)\n",
            "Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n",
            "Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m132.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading multiprocess-0.70.14-py310-none-any.whl (134 kB)\n",
            "Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)\n",
            "Building wheels for collected packages: rouge_score\n",
            "  Building wheel for rouge_score (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for rouge_score: filename=rouge_score-0.1.2-py3-none-any.whl size=24935 sha256=87038676c06df26e8586032626f7205d7bf1c9b924ba62b2f7741d9ea787fcdb\n",
            "  Stored in directory: /root/.cache/pip/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4\n",
            "Successfully built rouge_score\n",
            "Installing collected packages: tokenizers, xxhash, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cublas-cu11, loralib, dill, rouge_score, responses, nvidia-cudnn-cu11, multiprocess, transformers, torch, peft, datasets, evaluate\n",
            "  Attempting uninstall: tokenizers\n",
            "    Found existing installation: tokenizers 0.20.3\n",
            "    Uninstalling tokenizers-0.20.3:\n",
            "      Successfully uninstalled tokenizers-0.20.3\n",
            "  Attempting uninstall: transformers\n",
            "    Found existing installation: transformers 4.46.2\n",
            "    Uninstalling transformers-4.46.2:\n",
            "      Successfully uninstalled transformers-4.46.2\n",
            "  Attempting uninstall: torch\n",
            "    Found existing installation: torch 2.5.1+cu121\n",
            "    Uninstalling torch-2.5.1+cu121:\n",
            "      Successfully uninstalled torch-2.5.1+cu121\n",
            "  Attempting uninstall: peft\n",
            "    Found existing installation: peft 0.13.2\n",
            "    Uninstalling peft-0.13.2:\n",
            "      Successfully uninstalled peft-0.13.2\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "sentence-transformers 3.2.1 requires transformers<5.0.0,>=4.41.0, but you have transformers 4.27.2 which is incompatible.\n",
            "torchaudio 2.5.1+cu121 requires torch==2.5.1, but you have torch 1.13.1 which is incompatible.\n",
            "torchvision 0.20.1+cu121 requires torch==2.5.1, but you have torch 1.13.1 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed datasets-2.11.0 dill-0.3.6 evaluate-0.4.0 loralib-0.1.1 multiprocess-0.70.14 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 peft-0.3.0 responses-0.18.0 rouge_score-0.1.2 tokenizers-0.13.3 torch-1.13.1 transformers-4.27.2 xxhash-3.5.0\n"
          ]
        }
      ],
      "source": [
        "%pip install --upgrade pip\n",
        "%pip install \\\n",
        "    torch==1.13.1 \\\n",
        "    datasets==2.11.0 \\\n",
        "    evaluate==0.4.0 \\\n",
        "    transformers==4.27.2 \\\n",
        "    rouge_score==0.1.2 \\\n",
        "    loralib==0.1.1 \\\n",
        "    peft==0.3.0 \\\n",
        "    sentencepiece \\\n",
        "    openai \\\n",
        "    pandas \\\n",
        "    numpy \\\n",
        "    matplotlib \\\n",
        "    tqdm \\\n",
        "    evaluate\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "hY9Q1EkhD0sM"
      },
      "source": [
        "Import the necessary components.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Core Libraries\n",
        "import torch  # PyTorch for model training\n",
        "from transformers import T5Tokenizer, T5ForConditionalGeneration, AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig\n",
        "from transformers import Trainer, TrainingArguments\n",
        "from datasets import Dataset, load_metric\n",
        "\n",
        "# Utility Libraries\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import os\n",
        "import random\n",
        "import time\n",
        "import evaluate\n"
      ],
      "metadata": {
        "id": "LP79n0tKSjfp"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "MTpj8xMlD0sN"
      },
      "source": [
        "<a name='1.2'></a>\n",
        "### 1.2 - Load Dataset and LLM."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data = pd.read_csv(\"hittite_ds.csv\", index_col = 0)\n",
        "data.columns = ['txtid','lnr','cth','word', 'text', 'gloss','trans']\n",
        "data.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "NtMBUURyTGNI",
        "outputId": "203cdd02-7ef1-4b83-85f5-5d160f495c34"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        txtid     lnr  cth      word             text            gloss  \\\n",
              "0  IBoT 1.30+   Vs. 1  821   LUGALuš       ⸢LUGAL⸣-uš  FNL(u).NOM.SG.C   \n",
              "1  IBoT 1.30+   Vs. 1  821     kuapi         ku-wa-pí              CNJ   \n",
              "2  IBoT 1.30+   Vs. 1  821  DINGIRaš   DINGIR{MEŠ}-aš           D/L.PL   \n",
              "3  IBoT 1.30+   Vs. 1  821   aruaizi  a-ru-wa-a-ez-zi          3SG.PRS   \n",
              "4  IBoT 1.30+   Vs. 1  821    GUDU₁₂       {LÚ}GUDU₁₂      NOM.SG(UNM)   \n",
              "\n",
              "            trans  \n",
              "0           König  \n",
              "1      sobald als  \n",
              "2        Gottheit  \n",
              "3  sich verneigen  \n",
              "4       Gesalbter  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-2f91ba24-a44c-4d29-9c0c-f324304f1eea\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>txtid</th>\n",
              "      <th>lnr</th>\n",
              "      <th>cth</th>\n",
              "      <th>word</th>\n",
              "      <th>text</th>\n",
              "      <th>gloss</th>\n",
              "      <th>trans</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>LUGALuš</td>\n",
              "      <td>⸢LUGAL⸣-uš</td>\n",
              "      <td>FNL(u).NOM.SG.C</td>\n",
              "      <td>König</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>kuapi</td>\n",
              "      <td>ku-wa-pí</td>\n",
              "      <td>CNJ</td>\n",
              "      <td>sobald als</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>DINGIRaš</td>\n",
              "      <td>DINGIR{MEŠ}-aš</td>\n",
              "      <td>D/L.PL</td>\n",
              "      <td>Gottheit</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>aruaizi</td>\n",
              "      <td>a-ru-wa-a-ez-zi</td>\n",
              "      <td>3SG.PRS</td>\n",
              "      <td>sich verneigen</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>GUDU₁₂</td>\n",
              "      <td>{LÚ}GUDU₁₂</td>\n",
              "      <td>NOM.SG(UNM)</td>\n",
              "      <td>Gesalbter</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2f91ba24-a44c-4d29-9c0c-f324304f1eea')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-2f91ba24-a44c-4d29-9c0c-f324304f1eea button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-2f91ba24-a44c-4d29-9c0c-f324304f1eea');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-ebfbbf62-0c76-48af-b71d-0cd782c9c1d6\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ebfbbf62-0c76-48af-b71d-0cd782c9c1d6')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-ebfbbf62-0c76-48af-b71d-0cd782c9c1d6 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "422gwbGSTJsI",
        "outputId": "c37c9456-c8b0-4e6a-99db-76885df9bd34"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Index: 170496 entries, 0 to 170495\n",
            "Data columns (total 7 columns):\n",
            " #   Column  Non-Null Count   Dtype \n",
            "---  ------  --------------   ----- \n",
            " 0   txtid   170496 non-null  object\n",
            " 1   lnr     170496 non-null  object\n",
            " 2   cth     170496 non-null  int64 \n",
            " 3   word    170496 non-null  object\n",
            " 4   text    170496 non-null  object\n",
            " 5   gloss   170469 non-null  object\n",
            " 6   trans   170496 non-null  object\n",
            "dtypes: int64(1), object(6)\n",
            "memory usage: 10.4+ MB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "hf_dataset = Dataset.from_pandas(data[['word','gloss']])\n",
        "hf_dataset = hf_dataset.remove_columns(\"__index_level_0__\") if \"__index_level_0__\" in hf_dataset.column_names else hf_dataset\n",
        "\n",
        "hf_dataset"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "to2B9e5NTsPF",
        "outputId": "4274b5d8-29ed-4d8f-87b1-5372890d2c32"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['word', 'gloss'],\n",
              "    num_rows: 170496\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Split into train, validation, and test sets\n",
        "splits = hf_dataset.train_test_split(test_size=0.2, seed=43)  # 80% train, 20% test\n",
        "train_dataset = splits[\"train\"]\n",
        "test_dataset = splits[\"test\"]\n",
        "\n",
        "# Further split test set into validation and test\n",
        "val_test_splits = test_dataset.train_test_split(test_size=0.5, seed=43)  # 50/50 split\n",
        "val_dataset = val_test_splits[\"train\"]\n",
        "test_dataset = val_test_splits[\"test\"]"
      ],
      "metadata": {
        "id": "HCnnSYqrW2Lf"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "lsBukLzfVM6E"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "u7gZdCO6D0sN"
      },
      "source": [
        "Load the pre-trained [FLAN-T5 model](https://huggingface.co/docs/transformers/model_doc/flan-t5) and its tokenizer directly from HuggingFace. Notice that we can use the [small version](https://huggingface.co/google/flan-t5-base) of FLAN-T5. Setting `torch_dtype=torch.bfloat16` specifies the memory type to be used by this model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "10MH2Cf8D0sN",
        "outputId": "09c58d06-f54a-4d76-e6e9-bb94e24c2856"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:797: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        }
      ],
      "source": [
        "model_name='google/flan-t5-base'\n",
        "\n",
        "original_model = AutoModelForSeq2SeqLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_name)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "# original_model = original_model.to(device)\n"
      ],
      "metadata": {
        "id": "_LjCtm0ZP-P9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#original_model.config"
      ],
      "metadata": {
        "id": "bWIKs9neQHjC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# test_input = \"Provide the gloss for the word: LUGALuš\"\n",
        "# tokens = tokenizer(test_input, return_tensors=\"pt\")\n",
        "# print(tokens)"
      ],
      "metadata": {
        "id": "zf0c-tlPQ5Ts"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "vdrIk4NbD0sN"
      },
      "source": [
        "It is possible to pull out the number of model parameters and find out how many of them are trainable. The following function can be used to do that, at this stage, you do not need to go into details of it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "gc98NsgvD0sN",
        "outputId": "7210d292-0eed-46bd-c3db-c72b738bec51"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'trainable model parameters: 247577856, all model parameters: 247577856, percentage of trainable model parameters: 100.00%'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "def print_number_of_trainable_model_parameters(model):\n",
        "    trainable_model_params = 0\n",
        "    all_model_params = 0\n",
        "    for _, param in model.named_parameters():\n",
        "        all_model_params += param.numel()\n",
        "        if param.requires_grad:\n",
        "            trainable_model_params += param.numel()\n",
        "    return f\"trainable model parameters: {trainable_model_params}, all model parameters: {all_model_params}, percentage of trainable model parameters: {100 * trainable_model_params / all_model_params:.2f}%\"\n",
        "\n",
        "print_number_of_trainable_model_parameters(original_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "eu8OiYJGD0sN"
      },
      "source": [
        "<a name='1.3'></a>\n",
        "### 1.3 - Test the Model with Zero Shot Inferencing\n",
        "\n",
        "Test the model with the zero shot inferencing."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "random_indices = random.sample(range(len(test_dataset)), 5)\n",
        "#device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "for idx in random_indices:\n",
        "    hittite_word = test_dataset[idx]['word']\n",
        "    expected_gloss = test_dataset[idx]['gloss']\n",
        "    prompt = f\"Provide the morphological gloss for the following Hittite word:\\n\\n{hittite_word}\\n\\nGloss:\"\n",
        "    inputs = tokenizer(prompt, return_tensors=\"pt\")\n",
        "    output = tokenizer.decode(\n",
        "        original_model.generate(inputs[\"input_ids\"], max_new_tokens=50)[0],\n",
        "        skip_special_tokens=True,\n",
        "    )\n",
        "    print(f\"Input: {hittite_word}\")\n",
        "    print(f\"Expected Gloss: {expected_gloss}\")\n",
        "    print(f\"Generated Gloss: {output}\\n\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DUQpJ6BRTxNb",
        "outputId": "342d2ff3-e097-457a-caa2-7550f03cd764"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input: peḫutezi\n",
            "Expected Gloss: 3SG.PRS\n",
            "Generated Gloss: eloquent\n",
            "\n",
            "Input: QA-TAM-MA\n",
            "Expected Gloss: ADV\n",
            "Generated Gloss: QA\n",
            "\n",
            "Input: ekuzi\n",
            "Expected Gloss: 3SG.PRS\n",
            "Generated Gloss: ekuzi\n",
            "\n",
            "Input: 1-ENpat\n",
            "Expected Gloss: QUANcar += pat\n",
            "Generated Gloss: enpat\n",
            "\n",
            "Input: walḫannianzi\n",
            "Expected Gloss: 3PL.PRS.IMPF\n",
            "Generated Gloss: walannianzi\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "mD76Ut9KD0sO"
      },
      "source": [
        "<a name='2'></a>\n",
        "## 2 - Perform Full Fine-Tuning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "fYkZr5wSD0sO"
      },
      "source": [
        "<a name='2.1'></a>\n",
        "### 2.1 - Preprocess the Dataset\n",
        "\n",
        "Training Prompt (Hittite Word or Phrase):\n",
        "Prepend the instruction Provide the morphological gloss for the following Hittite word: before the input word or phrase.\n",
        "\n",
        "Example:\n",
        "\n",
        "\n",
        "```\n",
        "Provide the morphological gloss for the following Hittite word:\n",
        "\n",
        "LUGALuš\n",
        "\n",
        "Gloss:\n",
        "```\n",
        "\n",
        "\n",
        "\n",
        "Training Response (Gloss):\n",
        "The gloss should be the expected morphological annotation for the word.\n",
        "\n",
        "Example:\n",
        "\n",
        "\n",
        "\n",
        "```\n",
        "FNL(u).NOM.SG.C\n",
        "```\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def preprocess_function(example):\n",
        "    # Ensure input and target are strings\n",
        "    return {\n",
        "        \"input_text\": f\"Provide the morphological gloss for the following Hittite word: {str(example['word'])}. Gloss:\",\n",
        "        \"target_text\": str(example[\"gloss\"])\n",
        "    }"
      ],
      "metadata": {
        "id": "D8bPsv3HnmjQ"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset_preprocessed = train_dataset.map(preprocess_function)\n",
        "val_dataset_preprocessed = val_dataset.map(preprocess_function)\n",
        "test_dataset_preprocessed = test_dataset.map(preprocess_function)\n",
        "train_dataset_preprocessed[378]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88,
          "referenced_widgets": [
            "85343a327039490e8a4dfc931ba27210",
            "619c7c94327c4fb3899498dbd654862d",
            "adc89256ae3d44458cf98515a8e413e3",
            "535ec9ebb3754e278546335c49c17406",
            "8aa393ed9b0d43e0aea24a7d7e52ae2d",
            "5da9690b29ea487d8cab2dd45117dc29",
            "21e08da3dc2d495ca41abbf4413f5414",
            "9cd711d37fc2499bbd50bcbabd1b8196",
            "df0c02b481db40259246baefe8536c10",
            "aad629ff351c434ea8b0627d47bdb120",
            "2b4d6ee2d6bf4ff8932dae38299d6871",
            "099dc5fd592b4c289060721896c6bf61",
            "2ef3ba1dd2574a9b917f4c0256d5a0b6",
            "793f2955d77741a9873e4a4d376428ba",
            "d5625d14a26745c3843519056dad8a08",
            "6199935de5634f4a87bac34bfac08388",
            "cc066d810e434a06b74a6de54a2a9a44",
            "b861e15f47f34b3e91fe9e729b3dcd5c",
            "718153c15e404d6891ee871b74f128d2",
            "8f35636f5ce745d185366a73f6da83b5",
            "64fe9f8dc5d84c0d9e58b8ea232ff408",
            "abe0904f731b45fc8d1305e003b54dc3",
            "8ff136703e194f3e83e42bb1a0edbcd2",
            "ec75e58b1cff476db99e547122a7be6d",
            "8aa72a4eb6374e788553dac08f8932b5",
            "cecb5ac41c504c0695c7fea2d6505d54",
            "3731fa6330c447f58c398071ae3c6a77",
            "1ec38bf039a24767becfc6f4d5c2e2ba",
            "fe077ff5d7b2455495e6a47caa57f76c",
            "09767fff72f74c629828990ebeb18bcc",
            "7c8aba747e754ed2b4cb7a5bb7b3cf1f",
            "16aca6ce4fd44d3aa573f782994c1941",
            "6e09467fe54c4918bf420c6ebc7f51fa"
          ]
        },
        "id": "kM5KZvhdnpCv",
        "outputId": "e016862e-f599-4c37-b9db-82865e0f71d1"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/136396 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "85343a327039490e8a4dfc931ba27210"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "099dc5fd592b4c289060721896c6bf61"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8ff136703e194f3e83e42bb1a0edbcd2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'word': 'AZU',\n",
              " 'gloss': 'NOM.SG(UNM)',\n",
              " 'input_text': 'Provide the morphological gloss for the following Hittite word: AZU. Gloss:',\n",
              " 'target_text': 'NOM.SG(UNM)'}"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Tokenize datasets\n",
        "def tokenize_function(example):\n",
        "    model_inputs = tokenizer(\n",
        "        example[\"input_text\"], max_length=512, padding=\"max_length\", truncation=True\n",
        "    )\n",
        "    labels = tokenizer(\n",
        "        example[\"target_text\"], max_length=128, padding=\"max_length\", truncation=True\n",
        "    )\n",
        "    model_inputs[\"labels\"] = labels[\"input_ids\"]\n",
        "    return model_inputs"
      ],
      "metadata": {
        "id": "QHFq197dYJYu"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset_tokenized = train_dataset_preprocessed.map(tokenize_function, batched=True)\n",
        "val_dataset_tokenized = val_dataset_preprocessed.map(tokenize_function, batched=True)\n",
        "test_dataset_tokenized = test_dataset_preprocessed.map(tokenize_function, batched=True)\n",
        "\n",
        "# Set dataset format for PyTorch\n",
        "train_dataset_tokenized.set_format(type=\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n",
        "val_dataset_tokenized.set_format(type=\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n",
        "test_dataset_tokenized.set_format(type=\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17,
          "referenced_widgets": [
            "58a3e6a8f9a14013962f0fe9035bdab1",
            "0e53fe88a9684dabb39f9241c8dc85a5",
            "8babf151548545a8a27c9dd16eeb6fe8",
            "a90d9bb0a09641479ac07e5671eddda5",
            "7339a840fd8f4437a7cb8438670412cd",
            "0f4ce1d6d1f54a988129cf9777353b4c",
            "84a81ade99a1414187a53e7e6ebea402",
            "45ff813f6a224563a94e0c156b9706a5",
            "38b299f3bbe14019a9b3dc01b6428e47",
            "f96ce1b8f6de4f118e7d669564fc6996",
            "bfff2e84853a469f844037dee8423eca",
            "f8387dfc63d748fdb536503f189cd3da",
            "2acb23e8d66341a5b3578d546891c55d",
            "1ddb8f95ea4647f18a81b0ef09204586",
            "5c0648ed4ed244aa984c8ad08f3b9ecf",
            "2ab7efa658ca49579da2dc8ffa7d3e47",
            "ff76d5e486ee4cf3b9de86ef7fba6163",
            "aa7ae91ee7524dea9de05cd4ffd95050",
            "8fd3baef499f4ba5802e34b408ad44a4",
            "f55bb6e15ad34f45a268a166e0cd67ca",
            "cf4b3905a5a4408198db6249317fa42b",
            "d791bd5dae6548249a14a9728215f9d0",
            "51aaaf6b9f3c41a580612b41415709a6",
            "5ac399a1dcbb4056aac0f9161c899fa8",
            "f48f1980db594a768b2fae012f36629c",
            "5d302c50cb32495a80bfca58d095fbe2",
            "9c44e38defc9448897ae942be0d229c3",
            "6b1b40797e044f9f90da701fc25fcd28",
            "253b87dd09df4d079d7f2dcef826ba6f",
            "97ce3d634a2447efbc83a171c4f89470",
            "229f08b181bc4c76ba94c50e4806c7d8",
            "d17ba9a7e1c34ea68e698cd2124959a3",
            "769d8ae6ed1a4b2c92b21b32db4cfe56"
          ]
        },
        "id": "Uq23c1WGn-M9",
        "outputId": "54a8c75e-09a3-454b-a648-0c8a8bad73cc"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/136396 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "58a3e6a8f9a14013962f0fe9035bdab1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f8387dfc63d748fdb536503f189cd3da"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "51aaaf6b9f3c41a580612b41415709a6"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "p9mfXe55D0sO"
      },
      "source": [
        "Check the shapes of all three parts of the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8vPK2-ZKD0sP",
        "outputId": "a0b67009-6a01-42c1-a848-8c32d24d8657"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shapes of the datasets:\n",
            "Training: (136396, 7)\n",
            "Validation: (17050, 7)\n",
            "Test: (17050, 7)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['word', 'gloss', 'input_text', 'target_text', 'input_ids', 'attention_mask', 'labels'],\n",
              "    num_rows: 17050\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "print(f\"Shapes of the datasets:\")\n",
        "print(f\"Training: {train_dataset_tokenized.shape}\")\n",
        "print(f\"Validation: {val_dataset_tokenized.shape}\")\n",
        "print(f\"Test: {test_dataset_tokenized.shape}\")\n",
        "\n",
        "test_dataset_tokenized"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dGuGhqBSD0sP"
      },
      "source": [
        "The output dataset is ready for fine-tuning."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h3jWp7Z3D0sP"
      },
      "source": [
        "<a name='2.2'></a>\n",
        "### 2.2 - Fine-Tune the Model with the Preprocessed Dataset\n",
        "\n",
        "Now utilize the built-in Hugging Face `Trainer` class (see the documentation [here](https://huggingface.co/docs/transformers/main_classes/trainer)). Pass the preprocessed dataset with reference to the original model. Other training parameters are found experimentally and there is no need to go into details about those at the moment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "tags": [],
        "id": "ZFl8rAKLD0sP"
      },
      "outputs": [],
      "source": [
        "output_dir = f'./glossing-training-{str(int(time.time()))}'\n",
        "\n",
        "training_args = TrainingArguments(\n",
        "    output_dir=output_dir,          # Directory to save model/checkpoints\n",
        "    learning_rate=1e-5,             # Small learning rate for fine-tuning\n",
        "    num_train_epochs=3,             # 3 full passes through the dataset\n",
        "    weight_decay=0.01,              # Regularization to avoid overfitting\n",
        "    logging_steps=50,                # Log every step for detailed progress\n",
        "    evaluation_strategy=\"epoch\",    # Evaluate after each epoch\n",
        "    report_to=\"none\"\n",
        ")\n",
        "\n",
        "trainer = Trainer(\n",
        "    model=original_model,\n",
        "    args=training_args,\n",
        "    train_dataset=train_dataset_tokenized,\n",
        "    eval_dataset=val_dataset_tokenized\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "-yXarFyQD0sP"
      },
      "source": [
        "Start training process...\n",
        "\n",
        "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi4gUGxlYXNlIGJlIHBhdGllbnQuPC90ZXh0PgogICAgPHRleHQgeD0iMTAwIiB5PSI1NiIgZm9udC1mYW1pbHk9IkFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE0IiBmaWxsPSIjMzMzMzMzIj5Zb3UgY2FuIHNhZmVseSBpZ25vcmUgdGhlIHdhcm5pbmcgbWVzc2FnZXMuPC90ZXh0Pgo8L3N2Zz4K\" alt=\"Time alert open medium\"/>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 261
        },
        "id": "IJ4uG9VWD0sP",
        "outputId": "0996f059-40ea-4a83-8735-68e080d98f2b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='51150' max='51150' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [51150/51150 2:45:27, Epoch 3/3]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Epoch</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>27.977500</td>\n",
              "      <td>25.251848</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>27.907500</td>\n",
              "      <td>25.248636</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>27.835000</td>\n",
              "      <td>25.252552</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TrainOutput(global_step=51150, training_loss=27.9436119257087, metrics={'train_runtime': 9928.6683, 'train_samples_per_second': 41.213, 'train_steps_per_second': 5.152, 'total_flos': 2.8019449153349222e+17, 'train_loss': 27.9436119257087, 'epoch': 3.0})"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "Zw82ScgED0sP"
      },
      "source": [
        "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "eY0ArOwaD0sQ"
      },
      "source": [
        "Create an instance of the `AutoModelForSeq2SeqLM` class for the instruct model:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Save the fine-tuned model\n",
        "trainer.save_model(output_dir)\n",
        "\n",
        "# Save the tokenizer\n",
        "tokenizer.save_pretrained(output_dir)\n",
        "\n",
        "print(f\"Model and tokenizer saved to {output_dir}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2NySqmEhDfs_",
        "outputId": "d5ce622f-f711-4248-8f3a-40a50600a91d"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model and tokenizer saved to ./glossing-training-1733262418\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "instructed_model = AutoModelForSeq2SeqLM.from_pretrained(output_dir)\n",
        "instructed_tokenizer = AutoTokenizer.from_pretrained(output_dir)"
      ],
      "metadata": {
        "id": "d_z8OWZ8D5sx"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GFoo2ypyD0sV"
      },
      "source": [
        "<a name='2.3'></a>\n",
        "### 2.3 - Evaluate the Model Qualitatively (Human Evaluation)\n",
        "\n",
        "As with many GenAI applications, a qualitative approach where you ask yourself the question \"Is my model behaving the way it is supposed to?\" is usually a good starting point. In the example below (the same one we started this notebook with), you can see how the fine-tuned model is able to create a reasonable summary of the dialogue compared to the original inability to understand what is being asked of the model."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "random_indices = random.sample(range(len(test_dataset)), 5)"
      ],
      "metadata": {
        "id": "Uzy0qRW1spxP"
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for idx in random_indices:\n",
        "    hittite_word = test_dataset[idx]['word']\n",
        "    expected_gloss = test_dataset[idx]['gloss']\n",
        "    prompt = f\"Provide the morphological gloss for the following Hittite word:\\n\\n{hittite_word}\\n\\nGloss:\"\n",
        "    inputs = instructed_tokenizer(prompt, return_tensors=\"pt\")\n",
        "    output = instructed_tokenizer.decode(\n",
        "        instructed_model.generate(inputs[\"input_ids\"], max_new_tokens=50)[0],\n",
        "        skip_special_tokens=True,\n",
        "    )\n",
        "    print(f\"Input: {hittite_word}\")\n",
        "    print(f\"Expected Gloss: {expected_gloss}\")\n",
        "    print(f\"Generated Gloss: {output}\\n\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "At2J6Zp_DuCS",
        "outputId": "fd77dc29-26b0-495c-8929-0e47ef85fda0"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input: ekuzi\n",
            "Expected Gloss: 3SG.PRS\n",
            "Generated Gloss: a slick\n",
            "\n",
            "Input: mezulla\n",
            "Expected Gloss: DN.D/L.SG(UNM)\n",
            "Generated Gloss: a slick\n",
            "\n",
            "Input: QA-TAM-MApat\n",
            "Expected Gloss: ADV\n",
            "Generated Gloss: a syllable\n",
            "\n",
            "Input: pai\n",
            "Expected Gloss: 3SG.PRS\n",
            "Generated Gloss: slick\n",
            "\n",
            "Input: DINGIRnana\n",
            "Expected Gloss: FNL(a).GEN.PL.C\n",
            "Generated Gloss: a slick\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PfuH5AKHD0sV"
      },
      "source": [
        "<a name='2.4'></a>\n",
        "### 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)\n",
        "\n",
        "The [ROUGE metric](https://en.wikipedia.org/wiki/ROUGE_(metric)) helps quantify the validity of summarizations produced by models. It compares summarizations to a \"baseline\" summary which is usually created by a human. While not perfect, it does indicate the overall increase in summarization effectiveness that we have accomplished by fine-tuning."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "a9efd95a6d51480eb89b16a88b8247dd",
            "15616e3c4c17420e98bcb95f4e3796ec",
            "5bc42b1f68ee4231bec1d71449605f34",
            "704cc69c6aef41ff9c22af4a226382ed",
            "e13200ef28ad4c07a7a3e86b100d29f6",
            "1360b14c41b940c299c1606ea5bd1dd8",
            "36259a4ae78e41d797f42d2217645d82",
            "4082ae3cd97c4c4dbdd8e3afdd537e65",
            "1b6b235871f2409d98af2f9731234331",
            "935e2157b5a74490a736a560c825321e",
            "f0ff46ceeb8c4531be96e0bf7029bfe2"
          ]
        },
        "id": "DR8Ds5qFD0sV",
        "outputId": "8dc3a84f-fd6c-4150-eb9d-4be720ec9335"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/6.27k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a9efd95a6d51480eb89b16a88b8247dd"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "rouge = evaluate.load('rouge')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i_UCKoPzD0sW"
      },
      "source": [
        "Generate the outputs for the sample of the test dataset (only 10 dialogues and summaries to save time), and save the results."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Move models to the appropriate device (GPU if available)\n",
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "hittite_words = test_dataset[1100:1200]['word']  # Hittite words/phrases\n",
        "human_baseline_glosses = test_dataset[1100:1200]['gloss']  # Expected glosses (human-provided)\n",
        "\n",
        "original_model = original_model.to(device)\n",
        "instructed_model = instructed_model.to(device)\n",
        "\n",
        "# Initialize lists to store model outputs\n",
        "original_model_glosses = []\n",
        "instructed_model_glosses = []\n",
        "\n",
        "# Iterate through the selected examples\n",
        "for hittite_word in hittite_words:\n",
        "    # Create the input prompt\n",
        "    prompt = f\"\"\"\n",
        "    Provide the morphological gloss for the following Hittite word:\n",
        "\n",
        "    {hittite_word}\n",
        "\n",
        "    Gloss:\n",
        "    \"\"\"\n",
        "    # Tokenize the input prompt\n",
        "    input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(device)  # Move input_ids to the same device as the model\n",
        "    input_ids_instructed = instructed_tokenizer(prompt, return_tensors=\"pt\").input_ids.to(device)\n",
        "\n",
        "    # Generate gloss using the pre-trained (original) model\n",
        "    original_model_outputs = original_model.generate(\n",
        "        input_ids=input_ids,\n",
        "        generation_config=GenerationConfig(max_new_tokens=50)\n",
        "    )\n",
        "    original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
        "    original_model_glosses.append(original_model_text_output)\n",
        "\n",
        "    # Generate gloss using the fine-tuned (instructed) model\n",
        "    instructed_model_outputs = instructed_model.generate(\n",
        "        input_ids=input_ids_instructed,\n",
        "        generation_config=GenerationConfig(max_new_tokens=50)\n",
        "    )\n",
        "    instructed_model_text_output = instructed_tokenizer.decode(instructed_model_outputs[0], skip_special_tokens=True)\n",
        "    instructed_model_glosses.append(instructed_model_text_output)\n",
        "\n",
        "# Combine results into a DataFrame for analysis\n",
        "zipped_glosses = list(zip(human_baseline_glosses, original_model_glosses, instructed_model_glosses))\n",
        "df = pd.DataFrame(zipped_glosses, columns=['human_baseline_glosses', 'original_model_glosses', 'instruct_model_glosses'])\n",
        "\n",
        "# Display the DataFrame\n",
        "df\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "GkHbWgYlsFAh",
        "outputId": "026e4c7f-bf90-4ff3-8c02-d4abcaa8a523"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                               human_baseline_glosses original_model_glosses  \\\n",
              "0                                             3PL.PRS                rhyming   \n",
              "1                                             3SG.PRS                  tiazi   \n",
              "2   { a → NOM.SG(UNM)} { b → ACC.SG(UNM)} { c → NO...                 GU4.MA   \n",
              "3                                             QUANcar          morphological   \n",
              "4                                             3PL.PRS             tetraploid   \n",
              "..                                                ...                    ...   \n",
              "95                                           ACC.SG.C              amorphous   \n",
              "96                                        GEN.SG(UNM)                synapse   \n",
              "97                                            3PL.PRS                akuanzi   \n",
              "98                                       3PL.PRS.IMPF               alzianzi   \n",
              "99                                            3SG.PRS                   epzi   \n",
              "\n",
              "       instruct_model_glosses  \n",
              "0                       slick  \n",
              "1                     a slick  \n",
              "2                     a slick  \n",
              "3                       slick  \n",
              "4                     a slick  \n",
              "..                        ...  \n",
              "95                    a slick  \n",
              "96                    a smear  \n",
              "97                    a slick  \n",
              "98  a slender slender slender  \n",
              "99                    a slick  \n",
              "\n",
              "[100 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e0bb2115-cd66-4660-9b4b-90940ba8d697\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>human_baseline_glosses</th>\n",
              "      <th>original_model_glosses</th>\n",
              "      <th>instruct_model_glosses</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3PL.PRS</td>\n",
              "      <td>rhyming</td>\n",
              "      <td>slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3SG.PRS</td>\n",
              "      <td>tiazi</td>\n",
              "      <td>a slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>{ a → NOM.SG(UNM)} { b → ACC.SG(UNM)} { c → NO...</td>\n",
              "      <td>GU4.MA</td>\n",
              "      <td>a slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>QUANcar</td>\n",
              "      <td>morphological</td>\n",
              "      <td>slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>3PL.PRS</td>\n",
              "      <td>tetraploid</td>\n",
              "      <td>a slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>ACC.SG.C</td>\n",
              "      <td>amorphous</td>\n",
              "      <td>a slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>GEN.SG(UNM)</td>\n",
              "      <td>synapse</td>\n",
              "      <td>a smear</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>3PL.PRS</td>\n",
              "      <td>akuanzi</td>\n",
              "      <td>a slick</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>3PL.PRS.IMPF</td>\n",
              "      <td>alzianzi</td>\n",
              "      <td>a slender slender slender</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>3SG.PRS</td>\n",
              "      <td>epzi</td>\n",
              "      <td>a slick</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e0bb2115-cd66-4660-9b4b-90940ba8d697')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e0bb2115-cd66-4660-9b4b-90940ba8d697 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e0bb2115-cd66-4660-9b4b-90940ba8d697');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-199965b3-3f71-493d-a45d-df8b548e0856\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-199965b3-3f71-493d-a45d-df8b548e0856')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-199965b3-3f71-493d-a45d-df8b548e0856 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_a2f810ae-344c-4c60-8d7f-155d6a6ca3db\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_a2f810ae-344c-4c60-8d7f-155d6a6ca3db button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "df",
              "summary": "{\n  \"name\": \"df\",\n  \"rows\": 100,\n  \"fields\": [\n    {\n      \"column\": \"human_baseline_glosses\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 45,\n        \"samples\": [\n          \"ADV\",\n          \"ACC.PL.C\",\n          \"HURR.ABS.SG\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"original_model_glosses\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 62,\n        \"samples\": [\n          \"ad.kid\",\n          \"DUMUli\",\n          \"rhyming\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"instruct_model_glosses\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 13,\n        \"samples\": [\n          \"apex\",\n          \"a slender slender slender\",\n          \"slick\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "jZVPjJQbSf6P"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "ImVgwnPmSgLX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "h6g_7n1rSgeu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "9tu5q7W8D0sW"
      },
      "source": [
        "Evaluate the models computing ROUGE metrics. Notice the improvement in the results!"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "original_model_results = rouge.compute(\n",
        "    predictions=original_model_glosses,\n",
        "    references=human_baseline_glosses,\n",
        "    use_aggregator=True,\n",
        "    use_stemmer=True,\n",
        ")\n",
        "\n",
        "# Compute ROUGE scores for the instructed (fine-tuned) model\n",
        "instructed_model_results = rouge.compute(\n",
        "    predictions=instructed_model_glosses,\n",
        "    references=human_baseline_glosses,\n",
        "    use_aggregator=True,\n",
        "    use_stemmer=True,\n",
        ")\n",
        "\n",
        "# Display results\n",
        "print(\"ORIGINAL MODEL:\")\n",
        "print(original_model_results)\n",
        "\n",
        "print(\"\\nINSTRUCTED MODEL:\")\n",
        "print(instructed_model_results)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0XF5oxQqIaKP",
        "outputId": "54482acc-01f8-4a80-a811-4e754fdccf48"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ORIGINAL MODEL:\n",
            "{'rouge1': 0.0016666666666666668, 'rouge2': 0.0, 'rougeL': 0.0016666666666666668, 'rougeLsum': 0.0016666666666666668}\n",
            "\n",
            "INSTRUCTED MODEL:\n",
            "{'rouge1': 0.0003773584905660377, 'rouge2': 0.0, 'rougeL': 0.0003773584905660377, 'rougeLsum': 0.0003773584905660377}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "What Each Metric Represents\n",
        "\n",
        "\n",
        "**rouge1 (Unigram Overlap):**\n",
        "\n",
        "Measures the overlap of individual words (unigrams) between the predictions and the references.\n",
        "A score of 0.001666 for the original model and 0.000377 for the instructed model means there is very little overlap between the predicted and ground truth glosses.\n",
        "\n",
        "**rouge2 (Bigram Overlap):**\n",
        "\n",
        "Measures the overlap of two-word sequences (bigrams) between predictions and references.\n",
        "A score of 0.0 means there is no overlap of bigrams between the generated and expected glosses.\n",
        "\n",
        "**rougeL (Longest Common Subsequence):**\n",
        "\n",
        "Captures the longest sequence of tokens that appears in both the predictions and the references, preserving the order.\n",
        "Scores here are the same as rouge1, indicating minimal commonality in token sequences.\n",
        "\n",
        "**rougeLsum:**\n",
        "\n",
        "Similar to rougeL, often used for summarization tasks. In our case, it matches rougeL."
      ],
      "metadata": {
        "id": "RjJ0inB_OVyL"
      }
    }
  ],
  "metadata": {
    "availableInstances": [
      {
        "_defaultOrder": 0,
        "_isFastLaunch": true,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 4,
        "name": "ml.t3.medium",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 1,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.t3.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 2,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.t3.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 3,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.t3.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 4,
        "_isFastLaunch": true,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.m5.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 5,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.m5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 6,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.m5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 7,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.m5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 8,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.m5.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 9,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.m5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 10,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.m5.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 11,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.m5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 12,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.m5d.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 13,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.m5d.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 14,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.m5d.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 15,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.m5d.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 16,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.m5d.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 17,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.m5d.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 18,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.m5d.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 19,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.m5d.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 20,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": true,
        "memoryGiB": 0,
        "name": "ml.geospatial.interactive",
        "supportedImageNames": [
          "sagemaker-geospatial-v1-0"
        ],
        "vcpuNum": 0
      },
      {
        "_defaultOrder": 21,
        "_isFastLaunch": true,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 4,
        "name": "ml.c5.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 22,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.c5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 23,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.c5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 24,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.c5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 25,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 72,
        "name": "ml.c5.9xlarge",
        "vcpuNum": 36
      },
      {
        "_defaultOrder": 26,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 96,
        "name": "ml.c5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 27,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 144,
        "name": "ml.c5.18xlarge",
        "vcpuNum": 72
      },
      {
        "_defaultOrder": 28,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.c5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 29,
        "_isFastLaunch": true,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.g4dn.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 30,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.g4dn.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 31,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.g4dn.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 32,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.g4dn.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 33,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.g4dn.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 34,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.g4dn.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 35,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 61,
        "name": "ml.p3.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 36,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 244,
        "name": "ml.p3.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 37,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 488,
        "name": "ml.p3.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 38,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 768,
        "name": "ml.p3dn.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 39,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.r5.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 40,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.r5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 41,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.r5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 42,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.r5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 43,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.r5.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 44,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.r5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 45,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 512,
        "name": "ml.r5.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 46,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 768,
        "name": "ml.r5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 47,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.g5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 48,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.g5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 49,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.g5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 50,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.g5.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 51,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.g5.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 52,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.g5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 53,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.g5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 54,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 768,
        "name": "ml.g5.48xlarge",
        "vcpuNum": 192
      },
      {
        "_defaultOrder": 55,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 1152,
        "name": "ml.p4d.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 56,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 1152,
        "name": "ml.p4de.24xlarge",
        "vcpuNum": 96
      }
    ],
    "colab": {
      "provenance": [],
      "gpuType": "A100",
      "machine_shape": "hm",
      "toc_visible": true,
      "include_colab_link": true
    },
    "instance_type": "ml.m5.2xlarge",
    "kernelspec": {
      "display_name": "Python 3 (Data Science)",
      "language": "python",
      "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.10"
    },
    "vscode": {
      "interpreter": {
        "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
      }
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "85343a327039490e8a4dfc931ba27210": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_619c7c94327c4fb3899498dbd654862d",
              "IPY_MODEL_adc89256ae3d44458cf98515a8e413e3",
              "IPY_MODEL_535ec9ebb3754e278546335c49c17406"
            ],
            "layout": "IPY_MODEL_8aa393ed9b0d43e0aea24a7d7e52ae2d"
          }
        },
        "619c7c94327c4fb3899498dbd654862d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5da9690b29ea487d8cab2dd45117dc29",
            "placeholder": "​",
            "style": "IPY_MODEL_21e08da3dc2d495ca41abbf4413f5414",
            "value": "Map:  99%"
          }
        },
        "adc89256ae3d44458cf98515a8e413e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9cd711d37fc2499bbd50bcbabd1b8196",
            "max": 136396,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_df0c02b481db40259246baefe8536c10",
            "value": 136396
          }
        },
        "535ec9ebb3754e278546335c49c17406": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aad629ff351c434ea8b0627d47bdb120",
            "placeholder": "​",
            "style": "IPY_MODEL_2b4d6ee2d6bf4ff8932dae38299d6871",
            "value": " 135630/136396 [00:12&lt;00:00, 11060.76 examples/s]"
          }
        },
        "8aa393ed9b0d43e0aea24a7d7e52ae2d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "5da9690b29ea487d8cab2dd45117dc29": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "21e08da3dc2d495ca41abbf4413f5414": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9cd711d37fc2499bbd50bcbabd1b8196": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df0c02b481db40259246baefe8536c10": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "aad629ff351c434ea8b0627d47bdb120": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2b4d6ee2d6bf4ff8932dae38299d6871": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "099dc5fd592b4c289060721896c6bf61": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2ef3ba1dd2574a9b917f4c0256d5a0b6",
              "IPY_MODEL_793f2955d77741a9873e4a4d376428ba",
              "IPY_MODEL_d5625d14a26745c3843519056dad8a08"
            ],
            "layout": "IPY_MODEL_6199935de5634f4a87bac34bfac08388"
          }
        },
        "2ef3ba1dd2574a9b917f4c0256d5a0b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cc066d810e434a06b74a6de54a2a9a44",
            "placeholder": "​",
            "style": "IPY_MODEL_b861e15f47f34b3e91fe9e729b3dcd5c",
            "value": "Map:  96%"
          }
        },
        "793f2955d77741a9873e4a4d376428ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_718153c15e404d6891ee871b74f128d2",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_8f35636f5ce745d185366a73f6da83b5",
            "value": 17050
          }
        },
        "d5625d14a26745c3843519056dad8a08": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_64fe9f8dc5d84c0d9e58b8ea232ff408",
            "placeholder": "​",
            "style": "IPY_MODEL_abe0904f731b45fc8d1305e003b54dc3",
            "value": " 16292/17050 [00:01&lt;00:00, 11002.93 examples/s]"
          }
        },
        "6199935de5634f4a87bac34bfac08388": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "cc066d810e434a06b74a6de54a2a9a44": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b861e15f47f34b3e91fe9e729b3dcd5c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "718153c15e404d6891ee871b74f128d2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8f35636f5ce745d185366a73f6da83b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "64fe9f8dc5d84c0d9e58b8ea232ff408": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "abe0904f731b45fc8d1305e003b54dc3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8ff136703e194f3e83e42bb1a0edbcd2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ec75e58b1cff476db99e547122a7be6d",
              "IPY_MODEL_8aa72a4eb6374e788553dac08f8932b5",
              "IPY_MODEL_cecb5ac41c504c0695c7fea2d6505d54"
            ],
            "layout": "IPY_MODEL_3731fa6330c447f58c398071ae3c6a77"
          }
        },
        "ec75e58b1cff476db99e547122a7be6d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ec38bf039a24767becfc6f4d5c2e2ba",
            "placeholder": "​",
            "style": "IPY_MODEL_fe077ff5d7b2455495e6a47caa57f76c",
            "value": "Map:  99%"
          }
        },
        "8aa72a4eb6374e788553dac08f8932b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_09767fff72f74c629828990ebeb18bcc",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7c8aba747e754ed2b4cb7a5bb7b3cf1f",
            "value": 17050
          }
        },
        "cecb5ac41c504c0695c7fea2d6505d54": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_16aca6ce4fd44d3aa573f782994c1941",
            "placeholder": "​",
            "style": "IPY_MODEL_6e09467fe54c4918bf420c6ebc7f51fa",
            "value": " 16855/17050 [00:01&lt;00:00, 10873.93 examples/s]"
          }
        },
        "3731fa6330c447f58c398071ae3c6a77": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "1ec38bf039a24767becfc6f4d5c2e2ba": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fe077ff5d7b2455495e6a47caa57f76c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "09767fff72f74c629828990ebeb18bcc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7c8aba747e754ed2b4cb7a5bb7b3cf1f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "16aca6ce4fd44d3aa573f782994c1941": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e09467fe54c4918bf420c6ebc7f51fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "58a3e6a8f9a14013962f0fe9035bdab1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0e53fe88a9684dabb39f9241c8dc85a5",
              "IPY_MODEL_8babf151548545a8a27c9dd16eeb6fe8",
              "IPY_MODEL_a90d9bb0a09641479ac07e5671eddda5"
            ],
            "layout": "IPY_MODEL_7339a840fd8f4437a7cb8438670412cd"
          }
        },
        "0e53fe88a9684dabb39f9241c8dc85a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0f4ce1d6d1f54a988129cf9777353b4c",
            "placeholder": "​",
            "style": "IPY_MODEL_84a81ade99a1414187a53e7e6ebea402",
            "value": "Map: 100%"
          }
        },
        "8babf151548545a8a27c9dd16eeb6fe8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_45ff813f6a224563a94e0c156b9706a5",
            "max": 136396,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_38b299f3bbe14019a9b3dc01b6428e47",
            "value": 136396
          }
        },
        "a90d9bb0a09641479ac07e5671eddda5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f96ce1b8f6de4f118e7d669564fc6996",
            "placeholder": "​",
            "style": "IPY_MODEL_bfff2e84853a469f844037dee8423eca",
            "value": " 136000/136396 [00:32&lt;00:00, 4432.01 examples/s]"
          }
        },
        "7339a840fd8f4437a7cb8438670412cd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "0f4ce1d6d1f54a988129cf9777353b4c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "84a81ade99a1414187a53e7e6ebea402": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "45ff813f6a224563a94e0c156b9706a5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "38b299f3bbe14019a9b3dc01b6428e47": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f96ce1b8f6de4f118e7d669564fc6996": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bfff2e84853a469f844037dee8423eca": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f8387dfc63d748fdb536503f189cd3da": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2acb23e8d66341a5b3578d546891c55d",
              "IPY_MODEL_1ddb8f95ea4647f18a81b0ef09204586",
              "IPY_MODEL_5c0648ed4ed244aa984c8ad08f3b9ecf"
            ],
            "layout": "IPY_MODEL_2ab7efa658ca49579da2dc8ffa7d3e47"
          }
        },
        "2acb23e8d66341a5b3578d546891c55d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ff76d5e486ee4cf3b9de86ef7fba6163",
            "placeholder": "​",
            "style": "IPY_MODEL_aa7ae91ee7524dea9de05cd4ffd95050",
            "value": "Map: 100%"
          }
        },
        "1ddb8f95ea4647f18a81b0ef09204586": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8fd3baef499f4ba5802e34b408ad44a4",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f55bb6e15ad34f45a268a166e0cd67ca",
            "value": 17050
          }
        },
        "5c0648ed4ed244aa984c8ad08f3b9ecf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cf4b3905a5a4408198db6249317fa42b",
            "placeholder": "​",
            "style": "IPY_MODEL_d791bd5dae6548249a14a9728215f9d0",
            "value": " 17000/17050 [00:04&lt;00:00, 4040.91 examples/s]"
          }
        },
        "2ab7efa658ca49579da2dc8ffa7d3e47": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "ff76d5e486ee4cf3b9de86ef7fba6163": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa7ae91ee7524dea9de05cd4ffd95050": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8fd3baef499f4ba5802e34b408ad44a4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f55bb6e15ad34f45a268a166e0cd67ca": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "cf4b3905a5a4408198db6249317fa42b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d791bd5dae6548249a14a9728215f9d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "51aaaf6b9f3c41a580612b41415709a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5ac399a1dcbb4056aac0f9161c899fa8",
              "IPY_MODEL_f48f1980db594a768b2fae012f36629c",
              "IPY_MODEL_5d302c50cb32495a80bfca58d095fbe2"
            ],
            "layout": "IPY_MODEL_9c44e38defc9448897ae942be0d229c3"
          }
        },
        "5ac399a1dcbb4056aac0f9161c899fa8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6b1b40797e044f9f90da701fc25fcd28",
            "placeholder": "​",
            "style": "IPY_MODEL_253b87dd09df4d079d7f2dcef826ba6f",
            "value": "Map: 100%"
          }
        },
        "f48f1980db594a768b2fae012f36629c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_97ce3d634a2447efbc83a171c4f89470",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_229f08b181bc4c76ba94c50e4806c7d8",
            "value": 17050
          }
        },
        "5d302c50cb32495a80bfca58d095fbe2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d17ba9a7e1c34ea68e698cd2124959a3",
            "placeholder": "​",
            "style": "IPY_MODEL_769d8ae6ed1a4b2c92b21b32db4cfe56",
            "value": " 17000/17050 [00:03&lt;00:00, 4616.26 examples/s]"
          }
        },
        "9c44e38defc9448897ae942be0d229c3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "6b1b40797e044f9f90da701fc25fcd28": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "253b87dd09df4d079d7f2dcef826ba6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "97ce3d634a2447efbc83a171c4f89470": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "229f08b181bc4c76ba94c50e4806c7d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d17ba9a7e1c34ea68e698cd2124959a3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "769d8ae6ed1a4b2c92b21b32db4cfe56": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a9efd95a6d51480eb89b16a88b8247dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_15616e3c4c17420e98bcb95f4e3796ec",
              "IPY_MODEL_5bc42b1f68ee4231bec1d71449605f34",
              "IPY_MODEL_704cc69c6aef41ff9c22af4a226382ed"
            ],
            "layout": "IPY_MODEL_e13200ef28ad4c07a7a3e86b100d29f6"
          }
        },
        "15616e3c4c17420e98bcb95f4e3796ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1360b14c41b940c299c1606ea5bd1dd8",
            "placeholder": "​",
            "style": "IPY_MODEL_36259a4ae78e41d797f42d2217645d82",
            "value": "Downloading builder script: 100%"
          }
        },
        "5bc42b1f68ee4231bec1d71449605f34": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4082ae3cd97c4c4dbdd8e3afdd537e65",
            "max": 6270,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1b6b235871f2409d98af2f9731234331",
            "value": 6270
          }
        },
        "704cc69c6aef41ff9c22af4a226382ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_935e2157b5a74490a736a560c825321e",
            "placeholder": "​",
            "style": "IPY_MODEL_f0ff46ceeb8c4531be96e0bf7029bfe2",
            "value": " 6.27k/6.27k [00:00&lt;00:00, 492kB/s]"
          }
        },
        "e13200ef28ad4c07a7a3e86b100d29f6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1360b14c41b940c299c1606ea5bd1dd8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "36259a4ae78e41d797f42d2217645d82": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4082ae3cd97c4c4dbdd8e3afdd537e65": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b6b235871f2409d98af2f9731234331": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "935e2157b5a74490a736a560c825321e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f0ff46ceeb8c4531be96e0bf7029bfe2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}