fine_tune_ai_model_translation.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DigitalPasts/LLM-fine-tuning/blob/main/fine_tune_ai_model_translation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "OMAQZiNND0sG"
      },
      "source": [
        "# Fine-Tune Large Language Models for Hittite Translation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k9lGh_J0D0sH"
      },
      "source": [
        "In this notebook, we explore the fine-tuning of the [GermanT5/german-t5-oscar-ep1-prompted-germanquad](https://https://huggingface.co/GermanT5/german-t5-oscar-ep1-prompted-germanquad) for the task of Hittite translation, focusing on its adaptability to low-resource ancient languages."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d8qHP2YJD0sI"
      },
      "source": [
        "# Table of Contents"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "Xcn798btD0sI"
      },
      "source": [
        "- [ 1 - Set up Kernel, Load Required Dependencies, Dataset and LLM](#1)\n",
        "  - [ 1.1 - Set up Kernel and Required Dependencies](#1.1)\n",
        "  - [ 1.2 - Load Dataset and LLM](#1.2)\n",
        "  - [ 1.3 - Test the Model with Zero Shot Inferencing](#1.3)\n",
        "- [ 2 - Perform Full Fine-Tuning](#2)\n",
        "  - [ 2.1 - Preprocess the Dialog-Summary Dataset](#2.1)\n",
        "  - [ 2.2 - Fine-Tune the Model with the Preprocessed Dataset](#2.2)\n",
        "  - [ 2.3 - Evaluate the Model Qualitatively (Human Evaluation)](#2.3)\n",
        "  - [ 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)](#2.4)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vZXbV0hgD0sJ"
      },
      "source": [
        "<a name='1'></a>\n",
        "## 1 - Set up Kernel, Load Required Dependencies, Dataset and LLM"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eG6rUp0aD0sK"
      },
      "source": [
        "<a name='1.1'></a>\n",
        "### 1.1 - Set up Kernel and Required Dependencies"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "pKKSAMwRD0sL"
      },
      "source": [
        "Now install the required packages for the LLM and datasets.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tPEBI3qID0sL",
        "outputId": "08b2c959-70ae-4fb4-c64d-f39a538da98a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: pip in /usr/local/lib/python3.10/dist-packages (24.1.2)\n",
            "Collecting pip\n",
            "  Downloading pip-24.3.1-py3-none-any.whl.metadata (3.7 kB)\n",
            "Downloading pip-24.3.1-py3-none-any.whl (1.8 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: pip\n",
            "  Attempting uninstall: pip\n",
            "    Found existing installation: pip 24.1.2\n",
            "    Uninstalling pip-24.1.2:\n",
            "      Successfully uninstalled pip-24.1.2\n",
            "Successfully installed pip-24.3.1\n",
            "Collecting torch==1.13.1\n",
            "  Downloading torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl.metadata (24 kB)\n",
            "Collecting datasets==2.11.0\n",
            "  Downloading datasets-2.11.0-py3-none-any.whl.metadata (20 kB)\n",
            "Collecting evaluate==0.4.0\n",
            "  Downloading evaluate-0.4.0-py3-none-any.whl.metadata (9.4 kB)\n",
            "Collecting transformers==4.27.2\n",
            "  Downloading transformers-4.27.2-py3-none-any.whl.metadata (106 kB)\n",
            "Collecting rouge_score==0.1.2\n",
            "  Downloading rouge_score-0.1.2.tar.gz (17 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting loralib==0.1.1\n",
            "  Downloading loralib-0.1.1-py3-none-any.whl.metadata (14 kB)\n",
            "Collecting peft==0.3.0\n",
            "  Downloading peft-0.3.0-py3-none-any.whl.metadata (21 kB)\n",
            "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (0.2.0)\n",
            "Requirement already satisfied: openai in /usr/local/lib/python3.10/dist-packages (1.54.4)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (2.2.2)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (1.26.4)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (3.8.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (4.66.6)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==1.13.1) (4.12.2)\n",
            "Collecting nvidia-cuda-runtime-cu11==11.7.99 (from torch==1.13.1)\n",
            "  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cudnn-cu11==8.5.0.96 (from torch==1.13.1)\n",
            "  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cublas-cu11==11.10.3.66 (from torch==1.13.1)\n",
            "  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch==1.13.1)\n",
            "  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (17.0.0)\n",
            "Collecting dill<0.3.7,>=0.3.0 (from datasets==2.11.0)\n",
            "  Downloading dill-0.3.6-py3-none-any.whl.metadata (9.8 kB)\n",
            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (2.32.3)\n",
            "Collecting xxhash (from datasets==2.11.0)\n",
            "  Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
            "Collecting multiprocess (from datasets==2.11.0)\n",
            "  Downloading multiprocess-0.70.17-py310-none-any.whl.metadata (7.2 kB)\n",
            "Requirement already satisfied: fsspec>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]>=2021.11.1->datasets==2.11.0) (2024.10.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (3.11.2)\n",
            "Requirement already satisfied: huggingface-hub<1.0.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (0.26.2)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (24.2)\n",
            "Collecting responses<0.19 (from datasets==2.11.0)\n",
            "  Downloading responses-0.18.0-py3-none-any.whl.metadata (29 kB)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets==2.11.0) (6.0.2)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.27.2) (3.16.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.27.2) (2024.9.11)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.27.2)\n",
            "  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from rouge_score==0.1.2) (1.4.0)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from rouge_score==0.1.2) (3.9.1)\n",
            "Requirement already satisfied: six>=1.14.0 in /usr/local/lib/python3.10/dist-packages (from rouge_score==0.1.2) (1.16.0)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft==0.3.0) (5.9.5)\n",
            "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (from peft==0.3.0) (1.1.1)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==1.13.1) (75.1.0)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==1.13.1) (0.45.0)\n",
            "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from openai) (1.9.0)\n",
            "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai) (0.27.2)\n",
            "Requirement already satisfied: jiter<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from openai) (0.7.1)\n",
            "Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai) (2.9.2)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai) (1.3.1)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas) (2024.2)\n",
            "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas) (2024.2)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (1.3.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (4.55.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (1.4.7)\n",
            "Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (11.0.0)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib) (3.2.0)\n",
            "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (3.10)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (1.2.2)\n",
            "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (2.4.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (24.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (1.5.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (6.1.0)\n",
            "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (0.2.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (1.17.2)\n",
            "Requirement already satisfied: async-timeout<6.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.11.0) (4.0.3)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (2024.8.30)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (1.0.7)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai) (0.14.0)\n",
            "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai) (2.23.4)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets==2.11.0) (3.4.0)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets==2.11.0) (2.2.3)\n",
            "Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.10/dist-packages (from accelerate->peft==0.3.0) (0.4.5)\n",
            "INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.\n",
            "Collecting multiprocess (from datasets==2.11.0)\n",
            "  Downloading multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)\n",
            "  Downloading multiprocess-0.70.15-py310-none-any.whl.metadata (7.2 kB)\n",
            "  Downloading multiprocess-0.70.14-py310-none-any.whl.metadata (6.6 kB)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->rouge_score==0.1.2) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->rouge_score==0.1.2) (1.4.2)\n",
            "Downloading torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl (887.5 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m887.5/887.5 MB\u001b[0m \u001b[31m24.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading datasets-2.11.0-py3-none-any.whl (468 kB)\n",
            "Downloading evaluate-0.4.0-py3-none-any.whl (81 kB)\n",
            "Downloading transformers-4.27.2-py3-none-any.whl (6.8 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.8/6.8 MB\u001b[0m \u001b[31m100.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading loralib-0.1.1-py3-none-any.whl (8.8 kB)\n",
            "Downloading peft-0.3.0-py3-none-any.whl (56 kB)\n",
            "Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m317.1/317.1 MB\u001b[0m \u001b[31m68.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.0/21.0 MB\u001b[0m \u001b[31m69.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m849.3/849.3 kB\u001b[0m \u001b[31m46.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m557.1/557.1 MB\u001b[0m \u001b[31m40.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading dill-0.3.6-py3-none-any.whl (110 kB)\n",
            "Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n",
            "Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m126.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading multiprocess-0.70.14-py310-none-any.whl (134 kB)\n",
            "Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)\n",
            "Building wheels for collected packages: rouge_score\n",
            "  Building wheel for rouge_score (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for rouge_score: filename=rouge_score-0.1.2-py3-none-any.whl size=24935 sha256=8fd91d2ebccbdcf7e4d05ec1aee8de4cbc18f72a59c0f8e9f227438b08c95b20\n",
            "  Stored in directory: /root/.cache/pip/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4\n",
            "Successfully built rouge_score\n",
            "Installing collected packages: tokenizers, xxhash, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cublas-cu11, loralib, dill, rouge_score, responses, nvidia-cudnn-cu11, multiprocess, transformers, torch, peft, datasets, evaluate\n",
            "  Attempting uninstall: tokenizers\n",
            "    Found existing installation: tokenizers 0.20.3\n",
            "    Uninstalling tokenizers-0.20.3:\n",
            "      Successfully uninstalled tokenizers-0.20.3\n",
            "  Attempting uninstall: transformers\n",
            "    Found existing installation: transformers 4.46.2\n",
            "    Uninstalling transformers-4.46.2:\n",
            "      Successfully uninstalled transformers-4.46.2\n",
            "  Attempting uninstall: torch\n",
            "    Found existing installation: torch 2.5.1+cu121\n",
            "    Uninstalling torch-2.5.1+cu121:\n",
            "      Successfully uninstalled torch-2.5.1+cu121\n",
            "  Attempting uninstall: peft\n",
            "    Found existing installation: peft 0.13.2\n",
            "    Uninstalling peft-0.13.2:\n",
            "      Successfully uninstalled peft-0.13.2\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "sentence-transformers 3.2.1 requires transformers<5.0.0,>=4.41.0, but you have transformers 4.27.2 which is incompatible.\n",
            "torchaudio 2.5.1+cu121 requires torch==2.5.1, but you have torch 1.13.1 which is incompatible.\n",
            "torchvision 0.20.1+cu121 requires torch==2.5.1, but you have torch 1.13.1 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed datasets-2.11.0 dill-0.3.6 evaluate-0.4.0 loralib-0.1.1 multiprocess-0.70.14 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 peft-0.3.0 responses-0.18.0 rouge_score-0.1.2 tokenizers-0.13.3 torch-1.13.1 transformers-4.27.2 xxhash-3.5.0\n"
          ]
        }
      ],
      "source": [
        "%pip install --upgrade pip\n",
        "%pip install \\\n",
        "    torch==1.13.1 \\\n",
        "    datasets==2.11.0 \\\n",
        "    evaluate==0.4.0 \\\n",
        "    transformers==4.27.2 \\\n",
        "    rouge_score==0.1.2 \\\n",
        "    loralib==0.1.1 \\\n",
        "    peft==0.3.0 \\\n",
        "    sentencepiece \\\n",
        "    openai \\\n",
        "    pandas \\\n",
        "    numpy \\\n",
        "    matplotlib \\\n",
        "    tqdm \\\n",
        "    evaluate\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "hY9Q1EkhD0sM"
      },
      "source": [
        "Import the necessary components.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Core Libraries\n",
        "import torch  # PyTorch for model training\n",
        "from transformers import T5Tokenizer, T5ForConditionalGeneration, AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig\n",
        "from transformers import Trainer, TrainingArguments\n",
        "from datasets import Dataset, load_metric\n",
        "\n",
        "# Utility Libraries\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import os\n",
        "import random\n",
        "import time\n",
        "import evaluate\n"
      ],
      "metadata": {
        "id": "LP79n0tKSjfp"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "MTpj8xMlD0sN"
      },
      "source": [
        "<a name='1.2'></a>\n",
        "### 1.2 - Load Dataset and LLM."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data = pd.read_csv(\"hittite_ds.csv\", index_col = 0)\n",
        "data.columns = ['txtid','lnr','cth','word', 'text', 'gloss','trans']\n",
        "data.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "NtMBUURyTGNI",
        "outputId": "75e9add8-54bf-472d-9487-b17c37915413"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        txtid     lnr  cth      word             text            gloss  \\\n",
              "0  IBoT 1.30+   Vs. 1  821   LUGALuš       ⸢LUGAL⸣-uš  FNL(u).NOM.SG.C   \n",
              "1  IBoT 1.30+   Vs. 1  821     kuapi         ku-wa-pí              CNJ   \n",
              "2  IBoT 1.30+   Vs. 1  821  DINGIRaš   DINGIR{MEŠ}-aš           D/L.PL   \n",
              "3  IBoT 1.30+   Vs. 1  821   aruaizi  a-ru-wa-a-ez-zi          3SG.PRS   \n",
              "4  IBoT 1.30+   Vs. 1  821    GUDU₁₂       {LÚ}GUDU₁₂      NOM.SG(UNM)   \n",
              "\n",
              "            trans  \n",
              "0           König  \n",
              "1      sobald als  \n",
              "2        Gottheit  \n",
              "3  sich verneigen  \n",
              "4       Gesalbter  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-064c06af-b013-44f9-adb7-11f37fae5ac8\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>txtid</th>\n",
              "      <th>lnr</th>\n",
              "      <th>cth</th>\n",
              "      <th>word</th>\n",
              "      <th>text</th>\n",
              "      <th>gloss</th>\n",
              "      <th>trans</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>LUGALuš</td>\n",
              "      <td>⸢LUGAL⸣-uš</td>\n",
              "      <td>FNL(u).NOM.SG.C</td>\n",
              "      <td>König</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>kuapi</td>\n",
              "      <td>ku-wa-pí</td>\n",
              "      <td>CNJ</td>\n",
              "      <td>sobald als</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>DINGIRaš</td>\n",
              "      <td>DINGIR{MEŠ}-aš</td>\n",
              "      <td>D/L.PL</td>\n",
              "      <td>Gottheit</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>aruaizi</td>\n",
              "      <td>a-ru-wa-a-ez-zi</td>\n",
              "      <td>3SG.PRS</td>\n",
              "      <td>sich verneigen</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>IBoT 1.30+</td>\n",
              "      <td>Vs. 1</td>\n",
              "      <td>821</td>\n",
              "      <td>GUDU₁₂</td>\n",
              "      <td>{LÚ}GUDU₁₂</td>\n",
              "      <td>NOM.SG(UNM)</td>\n",
              "      <td>Gesalbter</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-064c06af-b013-44f9-adb7-11f37fae5ac8')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-064c06af-b013-44f9-adb7-11f37fae5ac8 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-064c06af-b013-44f9-adb7-11f37fae5ac8');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b3915f76-32fd-41d3-b3bc-8cfec67b55bc\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b3915f76-32fd-41d3-b3bc-8cfec67b55bc')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b3915f76-32fd-41d3-b3bc-8cfec67b55bc button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "422gwbGSTJsI",
        "outputId": "6207ae78-1651-4b6d-c679-5642426fa301"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Index: 170496 entries, 0 to 170495\n",
            "Data columns (total 7 columns):\n",
            " #   Column  Non-Null Count   Dtype \n",
            "---  ------  --------------   ----- \n",
            " 0   txtid   170496 non-null  object\n",
            " 1   lnr     170496 non-null  object\n",
            " 2   cth     170496 non-null  int64 \n",
            " 3   word    170496 non-null  object\n",
            " 4   text    170496 non-null  object\n",
            " 5   gloss   170469 non-null  object\n",
            " 6   trans   170496 non-null  object\n",
            "dtypes: int64(1), object(6)\n",
            "memory usage: 10.4+ MB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "hf_dataset = Dataset.from_pandas(data[['word','trans']])\n",
        "hf_dataset = hf_dataset.remove_columns(\"__index_level_0__\") if \"__index_level_0__\" in hf_dataset.column_names else hf_dataset\n",
        "\n",
        "hf_dataset"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "to2B9e5NTsPF",
        "outputId": "cfc947cd-3cf1-4217-a70a-c318fad5eb3c"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['word', 'trans'],\n",
              "    num_rows: 170496\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Split into train, validation, and test sets\n",
        "splits = hf_dataset.train_test_split(test_size=0.2, seed=43)  # 80% train, 20% test\n",
        "train_dataset = splits[\"train\"]\n",
        "test_dataset = splits[\"test\"]\n",
        "\n",
        "# Further split test set into validation and test\n",
        "val_test_splits = test_dataset.train_test_split(test_size=0.5, seed=43)  # 50/50 split\n",
        "val_dataset = val_test_splits[\"train\"]\n",
        "test_dataset = val_test_splits[\"test\"]"
      ],
      "metadata": {
        "id": "HCnnSYqrW2Lf"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "lsBukLzfVM6E"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "u7gZdCO6D0sN"
      },
      "source": [
        "Load the pre-trained [GermanT5 model](https://huggingface.co/GermanT5/german-t5-oscar-ep1-prompted-germanquad) and its tokenizer directly from HuggingFace."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "10MH2Cf8D0sN",
        "outputId": "a3dfac58-b395-4837-b489-7cd2a6b7c4e6"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:797: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        }
      ],
      "source": [
        "model_name='GermanT5/german-t5-oscar-ep1-prompted-germanquad'\n",
        "\n",
        "original_model = AutoModelForSeq2SeqLM.from_pretrained(model_name)#, torch_dtype=torch.bfloat16)\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_name)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "# original_model = original_model.to(device)\n"
      ],
      "metadata": {
        "id": "_LjCtm0ZP-P9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#original_model.config"
      ],
      "metadata": {
        "id": "bWIKs9neQHjC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# test_input = \"Provide the gloss for the word: LUGALuš\"\n",
        "# tokens = tokenizer(test_input, return_tensors=\"pt\")\n",
        "# print(tokens)"
      ],
      "metadata": {
        "id": "zf0c-tlPQ5Ts"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "vdrIk4NbD0sN"
      },
      "source": [
        "It is possible to pull out the number of model parameters and find out how many of them are trainable. The following function can be used to do that, at this stage, you do not need to go into details of it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "gc98NsgvD0sN",
        "outputId": "89b4d986-c20c-4e9b-ff40-9cb8a8fa7358"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'trainable model parameters: 247539456, all model parameters: 247539456, percentage of trainable model parameters: 100.00%'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "def print_number_of_trainable_model_parameters(model):\n",
        "    trainable_model_params = 0\n",
        "    all_model_params = 0\n",
        "    for _, param in model.named_parameters():\n",
        "        all_model_params += param.numel()\n",
        "        if param.requires_grad:\n",
        "            trainable_model_params += param.numel()\n",
        "    return f\"trainable model parameters: {trainable_model_params}, all model parameters: {all_model_params}, percentage of trainable model parameters: {100 * trainable_model_params / all_model_params:.2f}%\"\n",
        "\n",
        "print_number_of_trainable_model_parameters(original_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "eu8OiYJGD0sN"
      },
      "source": [
        "<a name='1.3'></a>\n",
        "### 1.3 - Test the Model with Zero Shot Inferencing\n",
        "\n",
        "Test the model with the zero shot inferencing."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "random_indices = random.sample(range(len(test_dataset)), 5)\n",
        "for idx in random_indices:\n",
        "    hittite_word = test_dataset[idx]['word']\n",
        "    expected_gloss = test_dataset[idx]['trans']\n",
        "    prompt = f\"Geben Sie die Übersetzung für das folgende hethitische Wort \\n\\n{hittite_word} an:\\n\\n die Übersetzung:\"\n",
        "    inputs = tokenizer(prompt, return_tensors=\"pt\")\n",
        "    output = tokenizer.decode(\n",
        "        original_model.generate(inputs[\"input_ids\"], max_new_tokens=50)[0],\n",
        "        skip_special_tokens=True,\n",
        "    )\n",
        "    print(f\"Input: {hittite_word}\")\n",
        "    print(f\"Expected translation: {expected_gloss}\")\n",
        "    print(f\"Generated translation: {output}\\n\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DUQpJ6BRTxNb",
        "outputId": "e3f113d2-5c95-4fa5-fd61-831759dcce1f"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input: anaḫi\n",
            "Expected translation: Kostprobe (einer Opfergabe)\n",
            "Generated translation: ''anaḫi''\n",
            "\n",
            "Input: ḫazi\n",
            "Expected translation: Ḫazzi\n",
            "Generated translation: ''subsidia''\n",
            "\n",
            "Input: ME-ŠE-DI\n",
            "Expected translation: Leibwächter\n",
            "Generated translation: ''ME-ŠE-DI'' („Ägyptisch-Arabisch“)\n",
            "\n",
            "Input: kuliuišna\n",
            "Expected translation: Kuliwišna\n",
            "Generated translation: kategorische\n",
            "\n",
            "Input: UŠ-KÉ-EN\n",
            "Expected translation: sich niederwerfen\n",
            "Generated translation: ''UŠ-KÉ-EN''\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "mD76Ut9KD0sO"
      },
      "source": [
        "<a name='2'></a>\n",
        "## 2 - Perform Full Fine-Tuning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "fYkZr5wSD0sO"
      },
      "source": [
        "<a name='2.1'></a>\n",
        "### 2.1 - Preprocess the Dataset\n",
        "\n",
        "Training Prompt (Hittite Word or Phrase):\n",
        "Prepend the instruction \"Provide the translation for the following Hittite word\": before the input word or phrase\n",
        "\n",
        "Example:\n",
        "\n",
        "\n",
        "```\n",
        "Provide the translation for the following Hittite word:\n",
        "\n",
        "LUGALuš\n",
        "\n",
        "Translation:\n",
        "```\n",
        "\n",
        "\n",
        "\n",
        "Training Response (Translation):\n",
        "The response should be the expected translation for the word.\n",
        "\n",
        "Example:\n",
        "\n",
        "\n",
        "\n",
        "```\n",
        "König\n",
        "```\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def preprocess_function(example):\n",
        "    # Ensure input and target are strings\n",
        "    return {\n",
        "        \"input_text\": f\"Geben Sie die Übersetzung für das folgende hethitische Wort {str(example['word'])} an \\n\\n die Übersetzung:\",\n",
        "        \"target_text\": str(example[\"trans\"])\n",
        "    }"
      ],
      "metadata": {
        "id": "D8bPsv3HnmjQ"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset_preprocessed = train_dataset.map(preprocess_function)\n",
        "val_dataset_preprocessed = val_dataset.map(preprocess_function)\n",
        "test_dataset_preprocessed = test_dataset.map(preprocess_function)\n",
        "train_dataset_preprocessed[378]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 106,
          "referenced_widgets": [
            "5202a690e9c64555b1dc53620a463f66",
            "8aa9ea31f82545c88a8d3cb3c0b5e82a",
            "93b5c2f0c7ba4b7faf9a3087fa2d3804",
            "e7ee07e27f6541a88aeb61b6b858a5a4",
            "26e563af940e4dd29f595c97e2fb87bd",
            "fdc5bd7e519a48deb4d471579e45c905",
            "177d16c747334385b0d7b24413c1422f",
            "a6141ea8e15a457f8c676859444133c8",
            "a74afbfe30944adcb378b05754936a6f",
            "1ee38124bf47411791263dc38373366b",
            "7f6bc1bbe1a342f2a6e9790cdf395973",
            "ec7b7623be804de688cc26de99ad5877",
            "9aa861c159fe400b912e847529c6543e",
            "6f6a516005724185a09f575787aa1cbb",
            "b9557b4f573b4e9680c4cc72a289b66e",
            "2b5d89d26e6145d59e91b939368899cf",
            "cf8a8b29acee4a9eb62894f10252b3a3",
            "26ec8bc0cb7e430dac352a439e33de0b",
            "5d3d12932089482196279ee5e014b842",
            "80c2c4b89405449cb77a5c44e9b905b1",
            "e51453f6830c4046b612e409ec91e44e",
            "6021dbf92e8c4c50897a07425c435d18",
            "ef8f592c16234070b29bc16a6cf8f8f6",
            "8f76a6e48277498da9bf648d33bf119d",
            "48c6e00047204867b255ec6f7b4139b8",
            "4830e7dc39f54ecf8ba0b6996d76f7de",
            "1cb28fbc189142d9aee4b950c2f45bed",
            "f2a080dbe15145dc95f9b69d6841a2c4",
            "3103dc8c869c4eee8e401ed30a3990a7",
            "9bb8ef1a14334fb9a1ace2cd108bc068",
            "556d6f8d733a4e65a1b90db82cdf959f",
            "dbd3e12f7e6f41729e55c8f7068c92fb",
            "0d3cfc2699f24ae29417603628aec3a7"
          ]
        },
        "id": "kM5KZvhdnpCv",
        "outputId": "398f3f53-06e5-4619-c8a9-06968f03eac3"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/136396 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "5202a690e9c64555b1dc53620a463f66"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ec7b7623be804de688cc26de99ad5877"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ef8f592c16234070b29bc16a6cf8f8f6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'word': 'AZU',\n",
              " 'trans': 'Opferschauer',\n",
              " 'input_text': 'Geben Sie die Übersetzung für das folgende hethitische Wort AZU an \\n\\n die Übersetzung:',\n",
              " 'target_text': 'Opferschauer'}"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Tokenize datasets\n",
        "def tokenize_function(example):\n",
        "    model_inputs = tokenizer(\n",
        "        example[\"input_text\"], max_length=512, padding=\"max_length\", truncation=True\n",
        "    )\n",
        "    labels = tokenizer(\n",
        "        example[\"target_text\"], max_length=128, padding=\"max_length\", truncation=True\n",
        "    )\n",
        "    model_inputs[\"labels\"] = labels[\"input_ids\"]\n",
        "    return model_inputs"
      ],
      "metadata": {
        "id": "QHFq197dYJYu"
      },
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset_tokenized = train_dataset_preprocessed.map(tokenize_function, batched=True)\n",
        "val_dataset_tokenized = val_dataset_preprocessed.map(tokenize_function, batched=True)\n",
        "test_dataset_tokenized = test_dataset_preprocessed.map(tokenize_function, batched=True)\n",
        "\n",
        "# Set dataset format for PyTorch\n",
        "train_dataset_tokenized.set_format(type=\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n",
        "val_dataset_tokenized.set_format(type=\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n",
        "test_dataset_tokenized.set_format(type=\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17,
          "referenced_widgets": [
            "972e378ed97d49a9989221f6c429dbbe",
            "9fb35413e7a64648b29758d60becc08b",
            "ff9e41d205d64e01934e6d4c78525455",
            "022c3f7b5f7f46408fc501707db210f2",
            "3c71fb9aefde4701bee340f72ba00302",
            "c3080fda3632465cadec459867a1fd89",
            "01caa8778bfd4e7b8d462f9d4fd62498",
            "5cf89819e6b84189a89dacee054d30c7",
            "c0cb9f47e61b4beaba8155ed3e90565b",
            "cb73053e523f405aaca5f012fa0d267c",
            "53e51805e688490d82640d6673e3a92d",
            "75f8b577a6ec4cbdaad6d500c906deb6",
            "27c6ab7be2a74101b80e997a5d533830",
            "c454e86bb56f4a6b8ba335f8f3ca4ea7",
            "00d6f76b766e410693f82773be9f9549",
            "16fb975e79d540ec94436a721ecfa277",
            "dcd1b96c5b0a4c9093a527f041c731c1",
            "89ded56ffb9c43ebbda18fbe6692859a",
            "59f0de9b49ea43eca23c0920a8fb001a",
            "9bacbcb53dc7440e81f80642539059ce",
            "c2a88daa38c447ee84412beb724bd815",
            "4cbcf87391cd42ff93947421f5d18c6e",
            "8e652bffa12b43c69d810de06e125b9a",
            "00bb2bba32954b8f9349283b21cd08e6",
            "10a2c69f01bf4bf9aaef57e6de2961bf",
            "b8a1acbd3ab9410e8fe21250851c7c53",
            "86beefc50f8649e291cc8766b1e25c24",
            "fd003205c9de417f8239c767e2ddea9a",
            "7a3afa8dc18c489683e4823fc5822282",
            "f4daf372271a407e9fd2d199d40fc9e5",
            "b9014acba588443880f03a660e70b6b5",
            "694d2bcb181d43b78612dbe4dbdeb929",
            "265c27b06c02484683677dc8fa2dd4e8"
          ]
        },
        "id": "Uq23c1WGn-M9",
        "outputId": "4f09c8c4-600d-4dc8-dbf0-653928086fc8"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/136396 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "972e378ed97d49a9989221f6c429dbbe"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "75f8b577a6ec4cbdaad6d500c906deb6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/17050 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8e652bffa12b43c69d810de06e125b9a"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "p9mfXe55D0sO"
      },
      "source": [
        "Check the shapes of all three parts of the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8vPK2-ZKD0sP",
        "outputId": "640a7bd6-7ca7-4fe0-c323-3ef3fcaa2b51"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shapes of the datasets:\n",
            "Training: (136396, 7)\n",
            "Validation: (17050, 7)\n",
            "Test: (17050, 7)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['word', 'trans', 'input_text', 'target_text', 'input_ids', 'attention_mask', 'labels'],\n",
              "    num_rows: 17050\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "print(f\"Shapes of the datasets:\")\n",
        "print(f\"Training: {train_dataset_tokenized.shape}\")\n",
        "print(f\"Validation: {val_dataset_tokenized.shape}\")\n",
        "print(f\"Test: {test_dataset_tokenized.shape}\")\n",
        "\n",
        "test_dataset_tokenized"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dGuGhqBSD0sP"
      },
      "source": [
        "The output dataset is ready for fine-tuning."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h3jWp7Z3D0sP"
      },
      "source": [
        "<a name='2.2'></a>\n",
        "### 2.2 - Fine-Tune the Model with the Preprocessed Dataset\n",
        "\n",
        "Now utilize the built-in Hugging Face `Trainer` class (see the documentation [here](https://huggingface.co/docs/transformers/main_classes/trainer)). Pass the preprocessed dataset with reference to the original model. Other training parameters are found experimentally and there is no need to go into details about those at the moment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "tags": [],
        "id": "ZFl8rAKLD0sP"
      },
      "outputs": [],
      "source": [
        "output_dir = f'./glossing-training-{str(int(time.time()))}'\n",
        "\n",
        "training_args = TrainingArguments(\n",
        "    output_dir=output_dir,\n",
        "    learning_rate=1e-5,\n",
        "    num_train_epochs=4,\n",
        "    weight_decay=0.01,\n",
        "    logging_steps=1,\n",
        "    #max_steps=1,\n",
        "    #per_device_train_batch_size=8,\n",
        "    evaluation_strategy=\"epoch\",\n",
        "    report_to=\"none\"\n",
        ")\n",
        "\n",
        "\n",
        "trainer = Trainer(\n",
        "    model=original_model,\n",
        "    args=training_args,\n",
        "    train_dataset=train_dataset_tokenized,\n",
        "    eval_dataset=val_dataset_tokenized\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "-yXarFyQD0sP"
      },
      "source": [
        "Start training process...\n",
        "\n",
        "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi4gUGxlYXNlIGJlIHBhdGllbnQuPC90ZXh0PgogICAgPHRleHQgeD0iMTAwIiB5PSI1NiIgZm9udC1mYW1pbHk9IkFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE0IiBmaWxsPSIjMzMzMzMzIj5Zb3UgY2FuIHNhZmVseSBpZ25vcmUgdGhlIHdhcm5pbmcgbWVzc2FnZXMuPC90ZXh0Pgo8L3N2Zz4K\" alt=\"Time alert open medium\"/>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 310
        },
        "id": "IJ4uG9VWD0sP",
        "outputId": "c1efcc82-2b04-42ab-ac06-aa666f7d6333"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='68200' max='68200' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [68200/68200 7:12:23, Epoch 4/4]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Epoch</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.002100</td>\n",
              "      <td>0.011685</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.029400</td>\n",
              "      <td>0.007539</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.034100</td>\n",
              "      <td>0.006213</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.002800</td>\n",
              "      <td>0.005919</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TrainOutput(global_step=68200, training_loss=0.027722070564675638, metrics={'train_runtime': 25944.7089, 'train_samples_per_second': 21.029, 'train_steps_per_second': 2.629, 'total_flos': 3.73560475524268e+17, 'train_loss': 0.027722070564675638, 'epoch': 4.0})"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "Zw82ScgED0sP"
      },
      "source": [
        "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "eY0ArOwaD0sQ"
      },
      "source": [
        "Create an instance of the `AutoModelForSeq2SeqLM` class for the instruct model:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Save the fine-tuned model\n",
        "trainer.save_model(output_dir)\n",
        "\n",
        "# Save the tokenizer\n",
        "tokenizer.save_pretrained(output_dir)\n",
        "\n",
        "print(f\"Model and tokenizer saved to {output_dir}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2NySqmEhDfs_",
        "outputId": "ec472db5-4a80-4d15-c27c-21cdcffc8e93"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model and tokenizer saved to ./glossing-training-1733253000\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "instructed_model = AutoModelForSeq2SeqLM.from_pretrained(output_dir)\n",
        "instructed_tokenizer = AutoTokenizer.from_pretrained(output_dir)"
      ],
      "metadata": {
        "id": "d_z8OWZ8D5sx"
      },
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GFoo2ypyD0sV"
      },
      "source": [
        "<a name='2.3'></a>\n",
        "### 2.3 - Evaluate the Model Qualitatively (Human Evaluation)\n",
        "\n",
        "As with many GenAI applications, a qualitative approach where you ask yourself the question \"Is my model behaving the way it is supposed to?\" is usually a good starting point. In the example below, you can see how the fine-tuned model is able to create a reasonable translations compared to the original inability to understand what is being asked of the model."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "random_indices = random.sample(range(len(test_dataset)), 5)"
      ],
      "metadata": {
        "id": "nVpkncUYhSoR"
      },
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for idx in random_indices:\n",
        "    hittite_word = test_dataset[idx]['word']\n",
        "    expected_gloss = test_dataset[idx]['trans']\n",
        "    prompt = f\"Provide the translation for the following Hittite word:\\n\\n{hittite_word}\\n\\nTranslation:\"\n",
        "    inputs = instructed_tokenizer(prompt, return_tensors=\"pt\")\n",
        "    output = instructed_tokenizer.decode(\n",
        "        instructed_model.generate(inputs[\"input_ids\"], max_new_tokens=50)[0],\n",
        "        skip_special_tokens=True,\n",
        "    )\n",
        "    print(f\"Input: {hittite_word}\")\n",
        "    print(f\"Expected translation: {expected_gloss}\")\n",
        "    print(f\"Generated translation: {output}\\n\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "At2J6Zp_DuCS",
        "outputId": "7617c79d-f77d-43dd-c0cb-74b0ec360d07"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input: ekuzi\n",
            "Expected translation: trinken\n",
            "Generated translation: (Gefäß)\n",
            "\n",
            "Input: mezulla\n",
            "Expected translation: Mez(z)ul(l)a\n",
            "Generated translation: Mez(z)ul(l)a\n",
            "\n",
            "Input: QA-TAM-MApat\n",
            "Expected translation: ebenso\n",
            "Generated translation: ebenso\n",
            "\n",
            "Input: pai\n",
            "Expected translation: geben\n",
            "Generated translation: (u.B.)\n",
            "\n",
            "Input: DINGIRnana\n",
            "Expected translation: Gottheit\n",
            "Generated translation: (Priesterin)\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PfuH5AKHD0sV"
      },
      "source": [
        "<a name='2.4'></a>\n",
        "### 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)\n",
        "\n",
        "The [ROUGE metric](https://en.wikipedia.org/wiki/ROUGE_(metric)) helps quantify the validity of summarizations produced by models. It compares summarizations to a \"baseline\" summary which is usually created by a human. While not perfect, it does indicate the overall increase in summarization effectiveness that we have accomplished by fine-tuning."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "tags": [],
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "1b3adf9bb9f24848b6d02414285c4095",
            "9fcdfeeed8bb46bfb47a2efae08d03c2",
            "7a75e256f56a44afad9ce283ae1db14c",
            "5e3f7ef0ffe34c7f963ca2c69c452c34",
            "92dbc63ecc334ac5850c07f76cd564ab",
            "2039a9558cbe4213beb79840817751ed",
            "1958295ae13d4af787474536c79a9a59",
            "d59bf5fd5b3f4620a8783c17c3effac0",
            "7d346d3ae648493cbdbf70f157575fa7",
            "f0b83a5b5a12457db29746806a0abc8c",
            "b5648e1b7b3c432092abc73ee1c5976f"
          ]
        },
        "id": "DR8Ds5qFD0sV",
        "outputId": "545a6796-c55b-4288-893e-e117f513b8f5"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/6.27k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1b3adf9bb9f24848b6d02414285c4095"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "rouge = evaluate.load('rouge')"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Move models to the appropriate device (GPU if available)\n",
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "hittite_words = test_dataset[10450:10550]['word']  # Hittite words/phrases\n",
        "human_baseline_tr = test_dataset[10450:10550]['trans']  # Expected glosses (human-provided)\n",
        "\n",
        "original_model = original_model.to(device)\n",
        "instructed_model = instructed_model.to(device)\n",
        "\n",
        "# Initialize lists to store model outputs\n",
        "original_model_tr = []\n",
        "instructed_model_tr = []\n",
        "\n",
        "# Iterate through the selected examples\n",
        "for hittite_word in hittite_words:\n",
        "    # Create the input prompt\n",
        "    prompt = f\"\"\"\n",
        "    Geben Sie die Übersetzung für das folgende hethitische Wort\n",
        "\n",
        "    {hittite_word} an\n",
        "\n",
        "    die Übersetzung:\n",
        "    \"\"\"\n",
        "    # Tokenize the input prompt\n",
        "    input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(device)  # Move input_ids to the same device as the model\n",
        "    input_ids_instructed = instructed_tokenizer(prompt, return_tensors=\"pt\").input_ids.to(device)\n",
        "\n",
        "    # Generate gloss using the pre-trained (original) model\n",
        "    original_model_outputs = original_model.generate(\n",
        "        input_ids=input_ids,\n",
        "        generation_config=GenerationConfig(max_new_tokens=50)\n",
        "    )\n",
        "    original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
        "    original_model_tr.append(original_model_text_output)\n",
        "\n",
        "    # Generate gloss using the fine-tuned (instructed) model\n",
        "    instructed_model_outputs = instructed_model.generate(\n",
        "        input_ids=input_ids_instructed,\n",
        "        generation_config=GenerationConfig(max_new_tokens=50)\n",
        "    )\n",
        "    instructed_model_text_output = instructed_tokenizer.decode(instructed_model_outputs[0], skip_special_tokens=True)\n",
        "    instructed_model_tr.append(instructed_model_text_output)\n",
        "\n",
        "# Combine results into a DataFrame for analysis\n",
        "zipped_glosses = list(zip(human_baseline_tr, original_model_tr, instructed_model_tr))\n",
        "df = pd.DataFrame(zipped_glosses, columns=['human_baseline_tr', 'original_model_tr', 'instruct_model_tr'])\n",
        "\n",
        "# Display the DataFrame\n",
        "df\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "nd6sy7n4hcDo",
        "outputId": "0c277004-0533-493e-b872-34f245ad6a13"
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   human_baseline_tr                                  original_model_tr  \\\n",
              "0             Großer  Geben Sie die erste von 100 Sanskrit-Übersetzu...   \n",
              "1           Brotlaib                                         ninda.gura   \n",
              "2        Totentempel                                              šišta   \n",
              "3               Berg                                       nihilistisch   \n",
              "4        Linsensuppe                                        TU Chemnitz   \n",
              "..               ...                                                ...   \n",
              "95            ziehen  Geben Sie die erste von 100 Sanskrit-Übersetzu...   \n",
              "96              dann                                              namma   \n",
              "97        Wettergott                                             ŠKURna   \n",
              "98            Siegel                                              KIŠIB   \n",
              "99              Herd                                          ''GUNNI''   \n",
              "\n",
              "   instruct_model_tr  \n",
              "0             Großer  \n",
              "1           Brotlaib  \n",
              "2              Ḫišta  \n",
              "3               Papi  \n",
              "4     Saubohnensuppe  \n",
              "..               ...  \n",
              "95            ziehen  \n",
              "96              dann  \n",
              "97        Wettergott  \n",
              "98            Siegel  \n",
              "99              Herd  \n",
              "\n",
              "[100 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-f7026965-bb0c-48df-a14e-a1f752147123\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>human_baseline_tr</th>\n",
              "      <th>original_model_tr</th>\n",
              "      <th>instruct_model_tr</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Großer</td>\n",
              "      <td>Geben Sie die erste von 100 Sanskrit-Übersetzu...</td>\n",
              "      <td>Großer</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Brotlaib</td>\n",
              "      <td>ninda.gura</td>\n",
              "      <td>Brotlaib</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Totentempel</td>\n",
              "      <td>šišta</td>\n",
              "      <td>Ḫišta</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Berg</td>\n",
              "      <td>nihilistisch</td>\n",
              "      <td>Papi</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Linsensuppe</td>\n",
              "      <td>TU Chemnitz</td>\n",
              "      <td>Saubohnensuppe</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>ziehen</td>\n",
              "      <td>Geben Sie die erste von 100 Sanskrit-Übersetzu...</td>\n",
              "      <td>ziehen</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>dann</td>\n",
              "      <td>namma</td>\n",
              "      <td>dann</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>Wettergott</td>\n",
              "      <td>ŠKURna</td>\n",
              "      <td>Wettergott</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>Siegel</td>\n",
              "      <td>KIŠIB</td>\n",
              "      <td>Siegel</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>Herd</td>\n",
              "      <td>''GUNNI''</td>\n",
              "      <td>Herd</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f7026965-bb0c-48df-a14e-a1f752147123')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-f7026965-bb0c-48df-a14e-a1f752147123 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-f7026965-bb0c-48df-a14e-a1f752147123');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-78b69664-fb65-4d27-b1a8-3f7c44e0c97e\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-78b69664-fb65-4d27-b1a8-3f7c44e0c97e')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-78b69664-fb65-4d27-b1a8-3f7c44e0c97e button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_a0ffe72d-9398-4b0f-b53e-517d3bee5864\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_a0ffe72d-9398-4b0f-b53e-517d3bee5864 button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "df",
              "summary": "{\n  \"name\": \"df\",\n  \"rows\": 100,\n  \"fields\": [\n    {\n      \"column\": \"human_baseline_tr\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 79,\n        \"samples\": [\n          \"libieren\",\n          \"Gro\\u00dfer\",\n          \"hinschaffen\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"original_model_tr\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 84,\n        \"samples\": [\n          \"GE\\u0160TIN\",\n          \"Geben Sie die erste von 100 Sanskrit-\\u00dcbersetzungen f\\u00fcr ''GAL'' ein, um ''GAL'' zu vermeiden.\",\n          \"m\\u00e2n an \\u00e2n\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"instruct_model_tr\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 77,\n        \"samples\": [\n          \"Saubohnensuppe\",\n          \"gehen\",\n          \"dreimal\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "tags": [],
        "id": "LaqFGg5OD0sW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9bb42000-45fd-49fc-bd14-3e49367eae54"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ORIGINAL MODEL:\n",
            "{'rouge1': 0.025555555555555557, 'rouge2': 0.02, 'rougeL': 0.025555555555555557, 'rougeLsum': 0.026666666666666665}\n",
            "\n",
            "INSTRUCTED MODEL:\n",
            "{'rouge1': 0.895, 'rouge2': 0.27, 'rougeL': 0.895, 'rougeLsum': 0.895}\n"
          ]
        }
      ],
      "source": [
        "original_model_results = rouge.compute(\n",
        "    predictions=original_model_tr,\n",
        "    references=human_baseline_tr,\n",
        "    use_aggregator=True,\n",
        "    use_stemmer=True,\n",
        ")\n",
        "\n",
        "# Compute ROUGE scores for the instructed (fine-tuned) model\n",
        "instructed_model_results = rouge.compute(\n",
        "    predictions=instructed_model_tr,\n",
        "    references=human_baseline_tr,\n",
        "    use_aggregator=True,\n",
        "    use_stemmer=True,\n",
        ")\n",
        "\n",
        "# Display results\n",
        "print(\"ORIGINAL MODEL:\")\n",
        "print(original_model_results)\n",
        "\n",
        "print(\"\\nINSTRUCTED MODEL:\")\n",
        "print(instructed_model_results)"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "zA78QXmmjdaw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The results indicate that the **fine-tuned (instructed) model** performs significantly better than the original pre-trained model on the task of Hittite translation:\n",
        "\n",
        "**Original Model**: Low ROUGE-1 (2.56%) and ROUGE-2 (2%) scores, showing minimal overlap with reference translations, which reflects its lack of task-specific adaptation.\n",
        "\n",
        "**Instructed Model**: High ROUGE-1 (89.5%) and ROUGE-L (89.5%) scores, indicating strong alignment with the reference translations. ROUGE-2 (27%) suggests there is still room for improvement in capturing multi-word patterns.\n",
        "\n",
        "***Conclusion: Fine-tuning improves the model's performance substantially, adapting it effectively for Hittite translation tasks.***"
      ],
      "metadata": {
        "id": "fZh4Q8szjevs"
      }
    }
  ],
  "metadata": {
    "availableInstances": [
      {
        "_defaultOrder": 0,
        "_isFastLaunch": true,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 4,
        "name": "ml.t3.medium",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 1,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.t3.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 2,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.t3.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 3,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.t3.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 4,
        "_isFastLaunch": true,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.m5.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 5,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.m5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 6,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.m5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 7,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.m5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 8,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.m5.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 9,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.m5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 10,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.m5.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 11,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.m5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 12,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.m5d.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 13,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.m5d.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 14,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.m5d.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 15,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.m5d.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 16,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.m5d.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 17,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.m5d.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 18,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.m5d.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 19,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.m5d.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 20,
        "_isFastLaunch": false,
        "category": "General purpose",
        "gpuNum": 0,
        "hideHardwareSpecs": true,
        "memoryGiB": 0,
        "name": "ml.geospatial.interactive",
        "supportedImageNames": [
          "sagemaker-geospatial-v1-0"
        ],
        "vcpuNum": 0
      },
      {
        "_defaultOrder": 21,
        "_isFastLaunch": true,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 4,
        "name": "ml.c5.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 22,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 8,
        "name": "ml.c5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 23,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.c5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 24,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.c5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 25,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 72,
        "name": "ml.c5.9xlarge",
        "vcpuNum": 36
      },
      {
        "_defaultOrder": 26,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 96,
        "name": "ml.c5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 27,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 144,
        "name": "ml.c5.18xlarge",
        "vcpuNum": 72
      },
      {
        "_defaultOrder": 28,
        "_isFastLaunch": false,
        "category": "Compute optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.c5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 29,
        "_isFastLaunch": true,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.g4dn.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 30,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.g4dn.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 31,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.g4dn.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 32,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.g4dn.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 33,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.g4dn.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 34,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.g4dn.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 35,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 61,
        "name": "ml.p3.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 36,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 244,
        "name": "ml.p3.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 37,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 488,
        "name": "ml.p3.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 38,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 768,
        "name": "ml.p3dn.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 39,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.r5.large",
        "vcpuNum": 2
      },
      {
        "_defaultOrder": 40,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.r5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 41,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.r5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 42,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.r5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 43,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.r5.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 44,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.r5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 45,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 512,
        "name": "ml.r5.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 46,
        "_isFastLaunch": false,
        "category": "Memory Optimized",
        "gpuNum": 0,
        "hideHardwareSpecs": false,
        "memoryGiB": 768,
        "name": "ml.r5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 47,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 16,
        "name": "ml.g5.xlarge",
        "vcpuNum": 4
      },
      {
        "_defaultOrder": 48,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 32,
        "name": "ml.g5.2xlarge",
        "vcpuNum": 8
      },
      {
        "_defaultOrder": 49,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 64,
        "name": "ml.g5.4xlarge",
        "vcpuNum": 16
      },
      {
        "_defaultOrder": 50,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 128,
        "name": "ml.g5.8xlarge",
        "vcpuNum": 32
      },
      {
        "_defaultOrder": 51,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 1,
        "hideHardwareSpecs": false,
        "memoryGiB": 256,
        "name": "ml.g5.16xlarge",
        "vcpuNum": 64
      },
      {
        "_defaultOrder": 52,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 192,
        "name": "ml.g5.12xlarge",
        "vcpuNum": 48
      },
      {
        "_defaultOrder": 53,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 4,
        "hideHardwareSpecs": false,
        "memoryGiB": 384,
        "name": "ml.g5.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 54,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 768,
        "name": "ml.g5.48xlarge",
        "vcpuNum": 192
      },
      {
        "_defaultOrder": 55,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 1152,
        "name": "ml.p4d.24xlarge",
        "vcpuNum": 96
      },
      {
        "_defaultOrder": 56,
        "_isFastLaunch": false,
        "category": "Accelerated computing",
        "gpuNum": 8,
        "hideHardwareSpecs": false,
        "memoryGiB": 1152,
        "name": "ml.p4de.24xlarge",
        "vcpuNum": 96
      }
    ],
    "colab": {
      "provenance": [],
      "gpuType": "A100",
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "instance_type": "ml.m5.2xlarge",
    "kernelspec": {
      "display_name": "Python 3 (Data Science)",
      "language": "python",
      "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.10"
    },
    "vscode": {
      "interpreter": {
        "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
      }
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "5202a690e9c64555b1dc53620a463f66": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8aa9ea31f82545c88a8d3cb3c0b5e82a",
              "IPY_MODEL_93b5c2f0c7ba4b7faf9a3087fa2d3804",
              "IPY_MODEL_e7ee07e27f6541a88aeb61b6b858a5a4"
            ],
            "layout": "IPY_MODEL_26e563af940e4dd29f595c97e2fb87bd"
          }
        },
        "8aa9ea31f82545c88a8d3cb3c0b5e82a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fdc5bd7e519a48deb4d471579e45c905",
            "placeholder": "​",
            "style": "IPY_MODEL_177d16c747334385b0d7b24413c1422f",
            "value": "Map: 100%"
          }
        },
        "93b5c2f0c7ba4b7faf9a3087fa2d3804": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a6141ea8e15a457f8c676859444133c8",
            "max": 136396,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a74afbfe30944adcb378b05754936a6f",
            "value": 136396
          }
        },
        "e7ee07e27f6541a88aeb61b6b858a5a4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ee38124bf47411791263dc38373366b",
            "placeholder": "​",
            "style": "IPY_MODEL_7f6bc1bbe1a342f2a6e9790cdf395973",
            "value": " 136026/136396 [00:12&lt;00:00, 10109.85 examples/s]"
          }
        },
        "26e563af940e4dd29f595c97e2fb87bd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "fdc5bd7e519a48deb4d471579e45c905": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "177d16c747334385b0d7b24413c1422f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a6141ea8e15a457f8c676859444133c8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a74afbfe30944adcb378b05754936a6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1ee38124bf47411791263dc38373366b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f6bc1bbe1a342f2a6e9790cdf395973": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ec7b7623be804de688cc26de99ad5877": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9aa861c159fe400b912e847529c6543e",
              "IPY_MODEL_6f6a516005724185a09f575787aa1cbb",
              "IPY_MODEL_b9557b4f573b4e9680c4cc72a289b66e"
            ],
            "layout": "IPY_MODEL_2b5d89d26e6145d59e91b939368899cf"
          }
        },
        "9aa861c159fe400b912e847529c6543e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cf8a8b29acee4a9eb62894f10252b3a3",
            "placeholder": "​",
            "style": "IPY_MODEL_26ec8bc0cb7e430dac352a439e33de0b",
            "value": "Map:  94%"
          }
        },
        "6f6a516005724185a09f575787aa1cbb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5d3d12932089482196279ee5e014b842",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_80c2c4b89405449cb77a5c44e9b905b1",
            "value": 17050
          }
        },
        "b9557b4f573b4e9680c4cc72a289b66e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e51453f6830c4046b612e409ec91e44e",
            "placeholder": "​",
            "style": "IPY_MODEL_6021dbf92e8c4c50897a07425c435d18",
            "value": " 16006/17050 [00:01&lt;00:00, 10881.50 examples/s]"
          }
        },
        "2b5d89d26e6145d59e91b939368899cf": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "cf8a8b29acee4a9eb62894f10252b3a3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "26ec8bc0cb7e430dac352a439e33de0b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5d3d12932089482196279ee5e014b842": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "80c2c4b89405449cb77a5c44e9b905b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e51453f6830c4046b612e409ec91e44e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6021dbf92e8c4c50897a07425c435d18": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ef8f592c16234070b29bc16a6cf8f8f6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8f76a6e48277498da9bf648d33bf119d",
              "IPY_MODEL_48c6e00047204867b255ec6f7b4139b8",
              "IPY_MODEL_4830e7dc39f54ecf8ba0b6996d76f7de"
            ],
            "layout": "IPY_MODEL_1cb28fbc189142d9aee4b950c2f45bed"
          }
        },
        "8f76a6e48277498da9bf648d33bf119d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f2a080dbe15145dc95f9b69d6841a2c4",
            "placeholder": "​",
            "style": "IPY_MODEL_3103dc8c869c4eee8e401ed30a3990a7",
            "value": "Map:  98%"
          }
        },
        "48c6e00047204867b255ec6f7b4139b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9bb8ef1a14334fb9a1ace2cd108bc068",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_556d6f8d733a4e65a1b90db82cdf959f",
            "value": 17050
          }
        },
        "4830e7dc39f54ecf8ba0b6996d76f7de": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dbd3e12f7e6f41729e55c8f7068c92fb",
            "placeholder": "​",
            "style": "IPY_MODEL_0d3cfc2699f24ae29417603628aec3a7",
            "value": " 16658/17050 [00:01&lt;00:00, 10773.14 examples/s]"
          }
        },
        "1cb28fbc189142d9aee4b950c2f45bed": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "f2a080dbe15145dc95f9b69d6841a2c4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3103dc8c869c4eee8e401ed30a3990a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9bb8ef1a14334fb9a1ace2cd108bc068": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "556d6f8d733a4e65a1b90db82cdf959f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "dbd3e12f7e6f41729e55c8f7068c92fb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0d3cfc2699f24ae29417603628aec3a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "972e378ed97d49a9989221f6c429dbbe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9fb35413e7a64648b29758d60becc08b",
              "IPY_MODEL_ff9e41d205d64e01934e6d4c78525455",
              "IPY_MODEL_022c3f7b5f7f46408fc501707db210f2"
            ],
            "layout": "IPY_MODEL_3c71fb9aefde4701bee340f72ba00302"
          }
        },
        "9fb35413e7a64648b29758d60becc08b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c3080fda3632465cadec459867a1fd89",
            "placeholder": "​",
            "style": "IPY_MODEL_01caa8778bfd4e7b8d462f9d4fd62498",
            "value": "Map: 100%"
          }
        },
        "ff9e41d205d64e01934e6d4c78525455": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5cf89819e6b84189a89dacee054d30c7",
            "max": 136396,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c0cb9f47e61b4beaba8155ed3e90565b",
            "value": 136396
          }
        },
        "022c3f7b5f7f46408fc501707db210f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cb73053e523f405aaca5f012fa0d267c",
            "placeholder": "​",
            "style": "IPY_MODEL_53e51805e688490d82640d6673e3a92d",
            "value": " 136000/136396 [00:32&lt;00:00, 4465.78 examples/s]"
          }
        },
        "3c71fb9aefde4701bee340f72ba00302": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "c3080fda3632465cadec459867a1fd89": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "01caa8778bfd4e7b8d462f9d4fd62498": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5cf89819e6b84189a89dacee054d30c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c0cb9f47e61b4beaba8155ed3e90565b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "cb73053e523f405aaca5f012fa0d267c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53e51805e688490d82640d6673e3a92d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "75f8b577a6ec4cbdaad6d500c906deb6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_27c6ab7be2a74101b80e997a5d533830",
              "IPY_MODEL_c454e86bb56f4a6b8ba335f8f3ca4ea7",
              "IPY_MODEL_00d6f76b766e410693f82773be9f9549"
            ],
            "layout": "IPY_MODEL_16fb975e79d540ec94436a721ecfa277"
          }
        },
        "27c6ab7be2a74101b80e997a5d533830": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dcd1b96c5b0a4c9093a527f041c731c1",
            "placeholder": "​",
            "style": "IPY_MODEL_89ded56ffb9c43ebbda18fbe6692859a",
            "value": "Map: 100%"
          }
        },
        "c454e86bb56f4a6b8ba335f8f3ca4ea7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_59f0de9b49ea43eca23c0920a8fb001a",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9bacbcb53dc7440e81f80642539059ce",
            "value": 17050
          }
        },
        "00d6f76b766e410693f82773be9f9549": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c2a88daa38c447ee84412beb724bd815",
            "placeholder": "​",
            "style": "IPY_MODEL_4cbcf87391cd42ff93947421f5d18c6e",
            "value": " 17000/17050 [00:04&lt;00:00, 4447.39 examples/s]"
          }
        },
        "16fb975e79d540ec94436a721ecfa277": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "dcd1b96c5b0a4c9093a527f041c731c1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "89ded56ffb9c43ebbda18fbe6692859a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "59f0de9b49ea43eca23c0920a8fb001a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9bacbcb53dc7440e81f80642539059ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c2a88daa38c447ee84412beb724bd815": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4cbcf87391cd42ff93947421f5d18c6e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8e652bffa12b43c69d810de06e125b9a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_00bb2bba32954b8f9349283b21cd08e6",
              "IPY_MODEL_10a2c69f01bf4bf9aaef57e6de2961bf",
              "IPY_MODEL_b8a1acbd3ab9410e8fe21250851c7c53"
            ],
            "layout": "IPY_MODEL_86beefc50f8649e291cc8766b1e25c24"
          }
        },
        "00bb2bba32954b8f9349283b21cd08e6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fd003205c9de417f8239c767e2ddea9a",
            "placeholder": "​",
            "style": "IPY_MODEL_7a3afa8dc18c489683e4823fc5822282",
            "value": "Map: 100%"
          }
        },
        "10a2c69f01bf4bf9aaef57e6de2961bf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f4daf372271a407e9fd2d199d40fc9e5",
            "max": 17050,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b9014acba588443880f03a660e70b6b5",
            "value": 17050
          }
        },
        "b8a1acbd3ab9410e8fe21250851c7c53": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_694d2bcb181d43b78612dbe4dbdeb929",
            "placeholder": "​",
            "style": "IPY_MODEL_265c27b06c02484683677dc8fa2dd4e8",
            "value": " 17000/17050 [00:03&lt;00:00, 4399.57 examples/s]"
          }
        },
        "86beefc50f8649e291cc8766b1e25c24": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "fd003205c9de417f8239c767e2ddea9a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7a3afa8dc18c489683e4823fc5822282": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f4daf372271a407e9fd2d199d40fc9e5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b9014acba588443880f03a660e70b6b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "694d2bcb181d43b78612dbe4dbdeb929": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "265c27b06c02484683677dc8fa2dd4e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1b3adf9bb9f24848b6d02414285c4095": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9fcdfeeed8bb46bfb47a2efae08d03c2",
              "IPY_MODEL_7a75e256f56a44afad9ce283ae1db14c",
              "IPY_MODEL_5e3f7ef0ffe34c7f963ca2c69c452c34"
            ],
            "layout": "IPY_MODEL_92dbc63ecc334ac5850c07f76cd564ab"
          }
        },
        "9fcdfeeed8bb46bfb47a2efae08d03c2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2039a9558cbe4213beb79840817751ed",
            "placeholder": "​",
            "style": "IPY_MODEL_1958295ae13d4af787474536c79a9a59",
            "value": "Downloading builder script: 100%"
          }
        },
        "7a75e256f56a44afad9ce283ae1db14c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d59bf5fd5b3f4620a8783c17c3effac0",
            "max": 6270,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7d346d3ae648493cbdbf70f157575fa7",
            "value": 6270
          }
        },
        "5e3f7ef0ffe34c7f963ca2c69c452c34": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f0b83a5b5a12457db29746806a0abc8c",
            "placeholder": "​",
            "style": "IPY_MODEL_b5648e1b7b3c432092abc73ee1c5976f",
            "value": " 6.27k/6.27k [00:00&lt;00:00, 506kB/s]"
          }
        },
        "92dbc63ecc334ac5850c07f76cd564ab": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2039a9558cbe4213beb79840817751ed": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1958295ae13d4af787474536c79a9a59": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d59bf5fd5b3f4620a8783c17c3effac0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7d346d3ae648493cbdbf70f157575fa7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f0b83a5b5a12457db29746806a0abc8c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b5648e1b7b3c432092abc73ee1c5976f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}