8_nn_library_implementation.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPbXITsO7MSATop0I/fctq8",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dlsyscourse/lecture8/blob/main/8_nn_library_implementation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mpn1ti5Urdsv"
      },
      "source": [
        "# Lecture 8: Neural Network Library Implementation\n",
        "\n",
        "In this lecture, we will to walk through neural network library design.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qXysoqn-vZuF"
      },
      "source": [
        "## Prepare the codebase\n",
        "\n",
        "To get started, we can clone the related lecture8 repo from the github."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JjEIRTyr8ajf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6cda9acd-d646-4ed7-9996-943e4fda777b"
      },
      "source": [
        "# Code to set up the assignment\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')\n",
        "%cd /content/drive/MyDrive/\n",
        "!mkdir -p 10714f24\n",
        "%cd /content/drive/MyDrive/10714f24\n",
        "\n",
        "# NOTE: Run the following line\n",
        "# - uncomment the following line if you run this section for the first time\n",
        "# - comment and skip the following line when you run this section for a second time\n",
        "#   so you will have a local copy of lecture8 under 10714f24/lecture8 that you can\n",
        "#   continue to edit and play with\n",
        "!git clone https://github.com/dlsyscourse/lecture8\n",
        "%rm -rf /content/needle\n",
        "!ln -s /content/drive/MyDrive/10714f24/lecture8 /content/needle\n",
        "%cd /content/needle"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n",
            "/content/drive/MyDrive\n",
            "/content/drive/MyDrive/10714f24\n",
            "Cloning into 'lecture8'...\n",
            "remote: Enumerating objects: 59, done.\u001b[K\n",
            "remote: Counting objects: 100% (59/59), done.\u001b[K\n",
            "remote: Compressing objects: 100% (43/43), done.\u001b[K\n",
            "remote: Total 59 (delta 19), reused 46 (delta 11), pack-reused 0 (from 0)\u001b[K\n",
            "Receiving objects: 100% (59/59), 61.11 KiB | 463.00 KiB/s, done.\n",
            "Resolving deltas: 100% (19/19), done.\n",
            "/content/drive/MyDrive/10714f24/lecture8\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DFxG3p3S1sBq"
      },
      "source": [
        "We can then run the following command to make the path to the package available in colab's environment as well as the PYTHONPATH."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ADB_KdhcsQWI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4cbc3324-b8f6-4a2b-9267-286acded524e"
      },
      "source": [
        "%set_env PYTHONPATH /content/needle/python:/env/python\n",
        "import sys\n",
        "sys.path.append(\"/content/needle/python\")"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "env: PYTHONPATH=/content/needle/python:/env/python\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BBIuE2jc1DaU"
      },
      "source": [
        "## Needle Refresh\n",
        "\n",
        "We will continue to improve needle from lecture4 and hw1. Let us first begin\n",
        "by review a few more elements of needle API."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x1Z8wSsI6PrU"
      },
      "source": [
        "### Mutating the data field of a needle Tensor\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N2bm_WB9uF4V"
      },
      "source": [
        "import needle as ndl"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1h5iAYFfBRED"
      },
      "source": [
        "w = ndl.Tensor([1,2,3], dtype=\"float32\")"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D__fhTKpOYPq"
      },
      "source": [
        "g = ndl.Tensor([1,1,1], dtype=\"float32\")"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4-rdA-qP65lL"
      },
      "source": [
        "By default, we create needle Tensors that sets requires_grad to be true. This will cause us to record the gradient graph by default."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WBWEbJbh6nTE"
      },
      "source": [
        "w = ndl.Tensor([1, 2, 3], dtype=\"float32\")"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jtDGjL6p6re7",
        "outputId": "62c44c98-56cf-4412-e497-7683c0ca7c6c"
      },
      "source": [
        "w.requires_grad"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2J-NJfyV7WUD"
      },
      "source": [
        "Let us run a sgd style update for a few steps."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "56ks8V4p7hpw"
      },
      "source": [
        "grad = ndl.Tensor([1, 1, 1], dtype=\"float32\")\n",
        "lr = 0.1\n",
        "for i in range(5):\n",
        "    w = w + (-lr) * grad"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l8GrVvpp7qIo"
      },
      "source": [
        "The problem with is approach is that we are actively building up a computational graph that tracks the history of all the past updates.\n",
        "Such un-necessary graph tracking can cause memory and speed issues."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_ga-r2sG3304",
        "outputId": "3be2ccca-9094-4f2e-f4b0-f85468b4aabd"
      },
      "source": [
        "w.op"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<needle.ops.ops_mathematic.EWiseAdd at 0x7bcd44228100>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VtxblbMyA_xP",
        "outputId": "474431f4-7416-4add-84cf-6a1975160768"
      },
      "source": [
        "w.inputs[0].op"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<needle.ops.ops_mathematic.EWiseAdd at 0x7bcd4422af20>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l8rAzQshBCT1",
        "outputId": "16e23f4c-ef02-438c-9aad-64f80632c009"
      },
      "source": [
        "w.inputs[0].inputs[0].op"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<needle.ops.ops_mathematic.EWiseAdd at 0x7bcd4422bfa0>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d0x9xlNnBZJm"
      },
      "source": [
        "Instead, we can create a \"detached\" tensor that does not requires grad. When we are"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BJXLjMI0BVH6",
        "outputId": "870f862f-faee-4c93-903f-7f5d98b303ee"
      },
      "source": [
        "w.data"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "needle.Tensor([0.4999999 1.4999999 2.5000005])"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fesMkvKiBhqS",
        "outputId": "30849755-eba8-49ca-c0c0-ce2bb496b473"
      },
      "source": [
        "w.data.requires_grad"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZQkTHKnTBsmh"
      },
      "source": [
        "new_w = w.data + (-lr) * grad.data"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4nAm79poCT8F",
        "outputId": "93533bc4-6659-4d98-a80e-b76ad70cc43a"
      },
      "source": [
        "new_w"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "needle.Tensor([0.39999992 1.3999999  2.4000006 ])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cgrNrMpcCVph",
        "outputId": "51c82364-a7f2-41d6-b7d2-b8aac97217c5"
      },
      "source": [
        "new_w.requires_grad"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_HY-NyQcCX5J",
        "outputId": "bdda129e-6faa-463e-d468-42af1c7801ef"
      },
      "source": [
        "new_w.inputs"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[]"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ftV4Yrf13Vvm"
      },
      "source": [
        "We can also set the data field of w to \"update\" the weight array."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2P9oXtrZ3MJW"
      },
      "source": [
        "w.data = w.data + (-lr) * grad.data"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2Mg84Pux3hnz",
        "outputId": "066f6eec-b625-4e7a-d611-a735556b0331"
      },
      "source": [
        "w"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "needle.Tensor([0.39999992 1.3999999  2.4000006 ])"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ba_FOgoLBkIF"
      },
      "source": [
        "w.data shares the same underlying cached data object with w, but does not have the requires grad field.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t4o4kqmG76GI"
      },
      "source": [
        "### Numerical Stability\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kvFEh2O7hZqN"
      },
      "source": [
        "Most computations in a deep learning model are executed using 32-bit floating point. We need to pay special attention to potential numerical problems. Softmax is one of the most commonly used operators in loss functions.\n",
        "Let $z = softmax(x)$, then we have\n",
        "\n",
        "\\begin{equation}\n",
        " z_i = \\frac{exp(x_i)}{\\sum_k exp(x_k)}\n",
        "\\end{equation}\n",
        "\n",
        "If we naively follow the formula to compute softmax, the result can be inaccurate."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "myQIm65thYZj"
      },
      "source": [
        "import numpy as np"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KSQU-QMvilDJ",
        "outputId": "38321d32-3cd1-4a18-985d-b230a63dfcb1"
      },
      "source": [
        "x = np.array([100, 100, 101], dtype=\"float32\")\n",
        "\n",
        "def softmax_naive(x):\n",
        "    z = np.exp(x)\n",
        "    return z / np.sum(z)\n",
        "\n",
        "p = softmax_naive(x)\n",
        "p"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-21-35eddb9a8ee0>:4: RuntimeWarning: overflow encountered in exp\n",
            "  z = np.exp(x)\n",
            "<ipython-input-21-35eddb9a8ee0>:5: RuntimeWarning: invalid value encountered in divide\n",
            "  return z / np.sum(z)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([nan, nan, nan], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s0cmFrvbjJJ3"
      },
      "source": [
        "Passing a large number(that is greator than 0) to exp function can easily result in overflow. Note that the following invariance hold for any constant $c$\n",
        "\n",
        "\\begin{equation}\n",
        " z_i = \\frac{exp(x_i)}{\\sum_k exp(x_k)} = \\frac{exp(x_i-c)}{\\sum_k exp(x_k-c)}\n",
        "\\end{equation}\n",
        "\n",
        "We can pick $c = max(x)$ so that all the inputs to the exp become smaller or equal to 0\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HiOn_Cavi0ek",
        "outputId": "78886460-330b-4171-f9bf-8a855685d597"
      },
      "source": [
        "x = np.array([1000, 10000, 100], dtype=\"float32\")\n",
        "def softmax_stable(x):\n",
        "    x = x - np.max(x)\n",
        "    z = np.exp(x)\n",
        "    return z / np.sum(z)\n",
        "\n",
        "softmax_stable(x)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0., 1., 0.], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9fwMLzdQkMi7"
      },
      "source": [
        "Similar principles hold when we compute logsoftmax, or logsumexp operations."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h81327zK7djn"
      },
      "source": [
        "## Designing a Neural Network Library\n",
        "\n",
        "Now we are ready to design the interface of a high-level neural network library."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vvipv1Ki8Sdf"
      },
      "source": [
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABSQAAAHGCAYAAABtrIXnAAAgAElEQVR4AezdCfw97Vz/8bedkD1LRSoq/yyhFC0osiZkSRLSokKLotJClkgL+kv9U0hSlKRSUagslSVSdpElsofs+T/e93197vvzu+6ZOTNz5pzvLK95PH6/c86cmWuueV5zrvnOZ67rGokJAQQQQACBwwo8V9Kn+YcBxwDHQMcx8KbDVkOkjgACCCCAAAIIIIAAAggggAACWxIgGElAlmOAY6DPMbClepF9RQABBBBAAAEEEEAAAQQQQACBAwpEIOKAmyBpBBBYsAB1xIILj6wjgAACCCCAAAIIIIAAAgggMEcBgg1zLBXyhMB8BKgj5lMW5AQBBBBAAAEEEEAAAQQQQACBVQgQbFhFMbITCBxMgDriYLQkjAACCCCAAAIIIIAAAggggMA2BQg2bLPc2WsE+gpQR/SVYjkEEEAAAQQQQAABBBBAAAEEEOglQLChFxMLIbBZAeqIzRY9O44AAggggAACCCCAAAIIIIDAYQQINhzGlVQRWIsAdcRaSpL9QAABBBBAAAEEEEAAAQQQQGAmAgQbZlIQZAOBmQpQR8y0YMgWAggggAACCCCAAAIIIIAAAksVINiw1JIj3wgcR4A64jjObAUBBBBAAAEEEEAAAQQQQACBzQgQbNhMUbOjCIwSoI4YxcZKCCCAAAIIIIAAAggggAACCCDQJkCwoU2G+QggYAHqCI4DBBBAAAEEEEAAAQQQQAABBBCYVIBgw6ScJIbA6gSoI1ZXpOwQAggggAACCCCAAAIIIIAAAicrQLDhZP3ZOgJzF6COmHsJkT8EEEAAAQQQQAABBBBAAAEEFiZAsGFhBUZ2ETiyAHXEkcHZHAIIIIAAAggggAACCCCAAAJrFyDYsPYSZv8Q2E+AOmI/P9ZGAAEEEEAAAQQQQAABBBBAAIFKgGBDBcJHBBA4RYA64hQOPiCAAAIIIIAAAggggAACCCCAwL4CBBv2FWR9BNYtQB2x7vJl7xBAAAEEEEAAAQQQQAABBBA4ugDBhqOTs0EEFiVAHbGo4iKzCCCAAAIIIIAAAggggAACCMxfgGDD/MuIHCJwkgLUESepz7YRQAABBBBAAAEEEEAAAQQQWKEAwYYVFiq7hMCEAtQRE2KSFAIIIIAAAggggAACCCCAAAIISAQbOAoQQKBLgDqiS4fvEEAAAQQQQAABBBBAAAEEEEBgsADBhsFkrIDApgSoIzZV3OwsAggggAACCCCAAAIIIIAAAocXINhweGO2gMCSBagjllx65B0BBBBAAAEEEEAAAQQQQACBGQoQbJhhoZAlBGYkQB0xo8IgKwgggAACCCCAAAIIIIAAAgisQYBgwxpKkX1A4HAC1BGHsyVlBBBAAAEEEEAAAQQQQAABBDYpQLBhk8XOTiPQW4A6ojcVCyKAAAIIIIAAAggggAACCCCAQB8Bgg19lFgGge0KUEdst+zZcwQQQAABBBBAAAEEEEAAAQQOIkCw4SCsJIrAagSoI1ZTlOwIAggggAACCCCAAAIIIIAAAvMQINgwj3IgFwjMVYA6Yq4lQ74QQAABBBBAAAEEEEAAAQQQWKgAwYaFFhzZRuBIAtQRR4JmMwgggAACCCCAAAIIIIAAAghsRYBgw1ZKmv1EYJwAdcQ4N9ZCAAEEEEAAAQQQQAABBBBAAIEWAYINLTDMRgCB0wSoIzgQEEAAAQQQQAABBBBAAAEEEEBgUgGCDZNykhgCqxOgjlhdkbJDCCCAAAIIIIAAAggggAACCJysAMGGk/Vn6wjMXYA6Yu4lRP4QQAABBBBAAAEEEEAAAQQQWJgAwYbuAru+pO+TdKfuxXp9e42S1u17LX36Qt6ut/8FA9ZpWrRp203zmtY99ryp9vnY+V7r9qgj1lqy7BcCCCCAAAIIIIAAAggggAACJyRAsKEb/imSbPSW7sV6ffuTJa1/rZb+FknfK+ma1Xx//ERZ5zYN3w2Z1bTtpnlD0txn2WPs8z75Y90zBagjzrTgHQIIIIAAAggggAACCCCAAAIITCBAsKEb8RgByZeVoOP9G7Ky1oDkMfa5gZNZIwSoI0agsQoCCCCAAAIIIIAAAggggAACCLQLEGxot/E3UwYkv0jSHSXdqNpkV3DuDmWdy1TrDP3Y1BqyLT9D0x6z/DH2eUy+WOesAtQRZzVhDgIIIIAAAggggAACCCCAAAII7CFAsKEbb8qAZNuWuoJzbesMnd8UkByaxpTLH2Ofp8zvltOijthy6bPvCCCAAAIIIIAAAggggAACCBxAgGBDN2pTQPKikp5W/l1Y0i0k/a6kN0v6oKQXSfoBSWerkva4iV7vkWX+E8rn95cu268un5+a1vu9Mu/aaZ7fnkPSbSX9paQ3SvqwpHdJeomkn5Z0vmr5poBknR+vcs+yvdi/tlc/ECemIXnZZ5+9vVtLchqvlPTfkv5B0q9JulZkJr2OLaeUBG/Lsel6ggkBBBBAAAEEEEAAAQQQQAABBBCYRICAZDdjU0DS3afD7Sck/W/6HPP9+pgq6Too+O6W9T6e1msaQ9KBzie1rBvbf66ks6d06m37q6Z5j9+RbqR/05L20LyM3efzS3pcR97sdJ+0v347tpyqZDb/Mcp88xAAIIAAAggggAACCCCAAAIIIIDANAIEG7oddwUk7eeWkXeS5FaD95D0mhI4+6Sky6fk6wDgrSTdTVIE6Z5RPt81rdMUkPz2FJhza0qPS3klSV8n6dnpuyundOpt+6umeV9e8uB85X/3Tem+QtIFStpD8zJ2nx+Rtu8WqLeU9GWS7iLprem726d9zgHJIeWUkuBtsgUDAQQQQAABBBBAAAEEEEAAAQQQmESAgGQ3466A5DsluWtwnm6YgjjuFh1TUwDQ33WNp9gUkIyWgi+MhNPrRVKLze9I85u23TQvrXLG23NJcotLHyvuFv55Z3xzZqvFIXnx6kP2+QqS3GrU23+mJOcnTxeX9B/l+3+XdJ7yZQ5IDimnnDbvz2wNjAUCCCCAAAIIIIAAAggggAACCCAwiQAByW7GXQHJH29Y3V2lo9Wjx2SMqS0AOCQ457TcyvDekr4+Ek6vV0nB0O9K85u23TQvrXLG218vaToo+LVnzD39zZi8eM0h+xytMz8lyU8Gb5pyS023FPWUA5JDyqmszksRoI7gUEAAAQQQQAABBBBAAAEEEEAAgUkFCDZ0c+4KSN64ZXU/aMa2P5y+bwsADgnOpeROGyPyapLuLOmhpfVgtCT0tqcISLoLehwj3503Xr13ELZvXrzqkH3+rZKH11XbzB8vkfIZ+50DkkPKKafL+zPLHwsEEEAAAQQQQAABBBBAAAEEEEBgEoEINk2S2AoT2RWQbHq6sxncdfhQAUl3SX6ApPekIFyUYzyx258jMOf8NAVDm+blIryBJI+D6bR+NX+R3o/Ji1cfEpB8VsnDc9J2m976CefO60PKlzkgOaScmtLe8rw4trZswL4jgAACCCCAAAIIIIAAAggggMCEAgQbujHnGJB8YgpEeuxGd0e+eenOfI7UXXyfgOQVJb23bOevJZ2zhWlMXpzUkIBklMFLW/Lg2edLY2dGq1QCkh1gA76ijhiAxaIIIIAAAggggAACCCCAAAIIILBbgGBDt1EEw96SFhsb6GprkTgkOHfhFHjz2I5nS/nyW3ed/lAJJI4NSHobry5pvL7hoT2xybF58fpD9vlBJS8flnT+2Hj1eu2yjI/nm5bvxpZTlfTmP1JHbP4QAAABBBBAAAEEEEAAAQQQQACBaQUINnR7HjMg6W7Y9VQ/ZdsPlYkyc5fqenIwLr4fE5B0C8u/KGl8QNKV6g2kz2Pz4iQiINlnn2+U9qlpeacX3bo/IumSJY8EJFNh7fE2jqc9kmBVBBBAAAEEEEAAAQQQQAABBBBA4EwBgg1nWjS9O0ZA0t2uXQ5/JulzJV0sZaQOSH5+Cs49QZLHcPTkQOIdUndtp/dj5Tu/NLXObJr3Kyl9Pyzn4i3/LiBpbF6cnyH77OU9fqT3yWNa3i/tt62elPLs1pQxEZAMif1eqSP282NtBBBAAAEEEEAAAQQQQAABBBCoBAg2VCDVx2MEJH8nBdRcHn5Sdkx1QNLzX56W90NsXiPpo2Xef0t6W3nvdB5VEmoKPtbzLp3SjeOi7fXxJd0xefGqQ/f5SyS9KeXvY+Xz/6Z5fyfpgiVffiEgmTD2eBvHwB5JsCoCCCCAAAIIIIAAAggggAACCCBwpgDBhjMtmt5FQPKN6ctLpCDY1dP8/LbpKdtusWjvf84LSrqCpL+V5O7G/t5jJcYUAclvjhmSPk/Ss1MevI6XcxpfKOl70jiTv1/Wa9p2Pe9zqjTj2Gh6jYDkmLw4S0P32etcSNLj0sN2Il9vlfQzpZVoYtLYcspp8P70Y9LWTAgggAACCCCAAAIIIIAAAggggMAkAhHUmSQxEjmqwKUkfaWkq0k6d7Xly0pysPRc1fxDfTx2Xi4n6atLd/JD7RPpni5AHcGRgAACCCCAAAIIIIAAAggggAACkwoQbJiUk8QQWJ0AdcTqipQdQgABBBBAAAEEEEAAAQQQQOBkBQg2nKw/W0dg7gLUEXMvIfKHAAIIIIAAAggggAACCCCAwMIECDYsrMDILgJHFqCOODI4m0MAAQQQQAABBBBAAAEEEEBg7QIEG9ZewuwfAvsJUEfs58faCCCAAAIIIIAAAggggAACCCBQCRBsqED4iAACpwhQR5zCwQcEEEAAAQQQQAABBBBAAAEEENhXgGDDvoKsj8C6Bagj1l2+7B0CCCCAAAIIIIAAAggggAACRxcg2HB0cjaIwKIEqCMWVVxkFgEEEEAAAQQQQAABBBBAAIH5CxBsmH8ZkUMETlKAOuIk9dk2AggggAACCCCAAAIIIIAAAisUINiwwkJllxCYUIA6YkJMkkIAAQQQQAABBBBAAAEEEEAAAYlgA0cBAgh0CVBHdOnwHQIIIIAAAggggAACCCCAAAIIDBYg2DCYjBUQ2JQAdcSmipudRQABBBBAAAEEEEAAAQQQQODwAgQbDm/MFhBYsgB1xJJLj7wjgAACCCCAAAIIIIAAAgggMEMBgg0zLBSyhMCMBKgjZlQYZAUBBBBAAAEEEEAAAQQQQACBNQgQbFhDKbIPCBxOgDricLakjAACCCCAAAIIIIAAAggggMAmBQg2bLLY2WkEegtQR/SmYkEEEEAAAQQQQAABBBBAAAEEEOgjQLChjxLLILBdAeqI7ZY9e44AAggggAACCCCAAAIIIIDAQQQINhyElUQRWI0AdcRqipIdQQABBBBAAAEEEEAAAQQQQGAeAgQb5lEO5AKBuQpQR8y1ZMgXAggggAACCCCAAAIIIIAAAgsVINiw0IIj2wgcSYA64kjQbAYBBBBAAAEEEEAAAQQQQACBrQgQbNhKSbOfCIwToI4Y58ZaCCCAAAIIIIAAAggggAACCCDQIkCwoQWG2QggcJoAdQQHAgIIIIAAAggggAACCCCAAAIITCpAsGFSThJDYHUC1BGrK1J2CAEEEEAAAQQQQAABBBBAAIGTFSDYcLL+bB2BuQtQR8y9hMgfAggggAACCCCAAAIIIIAAAgsTINiwsAIjuwgcWYA64sjgbA4BBBBAAAEEEEAAAQQQQACBtQsQbFh7CbN/COwnQB2xnx9rI4AAAggggAACCCCAAAIIIIBAJUCwoQLhIwIInCJAHXEKBx8QQAABBBBAAAEEEEAAAQQQQGBfAYIN+wqyPgLrFqCOWHf5sncIIIAAAggggAACCCCAAAIIHF2AYMPRydkgAosSoI5YVHGRWQQQQAABBBBAAAEEEEAAAQTmL0CwYf5lRA4ROEkB6oiT1GfbCCCAAAIIIIAAAggggAACCKxQgGDDCguVXUJgQgHqiAkxSQoBBBBAAAEEEEAAAQQQQAABBKQINvCKBccAx0DXMUB9iQACCCCAAAIIIIAAAggggAACCEwi8DgRhOkKwvAdxwfHgPTcSWobEkEAAQQQQAABBBBAAAEEEEAAAQQQQKASyMG36is+IoAAAggggAACCCCAAAIIIIAAAggggAAC0woQkJzWk9QQQAABBBBAAAEEEEAAAQQQQAABBBBAoEOAgGQHDl8hgAACCCCAAAIIIIAAAggggAACCCCAwLQCBCSn9SQ1BBBAAAEEEEAAAQQQQAABBBBAAAEEEOgQICDZgcNXCCCAAAIIIIAAAggggAACCCCAAAIIIDCtAAHJaT1JDQEEEEAAAQQQQAABBBBAAAEEEEAAAQQ6BAhIduDwFQIIIIAAAggggAACCCCAAAIIIIAAAghMK0BAclpPUkMAAQQQQAABBBBAAAEEEEAAAQQQQACBDgECkh04fIUAAggggAACCCCAAAIIIIAAAggggAAC0woQkJzWk9QQQAABBBBAAAEEEEAAAQQQQAABBBBAoEOAgGQHDl8hgAACCCCAAAIIIIAAAggggAACCCCAwLQCBCSn9SQ1BBBAAAEEEEAAAQQQQAABBBBAAAEEEOgQICDZgcNXCCCAAAIIIIAAAggggAACCCCAAAIIIDCtAAHJaT1JDQEEEEAAAQQQQAABBBBAAAEEEEAAAQQ6BAhIduDwFQIIIIAAAggggAACCCCAAAIIIIAAAghMK0BAclpPUkMAAQQQQAABBBBAAAEEEEAAAQQQQACBDgECkh04fIUAAggggAACCCCAAAIIIIAAAggggAAC0woQkJzWk9QQQAABBBBAAAEEEEAAAQQQQAABBBBAoEOAgGQHDl8hgAACCCCAAAIIIIAAAggggAACCCCAwLQCBCSn9SQ1BBBAAAEEEEAAAQQQQAABBBBAAAEEEOgQICDZgcNXCCCAAAIIIIAAAggggAACCCCAAAIIIDCtAAHJaT1JDQEEEEAAAQQQQAABBBBAAAEEEEAAAQQ6BAhIduDwFQIIIIAAAggggAACCCCAAAIIIIAAAghMK0BAclpPUkMAAQQQQAABBBBAAAEEEEAAAQQQQACBDgECkh04fIUAAggggAACCCCAAAIIIIAAAggggAAC0woQkJzWk9QQQAABBBBAAAEEEEAAAQQQQAABBBBAoEOAgGQHDl8hgAACCCCAAAIIIIAAAggggAACCCCAwLQCBCSn9SQ1BBBAAAEEEEAAAQQQQAABBBBAAAEEEOgQICDZgcNXCCCAAAIIIIAAAggggAACCCCAAAIIIDCtAAHJaT1JDQEEEEAAAQQQQAABBBBAAAEEEEAAAQQ6BAhIduDwFQIIIIAAAggggAACCCCAAAIIIIAAAghMK0BAclpPUkMAAQQQQAABBBBAAAEEEEAAAQQQQACBDgECkh04fIUAAggggAACCCCAAAIIIIAAAggggAAC0woQkJzWk9QQQAABBBBAAAEEEEAAAQQQQAABBBBAoEOAgGQHDl8hgAACCCCAAAIIIIAAAggggAACCCCAwLQCBCSn9SQ1BBBAAAEEEEAAAQQQQAABBBBAAAEEEOgQICDZgcNXCCCAAAIIIIAAAggggAACCCCAAAIIIDCtAAHJaT1JDQEEEEAAAQQQQAABBBBAAAEEEEAAAQQ6BAhIduDwFQIIIIAAAggggAACCCCAAAIIIIAAAghMK0BAclpPUkMAAQQQQAABBBBAAAEEEEAAAQQQQACBDgECkh04fIUAAggggAACCCCAAAIIIIAAAggggAAC0woQkJzWk9QQQAABBBBAAAEEEEAAAQQQQAABBBBAoAhcS9ITqn85IFl/5+WZEEAAAQQQQAABBBBAAAEEEEAAAQQQQACBUQLnlfR2STkI2fb+bZK8PBMCCCCAAAIIIIAAAggggAACCCCAAAIIIDBa4J49A5JejgkBBBBAAAEEEEAAAQQQQAABBBBAAAEEENhLoE8rSbeipHXkXsysjAACCCCAAAIIIIAAAggggAACCCCAAAIhsKuVJK0jQ4pXBBBAAAEEEEAAAQQQQAABBBBAAAEEENhboKuVJK0j9+YlAQQQQAABBBBAAIEjCVxOkv9d4kjbYzMIIIAAAggggAACewi0tZKkdeQeqKyKAAIIIIAAAgggcDSBc6Sx0Z96tK2yIQQQQAABBBBAAIHRAk2tJGkdOZqTFRFAAAEEEEAAAQSOLEBA8sjgbA4BBBCYs8BlS5P58885k+QNAQROE6hbSdI6kgNjrgJnL+cWd8m66FwzueB8rd2X7nwLPjjJOgIIINAhQECyA4evEEAAga0JfKw0m3/K1nac/UVggQK5lSStIxdYgBvKsseF+nT59+sb2u9j7eqafblYPdZRxHYQQACB4wtQxx/fnC0igAACsxWIgOSfHyGHd5f0SEm/dIRtsQkE1ioQrSRpHbnWEl7Hfq05YHboEupzrlyzLxerhz7CSB8BBBA4OQHq+Intn5vuAMedYF7PvCuOBRYcA9KbJq53pkzumAHJZ5b68qNT7sAG0uI8Qz1KPcoxsOsYmNt5Zs0Bs0OfdvqcK9fsy8XqoY8w0kcAAQROToA6fmL7XX8g8j0XERwDHAM+BuY6EZCca8mcmS/qEOoQjgGOgT7HwJm1xsm/W3PA7NC6BCSl9+r0fw87NPaI9LlJSH3cpz5mmW0fJ3O7STiiqjvYKkMDkp8p6fqS7iPpvpK+XtKFRuTO69xUknshPFDSj0q6laQrDkxrqnQGbrZ98ahs2pfgGwQQ2LLA3OsIApLzPzrnfgzNX5AcIrBugTnWEQQkxx9zWw9Ijpc7zprxe+N12wEnyp/y33UMHKdGWt5W+gYkv0XSv0n634YeyZ73Kkm367H7nyfpsZI+3JBOlOEfSbrSjrSmSmfHZoZ/HTsxfE3WQACBLQjMvY4gIDn/o3Dux9D8BckhAusWmGMdQUBy/DFHQHK83THWnOPv7Rj7zTYQQKCfAHVEt9OugOQ5y/MGwjFePyTpgw1BxUdIOlfLJr9Q0luqdd4p6bWSnF6k7Vd//j8HTqcl+f1mx07slwprI4DAWgVOso64jKRbSrq/pB+TdFtJvkjM05CA5FVKej8o6cGS7i3pjgOauve5yMp58/t9t1mnt8TPJ3kMLdGLPCOwNYE51hFtAclzS7qJJJ9HHiLpx0sXqkufYKFdU9JdJP1sOa99raTz75GffbuX9TlXHtr3gqUrW5zv3bXtWyVdcqDLobu2HTr9pt2d4++tKZ/MQwCBkxGgjuh23xWQ/MsUKPy4pJ+S5PO01/O/a0j6SUlxDWvvZ0s6W8NmX53Selq5rsyL+Xz/t2mZ17Sc/6dKJ297svcccJNRkhACqxQ4iTrC42vUd4MiH668nyDpYkU7KvOup2x/k6QXpco60sqvz5F0nYYS9J0sN6uvm9vHPL8+o2G9fbbZkNyiZ4XzoneCzCOAwMEE5lhHNAXMvl/S2zvOJb8hyYGwPH2GJF+Q+Fzx7y0XHHn5eH/xcrHi9Z4k6QLpXOSbaZ5uLullLfn5pKSXSrp2WbbPyz7dy4aeK6fyrffLLVN+uaNr2yeSZ71u/rxP1zZfcMbfCE/Jiab3+6Sfkhn1do6/t1E7wkoIIHAQAeqIbtaugOSt0zn5vyXdoCMpjyv5/rR83X3789N3vtF39pa0ziPphWlZX4Pmaap0cpqTvueAm5STxBBYncAx64jzSfq9VKHGtt9TLujis19fL+kK6e5SW0DSLSvzen7/Dkm+g+R083cfkHTlqgTr5vB5+XjvYGae9t1mTmsN78NpDfvCPiCAwPQCc6wjcsDMgcbHVOeL/5H0n9U874cfBBA3zELqT9NyXRcnsbxfHfwMl+8qAcn47ICkW2Y66BXz/Po+Sc5Xnuebdm492TVN0b1s6LlySt/YN7fszC1TwsHn/E9VLv8kyUHfpmnfLnJdF6ve3r7pN+V5yLxwGbIOyyKAwHYEqCO6y7qtjve51NeX4Xe97mRO+/Zr0vK+ts1dt++avvPfAV3T16Vl64epTZVO1/b3+i7A9kqElRFAYLUCx6oj3Ez9+aky9YXV90n6oiLrVibXlfTwdBH29zsCkjdMFyEeCNhdv+sLRTebf3ra7hurkvxqSbcp/95WlnNrF7ckiX8+CcQ0xTYjrbW8HusYWosX+4HA1gTmWEfkgFm+eeXWir5x5QsSTxct56pore998ZAgeXJLxtjHJ+cvOt5Hq34HGB1ocwvJSCMH3V5ZBsV3CwhP7lLuC5xfTct7vR8q3ze95PTGdi8beq6c0tf75Japtggjj7F1+9RF29/7/PwvaRl3YbtUA8i+XdvaLlZjU/umH+mMfQ2jseuzHgIIrFuAOqK7fNvqeA8DFnZ1Y5WuFJ+V1rtzWvDb0/wHpflNb92ox8FQBzV/oVpgqnSqZKf7GGjTpUhKCCCwJoFj1REO+sW23iXpah2Id2hoMdnUQjJX8A4etk2+G+UTR2zfXamapj7jYk29zaZ8LG1euC4t3+QXAQSOIzDHOiIHzCJ/d+/gcMvHWM7dtBxAjMkXLzEMyUdLEDO+a3p16/9I64llgRyQjO8cSHSwsm36kZSOb/J5vMJ6mrJ7WaTd51w5pa+36zEiw+Ulkj4nMlO9emxNP400lq1bkkzRta3tYtVZmSL9apcGf4x9H7wiKyCAwCYEqCO6i7mtjvdwIWHXtzeEt+SWlLHeo9Km/YCamO+bhXdK3w15O1U6Q7Y5aNnYyUErsTACCGxG4Bh1hFtHviJVutfqoesuaJE3v9YBSTebjy5kz+uR3t1Ser5Aa5p2XWQdYptN+VjavCinpeWb/CKAwHEE5lhH1AEzt6TfNeUxnK5aLewHzsR+3qP6rv7o1vyx7DeUL+uA5Osk+Zyza/rFlJYH1s/T1N3LIu1d50ovN6Wvx8+KsT3dUtVdorsm97iIHg9u/eqWJTFN0bWt7WLV25gi/cjr2Nc4tsauz3oIILBuAeqI7vJtq+N9LWo7DxHi3gp9J5+LPb6x1/2rtJLHjPyLdA73975evo+kq/f8G8DJTZVOytq0bzngpvUkNQTWJnCMOu4zpmQAACAASURBVOJGqbL1k8L6TK68o8WJ81gHJL8kpVk3XW9K3wMLx77+TNMCknZdZB1imy1ZWdTscF1UpsksAggcTWCOdUQdMLtiD408fvBNq+U/V5IfNON9/efqu/qju1x5uTenQezrgGRXF+ycnlsKxoXOe1NXcy8zdfey2O6uc6WXm9I331B0ALbPlAOD35lWmKJrW9vFqjczRfopu6PezvH3NmpHWAkBBA4iQB3RzdpWx7+hnLv9ALuhk28yxnk/r3uREqSMMsmvbnjzN5J8w9MPZm176I3TmyqdnLfJ3sdOTZYgCSGAwKoEjlFH+E5PbOe2A/T8NO5Yrw5IOplLl25buetcW/I/mdJ6YMtCfS6ypt5mS1YWNTvKaFGZJrMIIHA0gTnWETlg5tZ0faZvS+eRpgHon5G+9/jFTZOfih0eP5cWyAFJtwK8cPpu19unpjQvnxaeuntZJN3nXDml7/9L++cbg30me8ZDgewT0xRd29ouVr2NKdKPvI59jeNr7PqshwAC6xagjugu37Y63kOy2M7PRBg6uTef1/WNy6bpxpKeknr/RRnl15dK8t8QXdNU6XRtY/B3sRODV2QFBBDYhMAx6ohfTxcTnz1A1YPoR/6aApJtSfkOklureOD/75D0m9WYlPsEJKfeZlt6S5ofZbSkPJNXBBA4nsAc64gcMHOArc/kh6bEvvihbPV0s/T9o+svy2fPjzS+IC2TA5JuQTlkcmvKSPPr04pTdy+LpIcGJPf1dQsR75/Hn/YQMH2naInqB+TFNEXXtraLVW9jivQjr2Nf41gYuz7rIYDAugWoI7rLt62OP0QLyTon5y3Xr+6R8ceSPpDO7y43jzXp1pK7pqnS2bWdXt9zwPViYiEENitwjDri2aUydSXa1dy8LoTLpkq4KyDpFgluhelWEL4A8XZiv5pepwhITrXNep+X+DmMl5h38owAAocXmGMdkQOSv9GTwOM9xr40BSR9ERNDjby/GrvQm/CYU/FE73r4khyQrL/blb384JrccvNQF09DA5L7+r4puYf/kNc6wLtv17a2i9Uop33Tj3TGvobN2PVZDwEE1i1AHdFdvm11/KFu8nXlxg9mvZWkf0nnQffqGHI97fSnSqcrr63fccC10vAFAgikyu2QGK8s2/GTSYdMHsg+6rCmgKQHqn9MWiaWjVffVfpHSb9VPaFzn4Dk1Nsc4jHXZcN7rvkjXwggcLICc6wjckDSrfj7TLsCkk7DYxTH/noMxzzdIn3nB7flKQck+zxgJ6+bx2n+6fTFobqXDQ1I7uvrLuxhOubV43A1TWO7trVdrNbbGJt+nc7Qz2E0dD2WRwCBbQhQR3SXc1sdP+UwKH742j3Lv2/szs5p315Q0svSufDKZZ2p0umRhfGLcMCNt2NNBLYgcIw64hAtJN0lO98t8n78naQHSPLDBjzWY57yuE5jA5KH2GbO41LfH+MYWqoN+UYAgTODSXOyOFRA0g+ZiYfbPKfa4Rjr0QEyByDzlAOSfpr3kCm3kPzutOJcWkjuG5D8j3IR9l+lpYhbiwz9l1jO8nZo17a2i9WzJFxmDE2/LZ2+8zkn95ViOQS2KUAd0V3ubXV8fmiZhxLpO/nJ2mEeNyP98NYPlvkv75nQD6Z07lzWmSqdnlkYt1js/Li1WQsBBNYucIw6wt21YjtTjSH52JSmA5M32FFQX56WHxuQPMQ2d2R7EV9H2S4is2QSAQSOLjDHOuJQAUnj/kk53/ihKjFOpB9SEy0WH9dQAjkg+faG77tmzX0MyX0DkvEwAAckjzHt6trWdrHaN2+70u+bTttyc/y9teWV+QggcHwB6ohu87Y63sG/16bryet2J3Pat/l5CL5J6Po/pr9Ifyu40cuu6UFp23m86KnS2bX90d9zwI2mY0UENiFwjDriEE/ZfkeplN8ryReWu6bvTZX42IDkIba5K99L+P4Yx9ASHMgjAgg0C8yxjjhkQDI/3CbONx7bMRyaLmJyQDIHMptFT50bLS+d/hyfsr1vQPK3k91lTt31zk/+2+OhkqIb+1Rd29ouVqdKv3OnenwZx1mPRVkEAQQ2KEAd0V3obXW818o9Ejw0WA4M1qleT5LHkw7v21UL/Hj6zkOMuTV923Q5SW8uy3+kWnaqdNq2vff8ANg7IRJAAIFVChyjjsjjW/UdrN93oeLhAM5jHkPSFXbk283g+0xukRLrxAVivV7XuFiH2madhyV+Dtcl5p08I4DA4QXmWEccMiDpi5noZvxWSf4crfze2PKk6ByQtNcjehaLu4h/opzffIPO24ppyu5lkaZfu86VsdyUvj+czt8/HxvY8ZqHaYkxOafq2tZ2sTpV+jt2befXc/y97cw0CyCAwNEEqCO6qdvq+Fgrd8H2g1R/UtLVy/nX636ZJAcJ8/jHf91w7nfPCT90LcrDz1xwwDO3ovxMSXeT5J4TsdzvRUbK61TpVMlO9zEyPl2KpIQAAmsSOEYdcTZJr0gV6Vf0APTYGJE3v+aApCv7/ynfv6pHWl9VPXl7V0DSF3feRp4Otc28jaW+j3Jaav7JNwIIHFZgjnXElAGzJr38cBs/kdutHu3g+U1THZD02FKXbFqwmvfwdK6MloCxyNTdyyLdCEg2nStjmSl9fbHllij28+tFYyMdr/km5Lek5abo2ua/B+KYduvUPE2Rfk5vzPvI25h1WQcBBNYvQB3RXcZddbzXdMDwl9J5IDx93vYDXONzvPoGYw4y5q374TQeVzqW9avHof53SW+qrl/9nc+/TS0pp0on522y97FzkyV4Agm564v/mKv/fackN4X13WEHPJY8XaPs3+1nvBOO9rsMbjnjPB4za2vxOFYdcZtU2XocqKt2FNa3VneVnMcckPSqHvQ/8v7NHWl5/K7c0tLr+MncTdNTUppf1LDAIbbZsJnFzYpyWFzGV5ThfJ7s+j3Uu3yHdG69fv3lgT5PWXf67wCfl2KcvgNlmWT3FJhjHTFlwKyJJz/cJvbfQcnPa1q4POQmlotXj1WVu2DXq+axI90t7EL1AhN3L4vkd50rvdzUvm4ZGS7/LOlSkZmG17umZf33xnnSMlN0beu6WJ0i/ZTdUW/DadTKrIQAAqsXoI7oLuKzp4Yvj+9Y1A9X83MM4kF24erXT0lyi0df/+6aPH7k/03jTOd04v1rJP1IdT6r050qnTrdvT/HTuyd0Akm8PfpD4vYn/r1wyVS3fTH2Nis+49Gjzvnf4ee3NTX+/Svh97QHunHnQA/yZjpzDsjS/eI39Khy9SV+wvSb9ldy/zbukLZ8Pkkfa2kh6WWJK8rlbnzWAckfy6l5TtS95J07pKWb1B8vqRfTIFNl1PcgXpPaV5fj/vxyJSmT0BXk+RuX9FK5RDbPLT7MdI/1jF0jH1Z6jaiVY7Lwq2H+5wLfY6LVltez+O0HWOa8lwSXVX7/ME3dt+O+bfA2DzOfb051hFTB8yaysBdhWPf/dr1VM7cQtLnyngAzn9K8pOzr1RuvruVxXUk5fOV03a35rZpqu5lkX7edtO50stN7evzcIzj7P111/dvK40SIl8eY8v1WNRrviC8YXxZXqfo2tYVkJwi/SrLgz/GMTd4xRWu4Na0uUFL3wcr+m/TvN4Fj2TDDbt2aP/uokwu3b7YZN80NRZqmtdng74Gihuovll10hN1xLQlcH5JfniNn4Ltf18jyfOGThcr6dxJ0v0k+aaj/771cTdkmiqdIdvsXHYNB1wEJB1MeFH650ekR7fN2M93ShrSOqQL7xbpD8lDt8AkINlVEvP8bsqL6pPcw/jtHCMPDjp63IvYZry+q6FJuoORvgDzIL9erg5IOvjo+iDS8KvH6vBFSl0vuLu4/3jx+B15+XqMLl+45O/jfYxTeYhtHsP90NsIp0Nvh/TbBXJA0uXhB2jsmnKXUq9DQLJZ7Jh/CzTnYPlz51hHTB0wayqlm1bnFI/p2DblgOTvSPIFSR5/yoZuBekb8OHpVwcu79KWaJk/ZfcyJ7nrXOllDuHrC/l/qvbfBm9LXbrDxkHJ+7a47Nu1rSsg6U3um35LtnvPDoPeK6x4wa+sjhc/pbbP9FvVeg52H2Oa8tpibTfs8ljuNz5CYTRdmzfN65MVjwMYv8u6MUSf9adeJvIydbqkh0CjwBoOuAhIOjBRT77j8IWlmWvsq/vu970DVqeXPx/zImRsBZfze+j3U54kD53XY6S/Fo/43RzDLLbxo2VsjNh2fvX4UA4UupWBp+eUk3gdkPR3nyXp0aklSU7H7x2Y9NM94852BDjdtN6DELvFYz35zlY8jCDScx5iOsQ2I+2lvobTUvO/hnzXAcnn79gp32Rz8D7Kzq8EJJvRjvm3QHMOlj83jrM57YkvEOOC3efzPtM3pN/Md/RYwX+jukW+999/m/opzG1THZD0cteW5AfBhV9+9TnMwTmPkdx3mqp7mbe361x5KF8HJTzsihsgZI/83j0ido1VvU/XNpdr3Phs6863T/p9y7NtubBo+35L8+uApIfwcfl1TW7Z5J434ehXApLNYsc8PxKQbC6DMXPj2B6zLusgMFhgDQdcV0Ayg3gsrNjfP8pfjHx/zEqWgOTIQjrB1QhI7o/vFhQ3KU8i+wFJfhq3L8qGTh5Lyi1Rvl+SB/V3V3CPqRdBzaHpufWDLya+VJLHkowu2zmdqbeZ017a+6h3l5bvNeU3ApJ5MO0rduzgdcv50q2I4sKLgGQz2DH/FmjOwfLnbrWO8M1xdxv2/v/mjmJsCkjGKj4PeWxlt/i7hyQHWZoGtY/ld71O1b2sz7lyV172+d7naY9r7q5td5d0gzJcy5A0D9217dDpN+3rVn9vTRZ1QNI2Pk66Jt9sCMN4JSDZLHbM8+OxA5Kud+9Yrk1i78der9NCMgR53aRAVKRL3vm+AUnv4/PSSaQe5N4Dg3tMOQ+G/e4ynpxbQv1+GbsujBwgeVrVHfSPy7z8pD7/IXZbSX9ZWpq4G427nr6kBEXcPbXv1FXBuUXWQyQ9q6TvPDs/bmWWB+rO2xqbN3cz8aCqbo3qLvK+k+iuQx5HrysAd/Ey9p/93aXoZZJ+uTx0KOfL7z0egn099o/vUrproQdIt13bYO85jaH75vFGvD0HqZzPB0uK8QTfXMrf4w02TWM9mtKa67w11BFztd1KvrZ8DI2pXzymlesk/3PQ3H/Q/64k10cODnooAgfohwwVEgFJv3rga5dJV9e0eAKt62yfE718W0Dy1pKeUMZzdcDzHyT9mqRrdRzgQ+vObBLjytbJ+5xis29KX0QLt6YxJIecl1KSZ7zt+7eAVxhznj5jQxt4s9U6Iv628/57fKmuqSsg2bUe3yFQC2z191Y7+HMOSMYNO59vu6bnlnNiLG9PApLNYmsOSDbtcdTpQ5/5QECySZN5mxFYw0lpSEDSFyqxz76YiOmaqRVIfJ9f3fXFrbM8+UIwf5ff/0RZxheKT+pYzuv4hLarW0BJTm0VnLsHeUDznIf8/qXpoSCR1ti8OUBYj00U27KPnxTlz/VDXL6ujN8Ty+ZXt77x0w7z9NSSji+W/QCTvHzX0ySdxph9+42yDQeeHSjN24v3/qOjvgge65H3dQnvw2AJeSWP8xTY8jE0pn65TKqHfE6JBzCEY7y2PQ2+6SjIAcl4yqtvXjWdgxz4iAc8eey5toCkW1FF4DLylF8dDLxPQ2bG1J3ZpC3Q+e/FLT+8oy0gOfS81LAbvf4W8HpjztNN21vzvDhu1ryP9b75hrHHNvS++2nZuyYCkruE+L6vwFx+b2Nu2I1Zp8slByTjYUzubt/24Dc3UIhzcixvz6aA5NgbUUNv2IXJT7XsqM+Zvlnn69I8tZ0fvYxbFntM99eXBiFe3w+HvEhOoOP9Sdywq1tIutW4ryn9t4EbGvnvII+NHcM0RfbH3vB0IyS7+DiIqe163d+7wcw9yzpvLePa+gauW7W7pXT8LhlDMjR53YxAHPxL3uEhAUmfSGKfPb5NTG8o8/2EPl+s+WLli0sl8ZHy3R+Whd1K726S8oDGbsXneX7qricPTB7bcWXoJt0en87pPjt955NOn6mpgnNXnwgQemBzd0X1ifX6Vd4cKHQlGNOYvLkFZHQr8nhHrlA9/s7NS8uY2Fe/5oCku9y+r+yvTwh2uGo50TlY6uV9Ynf32ZgiIOmToL93sNMPDXELzF1dkMbsWwQMYh88pqBPBv7nLlQx/7GRwdIidIxHSmIxb2P/F5NhMjo7gS0fQ2Pqlxx8s51bRvoBFn6Knv9wjRaOHut0102aOBhyQNIPfoj6q6lrmoOQ3q6Dkg6CtAUkPZZrlK1bbfoCxk8A9fr+Yzu+u31kYo+6M5vsG5Acc15Ku3DG2z5/C4w9T5+xkY28iWNlI7t72m66RW/st7sT75oISO4S4vu+AnHc9V3+UMvF+XFIg4Ax63TlPwckr1d6cdmn7cFvP1t+t76m8ZirYVkHJMfeiBpzwy5Mntmyo9HF3I0r8tQUkPR1Vr72if2LV1+7OYC3a+rTeMdpjHVq2n4OSPphlZHn+tWtF3Mjk7F/XzRdmzfNc17dIyNf/9d5cmAy5hGQbCpd5q1aIA7+Je/kkICk70jHPrvrsSdfVMS8byzz8kv80eiTT566mqFHq5EX5hXKe99dirtrfQY992pNFVxsw2l9bcN2fizt1/ek72O9IXlzMNZGDoB63Lx6un/aVg5Ixknt7Q131fxUx3hCsluLxhQBSW/PD1Vw69W+05h9ixO5t2fnenpB2TcHdmMa6xHrL+k1fhtLyjN5nZfAlo+hMfVL/uPYD2ao//jPT7DNw4R0lXoOSHo5DyXicmnqmhZDm8TDGJoCkv5j3jeLnIYvglyf58l/fMfDn3zujOFDxtad2WTfgOSY81Let/p9n78Fhp6n622s/fMW6ojLloetPFySHyoV++xWkvH76CpnApJdOnw3RCCOvSHrHGLZfH50nvo0CBizTlfec0DS750H56XpwW/uheXzmb/3dc83p99xDkiOvRE1tvFHmEwRkHxg2icPveLrS7v4IY9xzm96iGxtfBI37HJA0mXklq7uZXIdSf67KcrW3zkoGT1Exv590XRt3jTPNg66e7v+57+D/HfDl5exbX2NHN/5lYBkfTTxefUC8QNY8o4OCUi68nGrEu/3r5Sd9gMp7l3+1eM6enmPX+jl/UdjnrouQnzXzGk2VSpXSRVP2x24vB2/rys4nxTjCX5PrBcun8+ZWtLkpwAPzZvTibtovpBrmnwSiCca5oCkn4hsO7eobJrsE8egn4buKQck3QJzyDR035x2nMidV+9HPbmLgvP43vLFPh512kv4HOWzhLySx3kKbPkYGlq/uATzH8dusV9PPi+5+5Fd2+rWep06IOmuTF7f5xGPXRSTx1aOG2ZuLeKpKSDph2d4fbe09MDuTVNuse7eAfvUndlk34DkmPNS0/7FvLa/BfY5T0faW3ndQh3hv2diP+PVf4/6AVJ9JgKSfZRYpo9AHH99lj3kMnF+dH76NggYs07XPtQBST+VPnzqB7/5nOjvfI50j7u2gGQ0jhh6I2rsDbsw2Tcg6fP/R8s+Ng23ks91feutvI7PiXka65TTyO9zQNJ1a9O4vH6+QpSve+15Gvv3RX1t7rSa5rlnYPxd5X2uHfzwrdyrpCl2cHpOj/d/GB1vi2xp0wJrOOCGBCR91yr2OXfZjoPAYzh43AsHEz2Af3Qb9jpDApKRni8c3Y37zpIeWlqSxB0mpzk2IOk77bEfbt7fNrkVaFPevXzfvPkEFdtyl7y2yYFRLxcBSXcLjPVeXAK7Du7mf3+WlvFJy1MEJP3AnLFT331z+nEid3P5psldJb0f0dVhrEdT2kuYF2W4hLySx3kKbPkYGlq/uATzH8c3bilStx63ax4vsWXR02bXAUnffPMDxpxGPg89oMxzK5D4o7kpIBlDlryuY6PuGh1l723sU3dmk30CkkPPS35gz66p7YJrivP0rm2v5fs4TtayP0374dYwHtrALXP8d6svTH2h2ndyK2S32vo3ST/fdyWWQ6BBYC6/tzg/9m0Q4F0Zs04DwRmz6oCkv4hhUeoHv/nhbbZzLwJPTQHJsTei9rlhFyb7BiT994T377+qob7K7p724ms8L+MHmvaZ2s6PY526tpkDkr6WbJrs/KayD74G9TT274um4GPTPI9baTPfwPXfBU1T2Hs5ApJNQsxbtYAPfP9b8jQkIPk1ZX+9zz6RxOQ7Ym5FGHcwwsWvMYbkkICku9/4ws7jLea0/D4uAv0+XwhGXppe6wrOzc8jXbc8aZscWI3lokvQ0LzFHUGn03S3KbbtgKuXiYCk757Ftvu8hkUEJHM37tjGrteh++b0hp7Ix3rsyvtcv4+ym2v+yNf8BbZ8DA2tX1yaY/847joS6oCkl43uS9E1zRcI8Ye6x8mKqSkg+axSv+fW97F8fvVTwV3+D5G0T9051iRa98dTtoeelzxm566p7YJrn/P0rm2u7fst1xFrK0v2Z/4Cc/m9xfmxb4MAy45Zp6tEmgKSTQ9+84NQYtx+j5PsqSkgOfZG1D437MJk34BkDGfiVpK58Uh+Hy35/BDQPlPb+XGsU9c2c0DSjUnaJndF928gHiY29u+L+trc22ua5+Ctt/fytgxJci/B+F0SkOyA4qt1CsTBv+S9GxKQjErI++2u0558hzqeKOq7dK7YPbajLyY83mMMzDskIBmtBb0dj9Xok5u767hrmx8wE93tIghXstL6UldwHmckys5PDm+bfBfdy3m/Yhqat7ytpjE2I924uI2ApB8KFHl0sPJ7d/yLAYYjIPnkSHjA69B9c9JDT+RjPQbsxqwWjTKcVabIzKIEtnwMDa1fXLBj/zjuOiiaApK5a5rr3xhCI7qjRXpNAUlfoLhcPcB92+RWmHGTz3f/96k7+5j8Z8lTbjVaByTzeck9CHadl/q0YGu74Mr7O/Q83Wa61vlbriPWWqbs13wF5vJ7G3N+HLNOV0k0BSSbHvx213J+iYe9Oc2mgOTYG1H73LAba1KfH90QJI6NXa/1NXGbcdv5caxT23Y8Pwck/bCctikeTOT999Tn74sYOzT/fVFfmzutpnkxLrcfatM25bwTkGxTYv5qBaLCWfIO9g1I+gQTd7f+KXVHi4fWeCwt36GqJ4+jYae68m2rZC+cLsIcpItub5GuuxNHAHRsQNItAeMpqbklS2wjXv10aufd++tpTN5cSca2/CTvtikuWiMg6Wbx0T3d45U1Te6C5Kd1+5+X9xQBSQcXh0xj9s3pDz2Rj/UYsi9zWnYNdcScPLeYly0fQ0PrFx8fY/847jq2mgKSXv7V5RzhrmlxF79unR51u4cxicnLu1x9Tj1/zKxer12W8XI3LRcLY84lTvbSKS1fyNSTW6/EcZYvGOoLrnxeukOdSPncdF5qWfS02W1/C4w9T3dta63fRdmtdf/YLwTmJDCX39uY8+OYdbrsmwKSXr5+8Ft0VfZQCzE1BSTH3ojK6w1p/OG87DLxg01d5jH0VOS/Pj+6IYiX8wPpdt2sa7uui7Tjte38mPd3qht2Oah328hAw6sfLOb99LMPPPX5m6vphmdT8LFpXtzAbXqYbMnCKX/jEJAMFV43IzCXk9I+4H0Ckh4wNsYE8T7fIG3wb0rFFN3W0lentWb0mD9epysg6SBjTH4iWbjm7cT3vjCL78cGJJ1W5Ot9kjz2ZT25e3Vsx0FXT2PzFk+aduXtC7968p2o2FYEJL3M35b5OQCc140ngbsVjcf39DQ2IDl233adyN1a1vuWT+RjPcouLuolynVRmSazsxLY8jE0pn7p88dx0936rkJvC0hG1zQPJRLDk3jM4zw1BSRvlOp8D0/SNEW3bqfrh8d5Glt3njvd6PPTTevJQcg4zroCkl5vzHmp3l7+nC+48t8CXmbMeTqnvZX3UXZb2V/2E4GTFJjL723M+XHMOl3WbQHJ/OC3a6TzS36YS1NAcuyNqH0aO0QPNTdCaZoiAJevY7xcHZCMsQ6jK3NTWn56thuR+LXPlM+PuYHOWKeubeaApPe5bYoWi/57xNPYG55NwcemeQ9O15F+OFnTlI8lApJNQsxbtcBcTkr7IEdA0gHHq6d/HkDcD2Fxt6x4IrX392nVxmKQYo915XVichAzgmNez9/niw3fwQq/G0ryAP7uouYnr8V8p+1K15O7artFRnTX9jIOyPWZmio4jx0Z2/G4FNEF3RW+u4d7UGJ/7wvNi5eNjM2bA6uxLXfRi6equrWJA6x5XMwckMx5fHq5C+Ws2MnrxfhiDgrHFOZDW0iO3bcxf9yM9Yh9XNJrlPuS8kxe5yWw5WNoTP0yNiDp8Z88PIf/1S0s2gKSuWuayyl3R4ujqCkg6e88fqTX8dMs75fOdb5B9qR0zsgPBtin7ozxLd16w+NB+1zn86rH84reD87ProDkmPNSWDS9tv0t4GXztvqep5u2sfZ5W64j1l627N/8BObyextzfhyzTlcJtAUk84Pf3lHOZ/lhb04zB5EulzYy9kbU2Bt20WOh6fztHgzRum9XQPL66bx9s7Q/8fYzU1oOgvaZckAyX0N73bFObdvNAUlfl160YUEPwxLHfwQtx97wbLo2b5rnGEFss+m63y5xo9TLEZBsKDhmrVsgfiBL3ssISMa+tL26lcYPVUFF7/etUkXhdf3U0Ki8/flV6XsH+Rz48pQHoI1t/kT5zhceMc+VooOlHiTY83xCcGtLv3eX5keVdbpemio4L/+wtB2n55NmDg56n+tm9WPzFk/sjv1ya8noeh7z/JoDks6j9y9///bUEsfz/eCfK6adHxuQdBJj9m3sHzdjPdKuLuJtlN0iMksmZymw5WNoTP0yNiD5u6mu9Q2fPLUFJL1MdE1zOeXuaLF+W0DyS9JDcLzux8rnGDfS83w+qFvVj607cytIp+1zUNxsUL7xogAAIABJREFUfEvKy66ApPdr6HkpLJpeu/4W8PJjztNN21nzPJen/zEhgMDhBebyextzfhyzjkXbbti1BSS9TrQ8DK96iKy2gOTYG1Fjb9i5V0Pk0UG2aITihjUetzC+2xWQ9D7HjUZfS94uDafl81wep9+90vpMx7xhlwOS3ud/rRrreNxsX4P6O9/EvFTagTE3PJuuzZvmeTPuhent+gbuvUrDHM9375E/SGXkZQhIpoLh7TYEfOD735KntoCkA4AOUHlMDI99GK36mvb1B6uWizZx8NF3Mjye1JtTZeGLsJjc3c1BwBgX60fLF27Knk8CTs9N430HxJW6x/OIi7bfj8Q6XqNrsy8M68mtIV9ZKrkoT++7t/Wl9cKlmf3YvN2tBGxjO359lyQHYmMszjog6Sw4KOpAb+xzrO9Wkx5HJE/7BCTHuMfF8dCn0znPYz3y/s79fZTV3PNJ/o4r4D/sfFe5z7TlY2hM/eLW9mHmVv9NU1OXbQcTYz3fkc/TM8p3DkzWU3RN87q5O1os1xaQ9PcXKkHM96ZtOx0/idPdv9yCsWkaU3e6Rb7TjLGJvR2fV/2EVj+UJwbkbwpI+sKxnoacl+p1689tfwvEckPP07HeVl7juN3K/rKfCJykwFx+b2OCi2PWsXXbDbuugGR+8JuvXy5fFVpbQNKLjb0RFX8zRBn1afzh1pzRitPr+RwZDV/82ddlfu0TkLxyCtp5HV9P5rQ8r09DmqA65g27HJDMvSbc8MXDmznv/uegoMfIzNOYG55Nwcemed6OW2ZmR5eRb6RGnqLRkj8TkMwlw/tNCMQPYRM7u2Mn3bX6SmWcRY8nkSdfCPnElO+m5O/b3nt5n+yu1nDxftnSvdwBzymmzyhdzh2EdH53Tfvkzc3gPUal774NmdxSJjzqVjND0tm17D77tivtpu/HejSlNbd51BFzK5F55OcPy8Dnd2+o2+occgzVIuv87C5rPi9E64w+ezmm7vRFxzUl+aE5Pu/tOx3rvOR8Dj1P77tvS1mfOmIpJUU+5ypwkXL90id/c/m9RfBtSIOAMevYpO2GXVdA0uvFg9/qh735u66ApL8feyNqzA07BxLj5mGUr4OUbtXpG42e1xaQrG/Y+aaoH8Ti5SMtv7r3oBvw1F2vva9d07Fu2PkmuYONzquHSHts6p3oef7ODZWabryOueHZ1FioaV7YOLbwJ2m4MufJN1XdVd83n/3Z/whIhhivxxJwi+c8xuuxtnvGduLgP2MGbxBAAIEkQB2RMHh7hoADknFseEy/rsBkLHfGyrxBAAEEkgB1RMLgLQIjBNx6z78j94C6zo71+b3tAJr467E3osbcsHNDketJ+oIJ9sEBCqfjMZt9w3FoIHJoFsY6dW3HY2j6+RAOPPe5gTn1Dc+mvNnRvQN9Y9Xbm+NEHTHHUjlcnjw84SskfctJBSY54A5XuKSMwBoEqCPWUIrT70MOSMYx0haYjO+nzwUpIoDAGgSoI9ZQiuzDSQrkBzv69+SxgR2EaZr4vTWpMA8BBEKAOiIktvGan5dyIoFJDrhtHGjsJQJjBagjxsqte72mgGQcK3VgMuavW4S9QwCBsQLUEWPlWA+B0wXcki1+R/n1z0sLsewU3+d5vEcAAQRCgDoiJLbxmgOSUfZHDUzGRrfBzV4igMBQgagjeG3+Yx+XdpcITIbR0GOP5RFAYBsCUUd4YP+PlCegfqiMteVxzD4gyU999YMJ/ACld5cH6vnhg37og8dK8/hmfoKqHxzghwW4/vEDCf30VD8E6o2S3iDp9ZJeK+k1ZXw4/yH+b+WJrH5A4L+UccY8HtvLJL1U0kskvVjSP0n6R0kvkvTCMvaXn57qhyv6wRF+mODzypNq/0bSX5cutM+S9Fel1ZofLOUg0Z9J+lNJfuCUxxXzQ/7+WNLTJPmGjx/w53Hc/ARWP/zQD2j8PUlPKk+7/R1JT5D0+DI+3m9L+q0ybtr/k+SHf/gpwY+R9GuSHi3J4+/9qqRHSnqEpF8pDyT8JUm/KMlP6P0FSQ+V9POSHiLpwZIeJOmBkn5O0gPKuHR+wJQfGPlTku4nyQ9z8AMOPV7cfcpDIf2gx3tL+hFJfmjED5Ux6O4p6R6SfkDS90v6vjLshx804Yc+fld5aOB3SrqrpLtI8pOEv0PSnSTdUdK3lXHi/HCq25cnAt9W0m0k3VrSrSTdsozzdwtJ31TG9LuZpJtIurGkG0nyU4D9EDA/4fgbyvht7vLqceb8tGSPreUuqx4f112hPXatWx5eS9JXlGCfu15eo4wD92XlARZXkeRx/Tyuu7tneox6PxTziyVdsTyIyw/8cCDRrRvd5doPZnTXWHe7/RxJny3pMpI89pzHRvcTeT+rjNN7MUnu0uuxIy8syduN31HTq48z59FTfF8+8oIAAgicIkAdcQrH6j80BSTjGDhKYDI2tnppdhABBEYJRB3B65l/xGMxzmLUAchKCCCwegHq1HF1Km64DT0GHPiOdVZfsbCDCCAwSiDqCF7PrC+3bnHQwGTgjjpaWQkBBFYvEHUEr5yUhh4DtJBcffXADiIwicDQuoXlOR9xDOx/DEzy4yURBBBYnQD16/7161oNf1fSOaY+4gNr6nRJDwEE1iFAHbGOcpx6LxhDcmpR0kNguwJxnjmPpPOVp6H66agXkHRBSZ8p6UKla6q7qbq76sUlXaJ0YXVXVndpdddWd3N1d1d3e71s6QLrrrDuEuuuse4me4XSbfaLShdad6V1l1p3rXU3W3e3vaqkq5WusFcv3V3dNdfddN1d1912v6p04XVXXnfpddded/N1d193+71+6QLsrsDuEuyuwe4m7O7C7jZ8U0nuQnzz0qXYXYvdzdjdjd3t2E+8dBdkd0W+XemafIfSXdndlr+9dGF2V2Z3aXbXZndzvlvp9vzdpQu0u0LfvXSNdjdpd5d2t+l7lS7U7krtLtXuWu1u1j9Wul3ft3TBdldsd8l212x303Z37Z+VdP/Shdtdud2l21273c3b3b3d7fthpQu4u4K7S7i7hrubuLuLu9v4o0oXcncld5dydy13N3N3N3e3898sXdDdFd1d0h9Xuqm7u/oTJfnCyF3Y3ZXdXdrdtd3d3N3d3eeoPypd4N0V3i0D3TXe3eTdXd7d5p8pyV3o/QAYd6l313p3s3d3++dIem7pgu+u+O6S7675Lyjd9d1t/x9KF3535XeXfnftdzd/d/d/eXlqqYcA8FAA/1qGBnh1GS7Awwa8rgwh4KEEPKSAhxbwMAO+medhB95ahiDwUAQeksBDE3iYgneVYQveU4Yw8FAGH0wtH+P3VL9+qjjF/O3WOOw5Agh0CUQdwSuByTgGaCHZ9YvhOwQQOKhAVEQH3QiJL06gKSAZLSLPXe0Nx1AFwkcEEDhFgDriFA4+IDBYwEH1+B3Vrw5EOnjrZTzF9+UjLwgggMApAtQRp3Cs/gNjSK6+iNlBBJYtwElp2eV3qNzngGRbIDK2zTEUErwigECTAHVEkwrzEOgv4Ba+8TuK10+WFqVuCZyn+D7P4z0CCCAQAtQRIbGN16aA5EFbRNasHHC1CJ+XKuAnL/pJjX5iIdN0AtQR01muKSUHJHcFImN/OYZCgte+Au6u6/rc//xUWaZugbOXrrr2cnflfSY/idfp+MnFx5qoI44lzXbWKuDhBuJ39InSxd3DEzRNsVzTd8xDAAEEqCO2dQzkgORRA5HBzAEXErwuXcB/gPl49nhLTNMJUEdMZ7mmlDx2Wt01u23/OIbaZJjvMfI8vp3H5svTZdLFtcfrY+oW8BiL8Tv7+u5Fd37rsQKdlse8O9YUeT/W9tgOAmsT+FJJHy/jbnqs1K5pa7+3k7jJ0uW/73e+SedxXz0Gq8dl9Vi5S57WVj5LLovI+9bqiNjvrb46IHkigcgA54ALCV6XLkBA8jAlSB1xGNctpcoxtKXSHravfgCEjw8/HCNPBCSzxu73BCR3G7EEAmsW8MOe/PCmPtPWzskncZOlTzmMWcbl/P50A8pl6YdmzX3ysembj/5XT2sqn3rflvp5a3XEUstpqnx/raSzTZXYmHQ44Maosc4cBfzkST910heyTNMJUEdMZ7nVlDiGtlryu/ebgORuoz5LEJDso8QyCCBgga2dk9cU8PKT711+boThp87/giR315/7dIt03NWBjzWVz9zLoW/+tlZH9HVhuQMJcMAdCJZkEViJAHXESgryBHeDY+gE8We+aQKS0xQQAclpHEkFgS0IbO2cvKaA1/NKYO/pCztQuwKSfuiSG5TcaGH7tObsbq2OWHNZLmLftnLAfbukp0l6mKRLSnqkpBdI+oCkf5H02JYm75cu63ndc0ryGC0PlfSSsk4U8sVL2j5RuCm9L7J+WdL1YoH0OjYvHi/E+fg2SR7A/rskPUXSu6puGh5b5CGSnlW+84Mn/ljSj0o6T8pH/fYukp4s6XWS/lvSK8vdN+9z23RLSb8n6fVlW87fvSRdpGGF80q6u6TnS3qnpA+W9fxwjBtOsLzz4e1fO6Xlgfk9z10EXEYPlvR3kj4k6c2Sfl9S21g73u+fl+Qy9XHyj5IeLunCJR2ne/20rbW+3Uodsdbym8N+ncQxdPkytlLUyW+V9LelLmirB4fUne425TrA/1wn3KrUJ2+R5G39gSSPs+nJXZV+pZwXXO+9RtKvNTx8ZOy5oWzmtJdbl6equv52Pf4PZVtt4zBeVdJvlTy5XnxHOb/dr9SZOe14P+R8F+vUr08odtH17NXl81PLgnWX7e8odfd7y7njRZJ+oKOLyZBzU523+HyIMo60hxxrsc45JN2zOPkY83nJ5XsPSRdLrU+axpAcUmYncfF+EnVEuPKKwNYEtvZ7O4k67VDHlK9hXH4eO3JJU1dAckn7sZW8bq2O2Eq5znY/t3LAObDkfX1feTJs7Hd+/ZQkn7RyU3IHpWKZK0j6r/T58aVUv07S29L8WN6v/yvprlXpj82LL9Sc5oNK8DNvxxffnjyOyH+25MXLv1SS9yNPblkRaec04733wXeu8uTg4m/u2I4v5mLy8s/uWN7byuOKDF3e22kaQ/I3yjYdeIyWOLFf8eqL9trEd+kcOIhl8qsDkw5mep6DuGufYt/Xvp/s3+EEjn0M3UySA1ex3frV9eAFqt0dWnfmgJlvVNTb8GffLPrGEqBs+t43f1z/xjT23OD1/VRqd99q2o7nuX68T2yovN5U0sc61vFDTXI97tWGnu+qTZ7x8d0t2/VDGTxl3xe2LOv9ekxZPl6GnptivabXnIepytjbGXqseR0HFLvOoQ5MRtnXAcmhZXYSF++R96ZyYB4CCEwrsLXf26467dA38sbc+KtL3F2zfQP0f0pd74Yg/uzGGJ6iAcZPlc/1i29KevknpS9inTGNNpxMn4YsNynb9U3EOO7cSMZ58UPtPPnVn91YqJ6G3rwbu09TlFGd9yV/jrJa8j6Q9wUJbOWAiwu92N+/kXRbSX6y13dK+vdUUd4+lV8OSPrizOu/QdLvSvIJ7BIlyOn5TsOBO1dqbp3hi17Pd0DPF8gxjc1LBA19EnK6vnD7q3KXzBdhny3pw+U7X2T+tKSvLK343ALG6/ifW864pUVMEbTzd88oJwa37PkZSW7t4/lO74qxgqQHpvTc0seDoXpbP1fy5XUcuIvJrTo9z/8eLenLS3q2fmOZ/1FJ5ysrDF3eq3UFJGPbvy7JF2v+lwOqbiEbk8diibTcktMnaj8B1uXtFlaRll8JSIYarwi0C8Rvpn2J6b5xa0TXjd6mW5D9iKTrSLqdJLfGjrw4eBfTmLozB6ucputV12eu/3841SH+zvXnA0q953rSLdsjH3eKTJQW2THfr33PU07iESlN/+Hvc9CXlTrKreki3Xx+c0DU899Uzoe+MeO62XV0LJ8fNjPmfJd275S3blF6N0kRmPS5x5/jBl7t65tAtvI52y0C3crUefykpLgh5w0MPTedkqnqQ52HKcp4zLHmbPmmWpSJj2O3NnFZudfB29N3XiYHJMeU2a6L94ppko+xb5MkRiIIINApsLXfW1uddowbeWNu/DUVXpzzouzi1T0bPMW13DPL5/rFvQy8jhthxBTrDG20MaQhi3syRF7r158oGWkrnzE378bs01RlFK5reI2yWsO+sA8LENjKAZeDgH8q6VxV2bgbtwON9nDXsQjY5YCkv3MriXOndSOo5QuCupuyt+GgnNd7blpnbF4iIOn0HMRzkCxP0TrGAVAHCOvpx9JJ4XvKl1+QLpyfWLqC5/Xcldrb8z8PpOzJ6zh46Hl1ixt/n5vlX/f0Vc64wHWrw9wC1V9fJW0jukDHBXHf5Z1OBBFvU7bplzgxOa8+4dWTu+37O19oxhTBWwcz6u7cDvy6y7nX8T8CkqHGKwLtAvF7aV9ium/i9+tWBFdrSDZ+v/6jPOryMXVnDlbVLR29WbdCiP32Hfs8uR6JFpweRiSmsecGBxIjCOuLkfr85tZ1HrrD+fGNM3dZd6uDyN+9IwPp9Y/K974JE9OY812s2/YaLddz4NPLZl/fGKpbauZzU7SyGHNuastXnYepynjMseYgt8/rLi+vX59DP7dqhZsDkmPKrO3isMtq3+/iWNw3HdZHAIHdAlv7vbXVace4kTf0xl9b6fnaxjftIj03SPFnBxo9xfXOmIBkHA99Gm3kbXm9XQ1ZfJPY+Yy/zbyOhxzzvPgbral8xt68C4ch+xSmfW/Oni6+7v/Db917yd7NRmArB1y+0Ltyi74rx/BwywNPOSD54jIvvzho5XU8rlPT5AuDSPMLywJj85IDkjevNuYLlGjG78Bi0+QxMOMO23PKAj9e8ufu6j5pNE0OZD6qnED8vVv/eJ/cfT0Ct/V6McbJ75QvfEEeDu52kLspehGflNyaJy46hy7vNLoCki4nBwHqyeNdOl8ODnjyhXx0YfzFMq9+8Z202BcCkrUOnxE4q0D8Xs76zbRzXA9+pPw+/cdv0+QbIO4W5DrNf/COrTtzwMznjnrymL3eb9etTXVP3AxxPmIae26Ip256Wx4cvmny+JRRDu7C63GIPV6k57nV/VdXK/kmnevkK6X5Y853afXGt30Ckj5P1ZPzH60r4/w75txUp5s/T13GY48191aIY+myOYPpfey7l8sByTFl1nRxmDZ1kLdxbB4kcRJFAIFTBLb2e2uq045xI2/Mjb9TCqrhQ1xf1WNIRiBubEDSRvUUf6fkRhtjGrI43dxYxefCPDWVz5ibd04zHHyM99mnQ5RR3relvt9aHbHUclpNvrdywMWFnlsWtk2+AAuPO5SFckCyvuj8nLS8g5Xuhlf/+7O0jCtjT2PzEgFJd6OuJ1+kRN7d3blt+r9lOY956SlaT7yifO7zEuu4lWS9v/E5ugj6YtOTL+zcIiny6KCfuyO6i7cf/BMtlcrig5f3el0BSY+v1TS5C6DzFF0YfKKNPDrw2DR9RlqGgGSTEPMQOFUgflOnzp3+U66T3SW4zzS27szBKnfDrqfopuTW802T6z+7NAUkh56nouWB7/K3Te66G+Xg1gme6nER3S3aFwE+112uLBMv2bbP+c5DXPSZ+gQkb9ySUAz34WCcpzHnprJq48vUZTz2WPONPZfdyxtzefpM3/CM8o2A5Ngya7o47Nj0JF9F3idJjEQQQKBTYGu/t6Y67Rg38sbc+OssuPJwN5fflAHJvo02nLcxDVm83pCA5Nibd95OBCT77tMhymhXGS7h+63VEUsok1XncSsHXAQB/35HaYZH3FXJAcnofhxJ+HMs3+c1LgLH5iUCkrn7d+TFY6RFHtz6pW1y17xYzt324qK47Y5aUzrefqSx6zUCn07HLTD/vIz5Va/n1pbRyiW2OXT5roBk2/7VY6rkMo1WspGf/BqtsAhIZhXeI9AsEL/35m+nm+uhKmJbbU+Vrrc2tu7MwaqmbUVAMteBedtR9zYFJIeep55V9jtavuft5PfxoK6HlJn+o//7G8YgDEPfUIthK3LdGN93vXqcxz5Tn4Bkk6/TjrGfIyA59tzUls+py3jsseYnxdvaD7Vpm9wKN8ojApJjy6zp4r1tu1PNj7xPlR7pIIBAu8DWfm9NddoxbuS5BIbc+GsvsTO/OUQLyb6NNpyLuPE3pCGL1xsSkBx7887biYDkkH2auozOLK3lvttaHbHcklpJzrdywEUQ8N86ys1PXQ0PDxTvKQckv7jMixd/juUfWh5+4gegtP2LJzmPzUsEJJ8cGUivfhBL5OWb0vz6bWzbd448/UlZzye4vpO37215TLK2fY3539qQqMfa9AOFfCHu8Toj3371xXE99V1+ioBkHs/ST8dtmi6U8kxAskmIeQicKhC/8VPnTv/p6um3GUGZXVsZW3dOHaxyPqN+Hnqecst0G/tBam2THxgW4xBGAC+WdWDSdu5m/vTU7d1puhWiz435fOeW9lHHt7163MM+05QByX3PTXV+py7jscdalK+fNt42Xbrh2B9bZk0X723bnWr+seqIqfJLOggsWWBrv7emOu0YN/J8jAy58dfnmDpEQLJvow3nL26mtq3Ttg9DApJjb9552xGQbMtf3RDF60xdRm0GS5q/tTpiSWWzyrxu5YCLCz13M75YS0m663B4eMB8TzkgGWNAlq/kMRnjQQJNgTcv5zEJv6L88/KexuYlApJNY0S6taPHD3P+f7Zsp+nFgyB7mX8qX8ZdIY/FVXebjvUfL+mfJT2mzIjxrF4bCzS8unWj9zvGpXQrG48T6bE66ulGadzGuOAaurzTnCIg6e7YcdHu7uRNk7sixnFCQLJJiHkInCoQv5dT507/6YLpt+mnazdNX1XqM9dprpPG1p1TB6uc17HnhgeV/f6wJD81tGm6drLxcBSu67z/vglTT+7enZ9I7psz+XwXQ5rU6zWd7+pl6s9TBiTHnJvq/OTPU5fx2GPtwaXsPLSIg8NN0zen8o1g/Ngya7p4b9rmlPOOVUdMmWfSQmCpAlv7vTXVaXGj55A38vLx0efGX16+7f3YgKQfZupyjyGqnP6Y4N2Yhize1pCA5Nibd2P3KaynKqNIb8mvW6sjllxWq8j7Vg64uNDz/j6gpeTiqaLujusLMk9dAUl/7yeQOk0H+FyR1VM82dpBLj9AwdPYvHQFJJ3ui0pe3tcSdPVDC6K8f/n0rMjjTca8puDaNdL3P1HW8ZOwY52blXn5xQ+s+c+yjJ/Y5imeNu6m/k3T08ryMXba0OWd5hQBSacT2/ZYnReuMuugrYOmsf9NZtUqi/8Y+7r4HWEHTkzgmMdQPC3RD2qpnzZtgPhj2t2X4ybMmLpz6mCV8zb23OCbOmHcdn6L1iA+v3m85JukdfyU5npyC8dI02Ptehpzviurtr5EQLLO9y5fJ1h32R5zbmrNWPWk76Zu42O65Y851vITxf03RT15DKwoG5dZBCS9XMwf8jdK08V7vc2pP8exNnW6pIcAAmcV2NrvralOO8aNvKE3/s5aUmed0xaQ9PWWy9UNT5qmaICyb0Ay0hnSkMX5GRKQHHvzztsZGmQ9RBk1+S9t3tbqiKWVz+ryu5UDLl/oOTjo4Fq0NHDQ6RHp4isPFLwrIOnxGsPQXd18EeXJ3ePcCiXG7HIT95jG5mVXQDLnxYPfR8sXB0r9VG6P0+i8vl/SxUtm3ILiVWW+n9LtrtSe5wt5X7BG8M0tQfPTWz1WWaR1u7KOk3QrUrfgDBOP6eYpTmBO585peX/nsRrfWdaJp3IPXd7pTBWQzBf3HoPELapcnteU5C4AsW9+JSB5WvHyHwKdAvGb6Vxooi/zTZa/lHSpkq5ba3sQ+2gB7ZbfMY2pO3cFzMYEq8aeG7wfUSd/UtL9SstPz3ePgCelessXYZ489ES0qve6+SE2ru8endbxw748Zae+57uyautLnGM8XqUDo9GDYZevE6wDkp4XDj7P9Tk3tWbsQAHJbNj3PO08Pr+Uh8v3XuWc5PkOLv9BKiv/1nJAMm+vb5k1Xbx3OU3x3THriCnySxoILFlga7+3pjot/61f3xCLst33Rt6YG3+x7bbXtoBkBFg/lK5vIw33nIiGIvsGJPPfWE3XQE0NWZyPHJD0TbQ8NZXPmJt3TnNoQPIQZZT3banvt1ZHLLWcVpPvrRxw+ULPf9B7v30x5qeKxgWq57kbXwTrXMi7ApJexmMhhqNf/VTVeOiJP79H0hXTETM2L7sCkt7Ew6q8vKMEICN/zlfdvdxd1X2CimV8MvMYk/HZr/WYY1euHoTgrvB+eENeJz+swRea8eRtL+OLxVdKcivEWMfB23ha7dDlve9TBSSd1i+mfEX+4tUnSY+f6c9NJ+NU1Kt4G/u9ip1hJ05E4JjHkG/ARCvI2G78IR6fXe97bNo8Da07dwXM9g1IDjlPeT++RNKbUr31sfI5n998IeNu7TH5IixMvNwbSr2cz1/u1panoee7vG7T+3iCdOTDN6087fL1Mk0ByaHnptO31vz/rjyMKWNvaeix5nXcYjWfY+2Uz58+B4dhDkh63aFl1nRx2Cw03dzI+3QpkhICCLQJbO331lanxQ2sQ93IG3Pjr63MYn5bQNKNPaJc3agjrmV9PeUHosV3+wYkxzZk8dAvkQe3+ndPRN/89NRUPvlm2pCbd0MDkocoo7Jbi36Jslr0TpD55Qhs5YCLIKBbvLm14GtSxWgDBw09RmJUjlGCOSCZW5DE9/HqIJ+7CuaLP6frFgkeCyNPY/PSJyDp7Xj/HOyLC1rnwxcr7rrl/WmaPGaj7wTmixqv5ws+tzJpmnwy8cVqDmZ6HV/8/6Ck+g6YW1i6JUcEDr2s/zkw7G3XXeKGLh/peiytmPzwBW9jyODGsa73209tjVauDqi69acDGe4W73S7HiAU6Sz9Ncpp6ftB/k9O4NjHkIOSbkWWb4I4D65r3DLSgaamaUjd6fov9ssPhKmnvsGq3CKhAvSaAAAgAElEQVR/7Lkhtu0/rB8n6b0pb86jHTy+4jliwfR6z/LgmtiXePWNNbe48MVHPQ0539Xr1p/9sDefmyII6nEwPe3y9TJNAclYd8i56bQNNvy3Kw9jyjg2M+RYi3X84BoH2+Oc5LLyee8F5aFEUXZ1QNLrDymzGGrGN2iPNUXej7U9toPAlgW29ntrCni5/I9xI2/Mjb+uY7MtIOnrVzdCibL1Tat8EyvW2zcg6byNacjiHnSRt3iNocDaymfMzbuhAUnvz9Rl1FV+S/kuymgp+SWfCxfYygGXL/SiyByE8x2Yy8eMCV7d+sSt/PywgNwSJSd9rLx4XAx3h3YQsumiMucp3nu5K0nyAxCaHkATy+VXX/y7S9/XlG5/dSAyL+v3dnGQ9rqlW3ndUmnf5ev19/3s/ckWDkzH78atcdY+xb6ufT/Zv8MJnOQx5C7AHj/XFx4el6jPNKbu7JPurmWmPDe4nvJ+RyuJrm27jnMrCg9P4brf58am4GWdRp/zXb3OMT8PPTcdM2+xrTHHmsvL51API3LeSKjn61zL7CTriJ50LIbAagS29nvruslyjBt5Y278tR1sEVjMNzNjWV+T+EZSlK9fHaT0w059zeXPOSC5T6ONMQ1ZfrzkJ4aL+dGS8a7yGXrzbuw+TVlGUR5Lfo1jaMn7QN4XJLCVA67pQu+kimlOeTkpgzlv97fLmJZ5jLmc318pJ3W3Jm17om1efunvt1JHLL2c5px/jqF+pcO5oZ8TS61PgDpifWXKHs1XgN9bc9kc8kbe2Bt/zTntnusbjG7FGOM/dy+937djGrKM2eKYm3dDt3PMMhqat2MvTx1xbPGNb28rB9ycLvTmlJeNH/6Nu/9DJeDoO3j3Ty2M/JT0h6Q7j+5KsIVpK3XEFsrypPaRY6ifPOeGfk4stT4B6oj1lSl7NF8Bfm/zLRtyhsAcBKgj5lAKG8rDVg64OV3ozSkvGzrUe++qu7R57Mj4bXhc0Dxml+e/WJKfzr6FKRy2sK/s42EEOIb6uXJu6OfEUusToI5YX5myR/MV4Pc237IhZwjMQYA6Yg6lsKE8bOWA84DufyXpkTMo2znlZQYcs8yCx+W6u6TnS3p3eViRn2j61+WBPX3GV5vljo3I1FbqiBE0rNJTgGOoHxTnhn5OLLU+AeqI9ZUpezRfAX5v8y0bcobAHASoI+ZQChvKAwfchgqbXR0t4IcjbHWijthqyU+33xxD01mSEgJrFKCOWGOpsk9zFeD3NteSIV8IzEOAOmIe5bCZXHDAbaao2VEERglQR4xiY6UkwDGUMHiLAAJnEaCOOAsJMxA4mAC/t4PRkjACqxCgjlhFMS5nJzjgllNW5BSBkxCgjjgJ9XVtk2NoXeXJ3iAwtQB1xNSipIdAuwC/t3YbvkEAgTOfo4AFAkcR4KR0FGY2gsBiBagjFlt0s8k4x9BsioKMIDBLAeqIWRYLmVqpAL+3lRYsu4XARALUERNBkkw/AQ64fk4shcBWBagjtlry0+03x9B0lqSEwBoFqCPWWKrs01wF+L3NtWTIFwLzEKCOmEc5bCYXHHCbKWp2FIFRAtQRo9hYKQlwDCUM3iKAwFkEqCPOQsIMBA4mwO/tYLQkjMAqBKgjVlGMy9kJDrjllBU5ReAkBKgjTkJ9XdvkGFpXebI3CEwtQB0xtSjpIdAuwO+t3YZvEECAMSQ5Bo4swEnpyOBsDoGFCVBHLKzAZphdjqEZFgpZQmBGAtQRMyoMsrJ6AX5vqy9idhCBvQSoI/biY+WhAhxwQ8VYHoFtCVBHbKu8D7G3HEOHUCVNBNYjQB2xnrJkT+YvwO9t/mVEDhE4SQHqiJPU3+C2OeA2WOjsMgIDBKgjBmCxaKMAx1AjCzMRQKAIUEdwKCBwPAF+b8ezZksILFGAOmKJpbbgPHPALbjwyDoCRxCgjjgC8so3wTG08gJm9xDYU4A6Yk9AVkdggAC/twFYLIrABgWoIzZY6Ce5yxxwJ6nPthGYvwB1xPzLaO455BiaewmRPwROVoA64mT92fq2BPi9bau82VsEhgpQRwwVY/m9BDjg9uJjZQRWL0AdsfoiPvgOcgwdnJgNILBoAeqIRRcfmV+YAL+3hRUY2UXgyALUEUcG3/rm4oDj9cxH3GOBBcfAWY+BrdeV7P94AX5PZ/09YYIJx8BZj4HxtQxrIoBAX4Goe/ouz3IIILAtAeqIbZX3ie/t43TWPwjjIOQVG44BjgEfA8898ZqKDCxZgPMM9QjnEo6BXccA55kl1/LkfUkCu36LfE99zTHAMeBjgAkBBBBAAAEEEEAAgcULnFPSL1Q3QF8t6eyL3zN2AAEEEFiWADcJCTYRcOQY2HUMcJNwWfU6uUUAAQQQQAABBBBoELiUpOdVwcjXS7pqw7LMQgABBBBAAAEEEEAAAQQQQAABBBBAAAEERgt8taS3V8HIp0u60OgUWREBBBBAAAEEEEAAAQQQQAABBBBAAAEEEGgQuKekT6Rg5Ccl3U/S2RqWZRYCCCCAAAIIIIAAAggggAACCCCAAAIIIDBK4AKS/iAFIj1O0X9J+vpRqbESAggggAACCCCAAAIIIIAAAggggAACCCDQIvAlkl5VBSNfKOlzWpZnNgIIIIAAAggggAACCCCAAAIIIIAAAgggMErgdpI+WAUjHy3pXKNSYyUEEEAAAQQQQAABBBBAAAEEEEAAAQQQQKBBwAHHX6oCkR+SdIeGZZmFAAIIIIAAAggggAACCCCAAAIIIIAAAgiMFriMpL+vgpGvkfSlo1NkRQQQQAABBBBAAAEEEEAAAQQQQAABBBBAoEHgupLeUQUj/1DSZzYsyywEEEAAAQQQQAABBBBAAAEEEEAAAQQQQGC0wI9I+mQKRn5C0n1Gp8aKCCCAAAIIIIAAAggggAACCCCAAAIIIIBAg4BbP/5RCkR+urSS/LqGZZmFAAIIIIAAAggggAACCCCAAAIIIIAAAgiMFriypNdWwci/k3Tp0SmyIgIIIIAAAggggAACCCCAAAIIIIAAAggg0CBwR0kfroKRj5B0zoZlmYUAAggggAACCCCAAAIIIIAAAggggAACCIwSOLekR1aByA9Kuu2o1FgJAQQQQAABBBBAAAEEEEAAAQQQQAABBBBoEfhcSS+qgpH/JumLW5ZnNgIIIIAAAggggAACCCCAAAIIIIAAAgggMErgGyS9qwpGPlnSBUalxkoIIIAAAggggAACCCCAAAIIIIAAAggggECDwNkk3VfSp1Iw8uOSfrhhWWYhgAACCCCAAAIIIIAAAggggAACCCCAAAKjBS4s6RkpEPlpSW+TdJ3RKbIiAggggAACCCCAAAIIIIAAAggggAACCCDQIHA1SW+ogpHPkXTJhmWZhQACCCCAAAIIIIAAAggggAACCCCAAAIIjBa4s6T/qYKRD5d0jtEpsiICCCCAAAIIIIAAAggggAACCCCAAAIIIFAJnEfSr1WByA9IumW1HB8RQAABBBBAAAEEEEAAAQQQQAABBBBAAIG9BC4n6cVVMPJfJF1hr1RZGQEEEEAAAQQQQAABBBBAAAEEEEAAAQQQqARuJOk9VTDyiZI+o1qOjwgggAACCCCAAAIIIIAAAggggAACCCCAwGiBs0v6KUmfSsHIj0m6x+gUWREBBBBAAAEEEEAAAQQQQAABBBBAAAEEEGgQuKikZ6ZA5KclvUXStRqWZRYCCCCAAAIIIIAAAggggAACCCCAAAIIIDBa4BqS3lQFI58l6RKjU2RFBBBAAAEEEEAAAQQQQAABBBBAAAEEEECgQeA7JX00BSP/V9JDJLn7NhMCCCCAAAIIIIAAAggggAACCCCAAAIIIDCJwHkl/XYKRLqL9vsk3XyS1EkEAQQQQAABBBBAAAEEEEAAAQQQQAABBBAoAp8v6WVVMPKfJXk+EwIIIIAAAggggAACCCCAAAIIIIAAAgggMJnAzUpLSLeIjH9uKXm+ybZAQggggAACCCCAAAIIIIAAAggggAACCCCweQGPCfkASR4jMgKRHjvyezcvAwACCCCAAAIIIIAAAggggAACCCCAAAIITCpwcUl+anYEIv3qp2pfc9KtkBgCCCCAAAIIIIAAAggggAACCCCAAAIIbF7gWpL+owpGPlPSRTcvAwACCCCAAAIIIIAAAggggAACCCCAAAIITCrg7tgfS8FId9d2t21332ZCAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQmEfgMSb+TApHuov0eSTeeJHUSQQABBBBAAAEEEEAAAQQQQAABBBBAAAEEisAVJL2iCka+WNLnIYQAAggggAACCCCAAAIIIIAAAggggAACCEwpcEtJH6iCkY+VdJ4pN0JaCCCAAAIIIIAAAggggAACCCCAAAIIILBtgXNIekgViPyIpLtsm4W9RwABBBBAAAEEEEAAAQQQQAABBBBAAIGpBS4p6TlVMPINkq429YZIDwEEEEAAAQQQQAABBBBAAAEEEEAAAQS2LXAdSW+rgpHPkHThbbOw9wgggAACCCCAAAIIIIAAAggggAACCCAwtcA9JH08BSM/JemnJJ1t6g2RHgIIIIAAAggggAACCCCAAAIIIIAAAghsV+ACkp6cApGflvQuSTfYLgl7jgACCCCAAAIIIIAAAggggAACCCCAAAKHEPhiSf9aBSP/QdLnHmJjpIkAAggggAACCCCAAAIIIIAAAggggAAC2xW4jaQPVsHIx0g693ZJ2HMEEEAAAQQQQAABBBBAAAEEEEAAAQQQmFrgnJIeXgUiPyzpjlNviPQQQAABBBBAAAEEEEAAAQQQQAABBBBAYNsCl5b0d1Uw8rWSrrxtFvYeAQQQQAABBBBAAAEEEEAAAQQQQAABBKYW+FpJ/1kFI58m6UJTb4j0EEAAAQQQQAABBBBAAAEEEEAAAQQQQGDbAj8s6RMpGPlJSffdNgl7jwACCCCAAAIIIIAAAggggAACCCCAAAJTC1xQ0lNTIPLTkt4p6XpTb4j0EEAAAQQQQAABBBBAAAEEEEAAAQQQQGDbAv9H0muqYOTzJV1m2yzsPQIIIIAAAggggAACCCCAAAIIIIAAAghMLfCtkj5UBSMfJelcU2+I9BBAAAEEEEAAAQQQQAABBBBAAAEEEEBguwIOOD6iCkQ6MHn77ZKw5wgggAACCCCAAAIIIIAAAggggAACCCBwCIHPlvSCKhj5KklXOsTGSBMBBBBAAAEEEEAAAQQQQAABBBBAAAEEtitwfUn/VQUjnyLJD7VhQgABBBBAAAEEEEAAAQQQQAABBBBAAAEEJhE4m6T7SPpkCkZ+QtK9J0mdRBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAgSJwIUlPT4HIT0t6u6SvQQgBBBBAAAEEEEAAAQQQQAABBBBAAAEEEJhS4CqSXlcFI58n6VJTboS0EEAAAQQQQAABBBBAAAEEEEAAAQQQQACBO0n6nyoY+UuSzgkNAggggAACCCCAAAIIIIAAAggggAACCCAwlcC5JT26CkT+t6RbT7UB0kEAAQQQQAABBBBAAAEEEEAAAQQQQAABBCxwWUn/WAUjXynpi+BBAAEEEEAAAQQQQAABBBBAAAEEEEAAAQSmFLihpHdXwcgnSTr/lBshLQQQQAABBBBAAAEEEEAAAQQQQAABBBDYtsDZJN1P0qdSMPLjku61bRb2HgEEEEAAAQQQQAABBBBAAAEEEEAAAQSmFriIpD9LgchPS3qrpK+aekOkhwACCCCAAAIIIIAAAggggAACCCCAAALbFri6pDdWwci/lvRZ22Zh7xFAAAEEEEAAAQQQQAABBBBAAAEEEEBgaoG7SPpICkb+r6SHSjrH1BsiPQQQQAABBBBAAAEEEEAAAQQQQAABBBDYrsB5Jf1mCkS6i/b7Jd1iuyTsOQIIIIAAAggggAACCCCAAAIIIIAAAggcQuDykl5SBSNfLukLD7Ex0kQAAQQQQAABBBBAAAEEEEAAAQQQQACB7QrcRNJ7q2DkEySdb7sk7DkCCCCAAAIIIIAAAggggAACCCCAAAIITC1wdkn3l+QxIt092/8+Jun7pt4Q6SGAAAIIIIAAAggggAACCCCAAAIIIIDAtgUuJukvUyDSwcg3S/qKbbOw9wgggAACCCCAAAIIIIAAAggggAACCCAwtcCXl+BjtIr0q4OTDlIyIYAAAggggAACCCCAAAIIIIAAAggggAACkwl8d+mWHcFId9d+oCR332ZCAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQmEfADah5fddH2g2xuOknqJIIAAggggAACCCCAAAIIIIDA/2/vTsDv++Z6gb//SMg8JENmkVKuubiGKyoiCrcMKdJFc7dJiit0o5uQekiZky63blelFDIUF6mQeRZxZegiMvzr+Wivf8t25rPP+Q7ntZ/n+zvne87ea639Wnvv5/m+f2vvRYAAAQIECAwCV0ryN6Mw8pVJLk+IAAECBAgQIECAAAECBAgQIECAAAECUwrcJsmHR2Hk45Oca8pKlEWAAAECBAgQIECAAAECBAgQIECAwGELnD3JQ5LUMyLb8yI/nuQee2CpyXEuu8bPhffQJlUQILC6wGWG8/eLVt/Emp3AIr92bbxYt763BAgQIECAAAECBAgQIEDgxAvUH7rP7YLICiTfluSae9qz3xjV3QLRRa//kOTFwwQ7591TO1VDgMBsgX8ezuFnzP7ap0sE5vnVfxS16+Azl5ThawIECBAgQIAAAQIECBAgcGIEvibJ33V/9NYfv3+Q5EJ73INNAsn2R3q9vjvJ7ffYXlVtJnDvJI9K8vDNNrfVMRZogdofHuM2HuemzfM7yYGk8/04H3HaRoAAAQIECBAgQIAAgSMU+N4kn+zCyM8keUCSM/bcpj6QfGiSuy/4+b4h0PrTrt0VSlbbv3bP7VbdegLPHvrsE+ttZu0TIDAvUDsBTT8WTZznd5IDSef7sTi0NIIAAQIECBAgQIAAAQLHR6Ce8/a0UaBXt0B//RE1sQ8kb7xGG66T5OXdfrwhybnX2N6q+xUQUOzXe5+1zQvU9tmGk1zXPL8KJD84/DzshO2g8/2EdZjmEiBAgAABAgQIECBAYJcCV0nymi7Eq9GFL0tSkyoc1bJpIFnt/ZIk/6/bn7se1U6od6mAgGIp0YldYV6gdmJ3aM8NP41+zvc9H0SqI0CAAAECBAgQIECAwHEV+NYk/78L7yqMfGySLzziBm8TSFbT79Pt0y8c8b6ofr6AgGK+zUn/5jQGavvsk9Po53zf5xGkLgIECBAgQIAAAQIECBxDgXMkqdv9KoBsP/+U5G7HpK3bBpLX6/ar/ghetHxVktsl+aEkP5fkR5PcJcmXLdpohe8unuT6Sa44Z92p6z1fkgqYfyTJQ5LUszVvNKfu+rjaVpNMPHjY91smufSC9Wd9VXV+S2f3Y0m+PUnt+yrLJgHFtnXOa9ei/rpAklt1XrWftd/bHiPz2lKfT3l8nDNJ9W8d4/89yX2H/bnEogYs+O6SwznzwCQ/nuSOSS42Wn8XgdqVknxbkgcl+cHhkRJfMKp3k18X9X0rb8r+2Kff1OfLNsfSJuf7UZx7rc+9EiBAgAABAgQIECBAgMCEAnVL8wu6wK4CyTcn+eoJ69i2qG0DyUt1+1e3o89abpPkpd16LZjtX5+f5AazNk5y3iRnDj8VYtZSz+KssKqfpfx/DN+1l13U+5NJ/nHOvvxtknq2ZlvqmZzjW/TbPn8qSU0itOy5mxVo/1KSj82ps8qpZ5KW0Xj5SOfW6q3XZlmvzxpvlGSbOqu4TfrrcknqWJy3n9Xu30lytRnt3fSjbY6PCgWb42OGBtREVe+Z00/V/l9LUqHVKksdZ++aU1YFkE9OcpGhoKkCyfafJ/XcxP54ae/fnuTOQ53VV23/L9jt0CZ93zbfpj9aGe11Cr96hmTbx2e0gme8bnu+THUsbXq+H8W5N4PRRwQIECBAgAABAgQIECAwhcANZ4QTv5ekRqEcp2XbQPI/duFF7d94qVFdLdBor+9NUpPgfGD0XQV9Vx8XMARcbdsKJC+a5BWjbev7PpDcRb01ErK1o14/NPz0n1Uo8B+G4KbCwvbdx4eAqWYkb5/V6x/P2N/20fmH7/v16335jcupCYbKpV8+OqprXE79XkFwv2xbZ5VVoVSra5X+qtF44/DtfUnemGS8D/X7V/QN3vD9tsdHhUhtHytorFCy/V6vNQr670ef1ecV6rUgcVbTK6D+rRnb1bnyydHn9Z8bV04yRSBZIwlfNCq/2lvPiK1Qrt+3+yd5fPfZOJBs667S981g2/5o5Uzpt8os21OcL1MdS+NzpfVD/zo+34/i3Gt95ZUAAQIECBAgQIAAAQIEJhao2zX7MOrTSX4qyRkT1zNFcdsGkj/RBRN1+3K/3KILzmrkW912Og5jrpWkgsz2R/Nb+wKG933AVbfA/nW3/oeT/FGSen7lVYf1d1FvhTUVAtZPBTL1h3xb6hbT13Zt+stuFOWfDrds1+2XtdTrf0lSwWXb57pFebzUSLp+dGUFdHULbbtFu76v/Xx1V87rh4mGWlkVit9h+Hn3sF6FWrfvfvqZ1aeos+pet7+q3c3id4dbqNs+1GvdDv/Cbp0Ks2uE7KbLFMdHHyL1wXqNVq1QvcKsWi48PGe1hYa1n/W4gllLXR/+vNvPCrzrGa01GVYt50lykyF4byHhiycIJKu/+pHGbxkeB9AeK1D7cNtR2/r15wWSq5yrtV9T9EeVM7XfskByqvNlqmNp3fO9zPZ97lWdFgIECBAgQIAAAQIECBCYWKD+sP+fXaBQ4UONMLrZxPVMWdw2gWQFJf2onAq6+uVPOovxd/169Wy6GrnTQqm6hbBf+oDrbd16T53xPL3abhf1VtsqVKrbSmctFSrUqLi2D/X6h0nONWvl4Rmibd3/M2Oduh29fV8BZwuHxqtWMFe3Mrd163mls5ZVnik3VZ3r9NcVurZXG882q/HD5E8v6dad1w9zNv+cj6c4PvoQqdnXc0LnLTfv2l6TW5XReKnwuJX1/iTXGK/Q/X6nGSMm63jbZPlvXb0V9tejJmYtdaw9r1u3tXVeILnKuVr1TNEfVc7UfssCyanOl10cS6uc70dx7s06rnxGgAABAgQIECBAgAABAlsI1LPtXjf6Y70ClHlB0hZVTbrpJoFkjUT6z0n6wKGeEdlGhVUD67lqLays52guW767s6sJY/qlD7haCFIjx2Ytu6z3cbMq7D57QrcPNYKt/uCft1Q7K6yu/anRlf1SM6+3ZxFWCNqPxuzXa+9r5FwbAVmj9WY9l3JZQDFlnev01907s3u2HZrzWqM5W//PC17nbHrWx1MdH+MQadbjCs6qdHjTB6rj58jWOfWqbv9qsqhly3d165fLJoFkjbhto3UrKK3RkIuWOrbeMap3XiDZ+mreuVr1TNUfu/BbFEhOeb5MfSyV67LzvdbZ97m36LjyHQECBAgQIECAAAECBAhsIFDhXPujvv0R/ugkU8xKu0Fz1tqkDyTrdtm67XreT02uUqP56rbqtp/1+okkXz6qtX5v69Tt1MuW/9St/4DRyuOAa9FzF3dVbwWM7ZbwUfPO+rVm3W77vEoIW7fb1vr1fMkKVNrSh7O/2D5c8tqHC/eYse6ygGLKOtfpr7t2ZuNb/se7UWFY3a5dz05c5Zgab1+/T3V8jEOkVWYB75+TOL5N/xs6h7o9fZWlwrz+2ZubBJJ9mx6xSqXDzOHtOK/XRYHkonO1qpuqP3bhtyiQnPJ8mfpYKtdl53uts+9zb8XDy2oECBAgQIAAAQIECBAgsEygAscK6fo/zmtUYN1OeVKWPpDs92PV9zW5zLzRXJcYRohWQLVsuV/n+ODRyuOAqz1Tb7TaWb/uot56Zt6ypWYgbm7LwrUqq7+9vx/VWCMxWzkV2KyylFF7ruAzZ2ywLKCYss51+qsmqGn7Ws+3/I4ZbZ/6oymOjz5EqtGpqyz98TEeDdo/i/WOqxQ2rFOzSTe/TQLJJ3bbLxrR2zep9r2fYGdRILnsXK1yp+iPXfgtCiSnPF+mPpbKdNn5XuscxbnXH0feEyBAgAABAgQIECBAgMAGAjUrbT8BRYUCNXprihmAN2jOxpv0gWRNvlOT8Sz6qYk26vbsJyWpW0bnPfNvUYNqmy9NUjN03y3Jr48CjkWB5AcXFbzku23qrb5etnx7F+58/7KVk/xmt34fSLbn9NVzBPuRk8uKrJGDdRzWyMvxsiygmLLOPpBc1l/VJzUpUQvV6rVuXa6A6ZrDLb3jfdnV7+scH32IVLarLDV5S9vP8W3Mj+2+u9QqhQ3r1EQmrcxNAsl2G3nNcL/O8qau3nmB5LK+X1bfOv2xC79FgeSU58vUx1K5Ljvfa53jdO4tOxZ8T4AAAQIECBAgQIAAAQLDTLfv7f4gr0DgfyU5/wnU6QPJfsblKXelQtoKmGrkXoVm/eiqFqb0r4sCyZr8ZtVlynp/e4VK+0Cy3i9b5gWSbx8dW73NKu/LeLwsCyimrLMPJFfprwslec6cfa4RxxX+1MQrN9gwAB9btN+3OT76EOnXWoFLXr+u28dxIFmzsVff1rlRQdGqy2W6MjcJJCs0rHorBF5neW5X77xAcpW+7+vcpj924bcokJzyfJn6WCrTZed7cz+qc6/V75UAAQIECBAgQIAAAQIEVhSomVVrJGELhmo0YX12UpddBpI16u8xnVUza681KutlSR4/GLbPFwWST1kB+qjq7QPJVW67fVpn04+QrIlsmsUmrxXijZdlAcWUdfaB5Cr91dr6jUme0U2GNGvfX5nka9sGG75OcXz0IVKNzltlWRRIvmbo85pYZp2lJldpTpsEklVfbb/KCOC+XTWJT6t3XiC5at9P0R+78FsUSE55vkx9LFU/LTvf+76s9/s698b1+p0AAQIECBAgQIAAAQIElgjU6AZ7FM8AAB+uSURBVMea9KX9EV6vNUpyV6MKlzRnsq93FUjWLdmvHnm9KMnPJqkJPeq5cf1So6Oa7TaB5FHVW/syVSD5zsGiZuH+lg1/ett6vyygmLLOTQPJ1uZzDbfz14Qr/ztJBdft2KjXGkVYoyU3WaY6PqYOkXYxwm8Vn3aO1uQ46yx/0/XJNoHkVP2xC79FgeSU58vUx1L147LzfV5f7/Lcm1enzwkQIECAAAECBAgQIEBgjsDVk7yx+wO8QpEK18ah2pzNj/XHuwok+3Ir9Lj5EoXrdL7bBJJHVW/t3lSBZM3QXcdYBZJTLcsCiinr3DaQHO9zTR5VwWwLz8qmJpJZ59bmVuZUx8fUIVLd9l37VT/7fIbks4Y6P5OkwqhVlnL/SNfebQLJqfpjF36LAskpz5epj6Xqw2Xn+yr9XOtMee6tWqf1CBAgQIAAAQIECBAgQCDJXZJ8rPvjuwKDmln7HKdEpw8Ephzt2Z6xWc+oqz+4ly336oy3CSSPqt7av6kCySd0FjV50qpLPafzoUnuP2ODZQHFlHWuGkieJ8kPDD9fP6PN44/Ol+SvOpv6j4J1l6mOj6lDpF3MEr2KzS93nndfZYMkt++2qevhNoHkVP2xC79FgeSU58vUx1J147Lz/SjOvRUPL6sRIECAAAECBAgQIEDgsAXOmeTRoz+8a1TQHU4Zyy4CyRpp1UZ71WQlqyxP7LbZNJA8qnrb/k0VSP5IZ/HzrfAlr/0t7/V8v/GyLKCYss5VA8kK9dtIu7oFeJXlhzqb71xlg26dKY+PqUOkb+j264Vdmxe9Lb+61bqda5s8Q/Km3fZvWHHUaT3Hs9W5TSA5ZX/swm9RIDnl+TL1sVTHzLLzfd/n3qLj2HcECBAgQIAAAQIECBAgMAjUc81eOvqj+7VJrnoKhXYRSNYf8v80+L1uBbOvGc28vWkgeVT1tl2cKpCsEWftuYn1euFWwYLXPtCtEWzjpQUUNQlTOY2XKetcNZCsNvzRcJycmaTOu2XLQ7rz8mbLVh59P+XxMXWIdMYw03UL+q47avusXyuQbevX6yaBZJVbj59o5Tw8SbVl3lLPgG3rttdNR0hO2R+78Kv2tX185ghkyvNl6mOpmrrsfK919nnujfj8SoAAAQIECBAgQIAAAQJjgZoJ9/3dH6L1B+nTk1TIchqXXQSS5fSSzvC2C+CuOBrlVd41M3e/rBNwHVW91d6pAskqq0ZGtjDkr5N8SQ8yel+32rZ167mTNfPyeKnZq9s6Vxl/Ofw+VZ3r9Nd9u3bVjOs1am7ectkk7xjW//iSdeeVMdXxsYsQqUZftz6qfvzqeTsxHGvjmZ43DSRv0dVb9Vf4dtFR3RdI0q4V9bzJ93XbbBpIVhVT9UeVNbXfokCy6pvqfNnFsbTK+b7vc290SPmVAAECBAgQIECAAAECBEqgRtj8VJL6Y7uFAjWbb90iepqXFjLUPk/5DMkHdY51W+4PJqnb4Gsp6ysk+cUkLVSpUVofHbb5QJL7JWkj4NYJuI6q3tqvKQPJiw+zuLdj8a1J7pzk0v9G+Nl/K6CrZ9nV6MJar47dCpdmLY/q+uNJSa6RpG7zrnraMlWd6/RXhVlv7tr2miTfOkym0dpVM9x/d5L3dOv9Vvtyzdepjo9dhEg1WcxfdPtYz1+tZ6teedjHcye5UZKHdX3+piRlVv2/aSBZxX/vaJRylff6JL+fpG7R7q+LP5zk8V07twkkp+qP2oep/ZYFklOdL7s4llY53/d97q15qlqdAAECBAgQIECAAAECp1+g/jBrs822AKhm8b3B6d/1s0Y91X5PGUhW+Di+7b3CxwrW2u3czfpVw4zlz+1CjvrukYP/OgHXUdVbTZ0ykKzyKnx8+cikXOrYbLd0N8MKJX9ywfE6HgXXths/43OKOtfpr2pyTU7TwujWrk8neVuSt88Iyup21EUjKRcwfDYUn+K43EWIVO2u0LHC1ubQXmvUdv0HSfu9XiuMvFqSGllav28TSFbd10/yd6M6+vrqGHvEgFujKOu7alP7j4b6at2+n/J8ndpvWSBZ9U1xvuziWFr1fN/nuTccOl4IECBAgAABAgQIECBAoARqpNhbRn+EP380cuw0S/UjaaYMJMvsi5P8apJPjHxbyFHB5GOT1OzJtbRwpcKoCjpq9FQtdQtyG0n5lOGzRS9HVW8fSN5qUQOH757WuVQQNWup4K1uYe9vkW1+7bVGl67yzMEa7fvOrs7avo718bJtnev2V9Vfz4/8lQXHSrW1Jl35r3NuSR/vw6Lfpzg+atRmPY+z2lXPXVxlqcdBtD6725INfmwIZNv6/WuF0RXWt5GJ1Yf1/baBZDWpjsMKJu+T5HFJXpykJkn6hWFEbWt2C0GrLf2ySd9P0R99G+r9FH414rL950mNKp63bHu+7OpYWvV83+e5N8/Q5wQIECBAgAABAgQIEDgogZoUop5F1/+xX7dD1sgYy3QC9fzDCujqttD7D7ehflMXqExX0+eWdFT1fm4rpvutgoPbJfnpJPdOcvPh1vd1aqhju8r5yiT1LMn+lu1Z5UxR56xy5312kSQ3TPIdw37W7cH1bMBrzdtgi89PwvFRo+dumaSe9/d9SWo26RqFeNTLe4frZo1unmrZRX/s22/f58sy+3XO932ee8va7XsCBAgQIECAAAECBAicSoEaxVMj8/ogskb6VNhjIUCAwKEIXCrJbw63id9xxZ2uyXbatbMedWEhQIAAAQIECBAgQIAAAQIElgjURCCv6P6grj+sX91NHLFkc18TIEDg1AjUbNr1iIS6DtYkNqssv91dP390lQ2sQ4AAAQIECBAgQIAAAQIEDlmgbnesGZzb6J56fWqS8xwyin0nQOCgBdozKOt6WI+sOGOBxvd0M27Xc01dOxdg+YoAAQIECBAgQIAAAQIEDlugJid4QPeHdP3hXZOk1DMNLQQIEDhkgfqPmn4G7+cluf0wKVU9g/Byw3MsnzH6z5wfOWQ0+06AAAECBAgQIECAAAECBBYJXDjJs0d/SL8ryfUWbeQ7AgQIHJDAbbqZ7PsR5LPeV3hZEyuZ/OuADhC7SoAAAQIECBAgQIAAAQKrC9TMvG8fhZF/kqRmXrUQIECAwL8LXDnJI5J8eHTNbKHkR5K8cEcznv97K7wjQIAAAQIECBAgQIAAAQInWOCeST7R/WF9ZpKHJKnbty0ECBAgMFvgC5NcJcnNk9x1eL307FV9SoAAAQIECBAgQIAAAQIECJTAuZM8oQsia3TPh5LcGg8BAgQIECBAgAABAgQIECBAgAABAgSmFLhCkr8ahZH1e31uIUCAAAECBAgQIECAAAECBAgQIECAwGQC3zSMhGzPO6vXGilZIyYtBAgQIECAAAECBAgQIECAAAECBAgQmESgngn54CT1jMgWRtazI79nktIVQoAAAQIECBAgQIAAAQIECBAgQIAAgUHgoklq1uwWRNZrzap9bUIECBAgQIAAAQIECBAgQIAAAQIECBCYUuB6Sd45CiOfneTCU1aiLAIECBAgQIAAAQIECBAgQIAAAQIECNwnyT93YWTdrv3AJHX7toUAAQIECBAgQIAAAQIECBAgQIAAAQKTCJwnyVO6ILJu0f5Akm+cpHSFECBAgAABAgQIECBAgAABAgQIECBAYBC4cpJXjcLIVyS5HCECBAgQIECAAAECBAgQIECAAAECBAhMKXC7JP84CiMfl+QLp6xEWQQIECBAgAABAgQIECBAgAABAgQIHLbA2ZM8dBREfjzJdx02i70nQIAAAQIECBAgQIAAAQIECBAgQGBqgYsnef4ojHxLkmtMXZHyCBAgQIAAAQIECBAgQIAAAQIECBA4bIEbJHn3KIx8VpILHjaLvSdAgAABAgQIECBAgAABAgQIECBAYGqBH0zyyS6M/EySn05yxtQVKY8AAQIECBAgQIAAAQIECBAgQIAAgcMVOG+Sp3dB5L8keX+Smx8uiT0nQIAAAQIECBAgQIAAAQIECBAgQGAXAldN8tpRGPnSJF+6i8qUSYAAAQIECBAgQIAAAQIECBAgQIDA4QrcIclHRmHkryY55+GS2HMCBAgQIECAAAECBAgQIECAAAECBKYWOEeSh4+CyI8lucvUFSmPAAECBAgQIECAAAECBAgQIECAAIHDFrhEkheNwsg3Jrn6YbPYewIECBAgQIAAAQIECBAgQIAAAQIEpha4UZK/H4WRv5vk/FNXpDwCBAgQIECAAAECBAgQIECAAAECBA5b4EeTfKoLIz+d5CcOm8TeEyBAgAABAgQIECBAgAABAgQIECCwicAZSe4xZ8PzJXlmF0T+S5L3JbnpnPV9TIAAAQIECBAgQIAAAQIECBAgQIAAgYUCt0tyZpJvHq31FUneMAoj/zzJJUfr+ZUAAQIECBAgQIAAAQIECBAgQIAAAQIrC7x8CB0/nORKw1Z3SvLRURj5yCRfsHKpViRAgAABAgQIECBAgAABAgQIECBAgMBI4Baj0PFvkjx69FkFk9822s6vBAgQIECAAAECBAgQIECAAAECBAgQWFvgBaPwsZ4R2f+8LsnV1i7VBgQIECBAgAABAgQIECBAgAABAgQIEBgJ3HAUPvZBZL1/RpKa1MZCgAABAgQIECBAgAABAgQIECBAgACBrQWevSCQ/EyS625dgwIIECBAgAABAgQIECBAgAABAgQIECCQ5JoLwsg2UvIdSS5KiwABAgQIECBAgAABAgQIECBAgAABAtsKPHOFQLKCyT9OcrZtK7M9AQIECBAgQIAAAQIECBAgQIAAAQKHK/DlSc5cMZB8f5LbHi6VPSdAgAABAgQIECBAgAABAgQIECBAYFuBJ88JI9+XpJ4r+eAkt0tymW0rsj0BAgQIECBAgAABAgQIECBAgMDhCPzZnNCpPR/Qa8KAwaEfA28/nEuiPSVAgAABAgQIECBAgAABAgR2LXDoQYv9FzY6BlY7BnZ9LVI+AQIECBAgQIAAAQIECBAgcCACLYw5kN21mwQIrCngGrEmmNUJECBAgAABAgQIECBAgACBxQLChsU+viVw6AKuEYd+BNh/AgQIECBAgAABAgQIECAwsYCwYWJQxRE4ZQKuEaesQ+0OAQIECBAgQIAAAQIECBA4agFhw1H3gPoJHG8B14jj3T9aR4AAAQIECBAgQIAAAQIETpyAsOHEdZkGE9irgGvEXrlVRoAAAQIECBAgQIAAAQIETr+AsOH097E9JLCNgGvENnq2JUCAAAECBAgQIECAAAECBD5PQNjweSQ+IECgE3CN6DC8JUCAAAECBAgQIECAAAECBLYXEDZsb6gEAqdZwDXiNPeufSNAgAABAgQIECBAgAABAkcgIGw4AnRVEjhBAq4RJ6izNJUAAQIECBAgQIAAAQIECJwEAWHDSeglbSRwdAKuEUdnr2YCBAgQIECAAAECBAgQIHAqBYQNp7Jb7RSByQRcIyajVBABAgQIECBAgAABAgQIECBQAsIGxwEBAosEXCMW6fiOAAECBAgQIECAAAECBAgQWFtA2LA2mQ0IHJSAa8RBdbedJUCAAAECBAgQIECAAAECuxcQNuzeWA0ETrKAa8RJ7j1tJ0CAAAECBAgQIECAAAECx1BA2HAMO0WTCBwjAdeIY9QZmkKAAAECBAgQIECAAAECBE6DgLDhNPSifSCwOwHXiN3ZKpkAAQIECBAgQIAAAQIECBykgLDhILvdThNYWcA1YmUqKxIgQIAAAQIECBAgQIAAAQKrCAgbVlGyDoHDFXCNONy+t+cECBAgQIAAAQIECBAgQGAnAsKGnbAqlMCpEXCNODVdaUcIECBAgAABAgQIECBAgMDxEBA2HI9+0AoCx1XANeK49ox2ESBAgAABAgQIECBAgACBEyogbDihHafZBPYk4BqxJ2jVECBAgAABAgQIECBAgACBQxEQNhxKT9tPApsJuEZs5mYrAgQIECBAgAABAgQIECBAYI6AsGEOjI8JEPisgGuEA4EAAQIECBAgQIAAAQIECBCYVEDYMCmnwgicOgHXiFPXpXaIAAECBAgQIECAAAECBAgcrYCw4Wj91U7guAu4Rhz3HtI+AgQIECBAgAABAgQIECBwwgSEDSeswzSXwJ4FXCP2DK46AgQIECBAgAABAgQIECBw2gWEDae9h+0fge0EXCO287M1AQIECBAgQIAAAQIECBAgMBIQNoxA/EqAwOcIuEZ8DodfCBAgQIAAAQIECBAgQIAAgW0FhA3bCtqewOkWcI043f1r7wgQIECAAAECBAgQIECAwN4FhA17J1chgRMl4BpxorpLYwkQIECAAAECBAgQIECAwPEXEDYc/z7SQgJHKeAacZT66iZAgAABAgQIECBAgAABAqdQQNhwCjvVLhGYUMA1YkJMRREgQIAAAQIECBAgQIAAAQKJsGE3R8HZk1wryb2TPD7Jy5I8I8mPJblJkvPsptqFpX5HkvskueLCtbb/cl/1bN9SJawi4BqxipJ1CBAgQIAAAQIECBAgQIAAgZUFhA0rU6284sWTPD//HvY24/717UmuvXKJq694+yT3mlP2p4Y23WH14jZac1/1bNQ4G60t0I7btTe0AQECBAgQIECAAAECBAgQIEBgloCwYZbK5p99bZJ3d2Hk3yX57ST3TfK4JH+b5Mzh+08kuefmVc3c8q+Gsh8449t9BYX7qmfGLvpoBwKuETtAVSQBAgQIECBAgAABAgQIEDhkAWHDdL1//iT/0IWRPzOn6Fsl+Wi33vXnrLfJx4sCyTsluUuSS25S8Brb7KueNZpk1S0EXCO2wLMpAQIECBAgQIAAAQIECBAg8PkCwobPN9n0k4d0IeP3LCmkni/5sWH9Fy5Zd52vFwWS65RjXQJNwDWiSXglQIAAAQIECBAgQIAAAQIEJhEQNkzCmHpuZAsYX7likY/sAsxbdtvcNcnvJnlY8tlyH5XkL5L8Y5JXJ/mNJF/XrV9vnzxs8+GhzNcPvz+zW++3hs/qtvK21CQ3VVc9d/KiSX6hq+slSR6UpCboqeU7h3Xfn+S9SV6U5NuG7/qXcT0/MGxX9Sz6qZC2X6o9ZfCCJLVfFbb+UpKb9isN7394KPvOSc423ApfkwhVWy83Y30frS7gGrG6lTUJECBAgAABAgQIECBAgACBFQSEDSsgrbDKt3Th4qyQblYRl03SnrdYQVtbfn4o60NJ3tmV2/qqXj+T5H5Jzhg26m8V79f7ZCu0q6uf1ObXhvJ/P8nr5tRVIWX99OX272sm8X5p+9TqedKCbfty6lb2ttx49CzOfr16Bufd24rDawWvtU6NUq0Qs1//8qN1/bqeQLNcbytrEyBAgAABAgQIECBAgAABAgTmCAgb5sCs+XFNWtMsL7bGtjWSsbb7w26bFki28p6X5I5JagThPZK8raurhZ8ViH539wzLZw2/98HdOCisKlsgWXVV0Fdh3g2T3CjJS7t66vu/THKbJF+ZpEZxVmBan7+1a3u9HddznaEt1b7+5ye78l+V5LxDOeXXyq59redefnWS2yWp0aetrd/U1dsCyTcP31cQ+5wkD09yrm49b9cXKO/6sRAgQIAAAQIECBAgQIAAAQIEJhEQNkzCmCcOoc1H1iyugsjqg7d02/WBZI1c/ILuu3pbt4fX+rVdBZrtlur6btEzJMdBYa3fB5LjkY5fNtRR9VQw+EWjdvz48H0Fmf13s+oZbfrZffqzYfvxbdW/Pnz+niQXGm1YFi8bvq/t29ICyWprBaTXbl943VqgTOvHQoAAAQIECBAgQIAAAQIECBCYREDYMAljfmcIbd63ZnFPG7arZyS2pQ8kr94+HL3WSMPWdzUCsS2bBpJ1a3g9e3G8tOdi/tz4iyS36Npwke77VQLJxw7b1kjGGo3ZL/WszNq3evbkrOVmXb1XGlboA8lbz9rIZxsLtONs4wJsSIAAAQIECBAgQIAAAQIECBDoBYQNvcbm7x8xhGSfnjGicVGpLx62q9uh29ICyfGt0O37eq1Rkq3v7tR9sWkg+UddGf3b9mzKWbOGV5DY2rBOIPn93Xbjci/dffeKJDUxzfjnD7p1vnlobAsk39U33vtJBFofT1KYQggQIECAAAECBAgQIECAAAECwoZpjoG63blZXmXFImtCmpqturarkZJtaYFkhZWLllZfTW7Tlk0DyWe3AkavLZC85+jz+nWTQPLmSSq0rbY/ekaZN+kc2/4tem3taoFkfxv3jOJ9tIFA899gU5sQIECAAAECBAgQIECAAAECBD5fQNjw+SabfNIHaQ9dsYCaVbr5/0y3TQskX9t9Nn5bE8C0bftnPx7nQLKeSfnBod3PTXKO8U4luWq3X+V4ryU/Vx7KaIHk02eU6aPtBNpxtl0ptiZAgAABAgQIECBAgAABAgQIDALChmkOhXr+4quHMK2egXiBFYp94bD+x5PUrcptaYHkJ5L0t0K37+v1pl1wV89ybMtxDSQvOEzAU8dbzYR94dbg0WuFlPVcyVrv20fftV9rYpvrDj8t1GyB5FPbSl4nE3CNmIxSQQQIECBAgAABAgQIECBAgEAJCBumOw76EY916/B4huhWU4WXj+zsxyMqWyBZffOzbaPRa5tEp8LMi3XftUBy1nazJptps2zv8pbtmgW8nlFZ+1Nh7dW69s5624Lalyep29rHSz+796WGLwWSY6XpfneNmM5SSQQIECBAgAABAgQIECBAgEAXisGYRqCCvRbg1C3Xt0ly0aHo8ySpW7t/r1unJmGp0YP90geSZyb5qSR1i3YttW4fZj58+Ly9vGQouyZ++dLRCMujCiTbhD/l8p2DR5mMf9o+3rjzKatLDjt37iQV+n5k+P55baeTCCQ7jInftuN54mIVR4AAAQIECBAgQIAAAQIECByqgLBh2p6v24n7wLD5VvDYAsH2WT1H8YtnVN8Hkm0CmM8keUeSCijb9n/dhZ2tmKd039d6dftzW1r9d2gfJNn1CMlLjNrT2j7r9Uldu355tN17ktRo0LbdB5LUMynbIpBsEtO/NvPpS1YiAQIECBAgQIAAAQIECBAgcJACwobddPsdk/xFkn/qQrSyroDxVUnun6RuZZ61tEDy/ya5dZI3jMqoMO4xSWrE4HipSV7qlucW3n2sW6EFkrftPvuVoextbtmukLR/ZmZfTz0bsx1jy177QLKaWM+QfNMohK0yatTkV3T7UG8FkiOQCX9t/TZhkYoiQIAAAQIECBAgQIAAAQIEDllA2LDb3q9JV74qyZ2S3CBJ3ba9bOkDybbuFZLUrcyXbx8c0Ov5klw/yTWS1HvLfgVcI/brrTYCBAgQIECAAAECBAgQIHDqBYQNx6+LZwWSx6+VWnQoAq4Rh9LT9pMAAQIECBAgQIAAAQIECOxJQNiwJ+g1qhFIroFl1Z0LuEbsnFgFBAgQIECAAAECBAgQIEDgsASEDcevvwWSx69PDrlFrhGH3Pv2nQABAgQIECBAgAABAgQI7EBA2LAD1C2LrMlcnpPkUVuWY3MCUwi4RkyhqAwCBAgQIECAAAECBAgQIEDgLAFhw1kU3hAgMEPANWIGio8IECBAgAABAgQIECBAgACBzQWEDZvb2ZLAIQi4RhxCL9tHAgQIECBAgAABAgQIECCwRwFhwx6xVUXgBAq4RpzATtNkAgQIECBAgAABAgQIECBwnAWEDce5d7SNwNELuEYcfR9oAQECBAgQIECAAAECBAgQOFUCwoZT1Z12hsDkAq4Rk5MqkAABAgQIECBAgAABAgQIHLaAsOGw+9/eE1gm4BqxTMj3BAgQIECAAAECBAgQIECAwFoCwoa1uKxM4OAEXCMOrsvtMAECBAgQIECAAAECBAgQ2K2AsGG3vkoncNIFXCNOeg9qPwECBAgQIECAAAECBAgQOGYCwoZj1iGaQ+CYCbhGHLMO0RwCBAgQIECAAAECBAgQIHDSBYQNJ70HtZ/AbgVcI3brq3QCBAgQIECAAAECBAgQIHBwAsKGg+tyO0xgLQHXiLW4rEyAAAECBAgQIECAAAECBAgsExA2LBPyPYHDFnCNOOz+t/cECBAgQIAAAQIECBAgQGByAWHD5KQKJHCqBFwjTlV32hkCBAgQIECAAAECBAgQIHD0AsKGo+8DLSBwnAVcI45z72gbAQIECBAgQIAAAQIECBA4gQLChhPYaZpMYI8CrhF7xFYVAQIECBAgQIAAAQIECBA4BAFhwyH0sn0ksLmAa8TmdrYkQIAAAQIECBAgQIAAAQIEZggIG2ag+IgAgbMEXCPOovCGAAECBAgQIECAAAECBAgQmEJA2DCFojIInF4B14jT27f2jAABAgQIECBAgAABAgQIHImAsOFI2FVK4MQIuEacmK7SUAIECBAgQIAAAQIECBAgcDIEhA0no5+0ksBRCbhGHJW8egkQIECAAAECBAgQIECAwCkVEDac0o61WwQmEnCNmAhSMQQIECBAgAABAgQIECBAgMC/CQgbHAkECCwScI1YpOM7AgQIECBAgAABAgQIECBAYG0BYcPaZDYgcFACrhEH1d12lgABAgQIECBAgAABAgQI7F5A2LB7YzUQOMkCrhEnufe0nQABAgQIECBAgAABAgQIHEMBYcMx7BRNInCMBFwjjlFnaAoBAgQIECBAgAABAgQIEDgNAsKG09CL9oHA7gRcI3Znq2QCBAgQIECAAAECBAgQIHCQAsKGg+x2O01gZQHXiJWprEiAAAECBAgQIECAAAECBAisItDCBq8JAwaOgfnHwCrXE+sQIECAAAECBAgQIECAAAECBJYKPDHzAwjhDBvHgGOgjoE/W3olsQIBAgQIECBAgAABAgROmMC/AlCa4V9mPJ5JAAAAAElFTkSuQmCC)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tMslg-JukaAu"
      },
      "source": [
        "### nn.Module interface\n",
        "\n",
        "Let us start with the Module interface. We first introduce a parameter class to indicate a Tensor is a trainable parameter."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yQVcEHP83ock"
      },
      "source": [
        "class Parameter(ndl.Tensor):\n",
        "    \"\"\"parameter\"\"\""
      ],
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_k-zVkXtX6nM"
      },
      "source": [
        "def _get_params(value):\n",
        "    if isinstance(value, Parameter):\n",
        "        return [value]\n",
        "    if isinstance(value, dict):\n",
        "        params = []\n",
        "        for k, v in value.items():\n",
        "            params += _get_params(v)\n",
        "        return params\n",
        "    if isinstance(value, Module):\n",
        "        return value.parameters()\n",
        "    return []\n",
        "\n",
        "class Module:\n",
        "    def parameters(self):\n",
        "        return _get_params(self.__dict__)\n",
        "\n",
        "    def __call__(self, *args, **kwargs):\n",
        "        return self.forward(*args, **kwargs)"
      ],
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sK_6tp254BDh"
      },
      "source": [
        "Now that we have the base Module interface, we can start to define different kind of modules. Let us define a simple scale add module, that computes $y = x \\times s + b$. The ScaleAdd is parameterized by $s$ and $b$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S9HCfsB6X6yS"
      },
      "source": [
        "class ScaleAdd(Module):\n",
        "    def __init__(self, init_s=1, init_b=0):\n",
        "        self.s = Parameter([init_s], dtype=\"float32\")\n",
        "        self.b = Parameter([init_b], dtype=\"float32\")\n",
        "\n",
        "    def forward(self, x):\n",
        "        return x * self.s + self.b\n"
      ],
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iiECrOzX6iLg"
      },
      "source": [
        "We allow a module to contain multiple submodules inside and compose them together"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-bI6WGhPX65e"
      },
      "source": [
        "class MultiPathScaleAdd(Module):\n",
        "    def __init__(self):\n",
        "        self.path0 = ScaleAdd()\n",
        "        self.path1 = ScaleAdd()\n",
        "\n",
        "    def forward(self, x):\n",
        "        return self.path0(x) + self.path1(x)"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "mpath = MultiPathScaleAdd()\n",
        "mpath.parameters()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CVbA1IqOawDz",
        "outputId": "6b796c05-1900-46b7-d5cb-6896d25e2225"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[needle.Tensor([1.]),\n",
              " needle.Tensor([0.]),\n",
              " needle.Tensor([1.]),\n",
              " needle.Tensor([0.])]"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qPlKFraI7E0z"
      },
      "source": [
        "### Loss function\n",
        "\n",
        "**NOTE**: You will need to fill in your implementation of autograd in hw1, restart the runtime in order to run the following piece of code."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pCqvVncW690P"
      },
      "source": [
        "class L2Loss(Module):\n",
        "    def forward(self, x ,y):\n",
        "        z = x + (-1) * y\n",
        "        return z * z"
      ],
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KbMtYmzI7Xu3"
      },
      "source": [
        "x = ndl.Tensor([2], dtype=\"float32\")\n",
        "y = ndl.Tensor([2], dtype=\"float32\")\n",
        "loss = L2Loss()(mpath(x), y)"
      ],
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xn1UEUWZ7q5Q"
      },
      "source": [
        "loss.backward()"
      ],
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UJaYYp0m-LT1"
      },
      "source": [
        "params = mpath.parameters()"
      ],
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AK55S7_bB74e"
      },
      "source": [
        "### Optimizer\n",
        "We are now ready to define the optimizer interface. There are two key functions here:\n",
        "\n",
        "- reset_grad: reset the gradient fields of each the the parameters\n",
        "- step: update the parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wxzM7saXEYx-"
      },
      "source": [],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pQKXuy3NCLEn"
      },
      "source": [
        "class Optimizer:\n",
        "    def __init__(self, params):\n",
        "        self.params = params\n",
        "\n",
        "    def reset_grad(self):\n",
        "        for p in self.params:\n",
        "            p.grad = None\n",
        "\n",
        "    def step(self):\n",
        "        raise NotImplemented()"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MZvoCqvqCT_p"
      },
      "source": [
        "class SGD(Optimizer):\n",
        "    def __init__(self, params, lr):\n",
        "        self.params = params\n",
        "        self.lr = lr\n",
        "\n",
        "    def step(self):\n",
        "        for w in self.params:\n",
        "            w.data = w.data + (-self.lr) * w.grad"
      ],
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "V4y9CmH6CsiW",
        "outputId": "adfb4647-1a48-41fa-a82a-a1f405d426c4"
      },
      "source": [
        "x = ndl.Tensor([2], dtype=\"float32\")\n",
        "y = ndl.Tensor([2], dtype=\"float32\")\n",
        "\n",
        "model = MultiPathScaleAdd()\n",
        "l2loss = L2Loss()\n",
        "opt = SGD(model.parameters(), lr=0.01)\n",
        "num_epoch = 10\n",
        "\n",
        "for epoch in range(num_epoch):\n",
        "    opt.reset_grad()\n",
        "    h = model(x)\n",
        "    loss = l2loss(h, y)\n",
        "    training_loss = loss.numpy()\n",
        "    loss.backward()\n",
        "    opt.step()\n",
        "\n",
        "    print(training_loss)"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[4.]\n",
            "[2.5600004]\n",
            "[1.6384]\n",
            "[1.0485764]\n",
            "[0.6710887]\n",
            "[0.4294967]\n",
            "[0.27487785]\n",
            "[0.17592174]\n",
            "[0.11258985]\n",
            "[0.07205748]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wk4UFfjXEGbd"
      },
      "source": [
        "### Initialization\n",
        "\n",
        "In the homework, you will need to implement the intialization function for the weight in linear transformations. Under a linear relu network where $y^{(l)} = x^{(l-1)} W^T, x^{(l)} = max(y^{(l)}, 0)$. Assume that $W\\in R^{n_{out} \\times n_{in}}$ A common way to do so is to intialize it as $\\mathcal{N}(0, \\sigma^2)$ where $\\sigma = \\sqrt{\\frac{2}{n_{in}}}$.\n",
        "\n",
        "Checkout Explaination from the original paper: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification\n",
        "\n",
        "\\begin{equation}\n",
        " y_i = \\sum_{j=1}^{n_{in}} x_j W_{i, j}   \n",
        "\\end{equation}\n",
        "\n",
        "\n",
        "\\begin{equation}\n",
        " \\mbox{Var}[y_i] =  n_{in} E[x_0^2] \\mbox{Var}[W_{i, j}] = n_{in} E[x_0^2] \\sigma^2\n",
        "\\end{equation}\n",
        "\n",
        "Considering the fact that x is also a result of relu of previous layer\n",
        "\n",
        "\\begin{equation}\n",
        " E[x_0^2] = E[relu(y^{(l-1)})^2] = \\frac{1}{2}  \\mbox{Var}[y^{(l-1)}]\n",
        "\\end{equation}\n",
        "\n",
        "We can get the variance value by requiring $\\mbox{Var}[y^{(l)}] = \\mbox{Var}[y^{(l-1)}]$.\n",
        "NOTE: the variance value was derived under a specific deep relu network."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_A1CrP14KvSW"
      },
      "source": [
        "## Additional contents on programming model\n",
        "\n",
        "In this section, we will review additional contents on autograd that we may need in future lectures.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oeODUJjm8J1Z"
      },
      "source": [
        "### Fused operator and Tuple Value\n",
        "\n",
        "Up until now each of the needle operator only returns a single output Tensor. In real world application scenarios, it is somtimes helpful to compute many outputs at once in a single (fused) operator.\n",
        "\n",
        "Needle is designed to support this feature. In order to do so, we need to introduce a new kind of Value -- Tuple.\n",
        "\n",
        "Open up the files on the left side panel, review the the following changes\n",
        "- `autograd.py`: The TensorTuple class\n",
        "- `ops/ops_tuple.py`: TupleGetItemOp and MakeTupleOp"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yOGfpeiL8JZZ"
      },
      "source": [
        "x = ndl.Tensor([1], dtype=\"float32\")"
      ],
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QWzIBukrf9X7"
      },
      "source": [
        "z = ndl.ops.fused_add_scalars(x, 1, 2)"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zg4jlJSygHYg"
      },
      "source": [
        "v0 = z[0]"
      ],
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KcfV71l0ggJa",
        "outputId": "d059b1ce-5c91-408d-97fa-e407519cb5fb"
      },
      "source": [
        "v0.op"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<needle.ops.ops_tuple.TupleGetItem at 0x7bcd44228160>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    }
  ]
}