Assignment_5_GVF.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Assignment 5 - GVF.ipynb",
      "version": "0.3.2",
      "views": {},
      "default_view": {},
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "bZEmFHASQubZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Assignment 5 - Track 2\n",
        "# Stéphanie Larocque et Philip Paquette\n",
        "\n",
        "## Q1 - Explain how SR can be expressed in the GVF framework (also see corresponding section in S&B if necessary). Expalin this connection using the GVF terminology: cumulant (this is not the notion of \"moments\" of a prob. distribution), termination condition, etc.\n",
        "\n",
        "------"
      ]
    },
    {
      "metadata": {
        "id": "RcpnWPimAVtB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "From Sutton\\&Barto's book, the General Value Function (GVF) equation is:\n",
        "\n",
        "$$v_{\\pi,C, \\gamma}(s) = \\mathbb{E}\\left[\\sum\\limits_{k=t}^\\infty C_{k+1}\\prod_{j=t+1}^{k}\\gamma(S_i)\\ \\bigg|\\  S_t = s, A_{t:\\infty}\\sim \\pi\\right]$$\n",
        "\n",
        "If we remove the dependency on the state for the discount factor, and keep a constant discount factor, we obtain:\n",
        "\n",
        "\\begin{align*}\n",
        "\tv_{\\pi, C, \\gamma}(s) \n",
        "\t&= \\mathbb{E}\\left[\\sum\\limits_{k=t}^\\infty \\gamma^{k-t}C_{k+1}\\ \\bigg|\\  S_t = s, A_{t:\\infty}\\sim \\pi\\right]\\\\\n",
        "\t&= \\mathbb{E}\\left[C(S_{t+1}) +\\gamma\\cdot v_{\\pi, C, \\gamma}(S_{t+1})\\ \\bigg|\\  S_t = s, A_{t:\\infty}\\sim \\pi\\right]\n",
        "\\end{align*}\n",
        "$$$$\n",
        "\n",
        "The function $C$ is the cumulant function, or pseudo-reward. It can be written in matrix form as\n",
        "$$ v_{\\pi, C, \\gamma} = C_\\pi + \\gamma P_\\pi v_{\\pi, C, \\gamma} $$\n",
        "\n",
        "and thus\n",
        "$$ v_{\\pi, C, \\gamma} = (I + \\gamma P_\\pi)^{-1}C_\\pi $$\n",
        "\n",
        "\n",
        "\n",
        "We find the SR value function \n",
        "$$v_\\pi(s)=\\phi_\\pi(s)^T r_\\pi$$\n",
        "with $\\phi_\\pi(s)^T = (I + \\gamma P_\\pi)^{-1}$ and $r_\\pi = C_\\pi$. The Successor Representation is thus a special case of General Value Function where the cumulant function is the reward function and with a constant discocunted function $\\gamma$ as termination function."
      ]
    },
    {
      "metadata": {
        "id": "BU1J49eMGcaB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Q2.1 - Learning SR"
      ]
    },
    {
      "metadata": {
        "id": "eBmpv47OVpNk",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "2c5bd1ff-49af-4040-839d-77a36986205b",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1523236185729,
          "user_tz": 240,
          "elapsed": 2803,
          "user": {
            "displayName": "Philip Paquette",
            "photoUrl": "//lh6.googleusercontent.com/-DGrBTu2a2aw/AAAAAAAAAAI/AAAAAAAAANs/SZizZJUV1nA/s50-c-k-no/photo.jpg",
            "userId": "101375949079620784780"
          }
        }
      },
      "cell_type": "code",
      "source": [
        "! pip install numpy\n",
        "import random\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "D0ElMvcPRL6R",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "base_uri": "https://localhost:8080/",
          "height": 503
        },
        "outputId": "161f3b30-8eed-4755-b5f2-f7bdf77d1114",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1523236477154,
          "user_tz": 240,
          "elapsed": 2379,
          "user": {
            "displayName": "Philip Paquette",
            "photoUrl": "//lh6.googleusercontent.com/-DGrBTu2a2aw/AAAAAAAAAAI/AAAAAAAAANs/SZizZJUV1nA/s50-c-k-no/photo.jpg",
            "userId": "101375949079620784780"
          }
        }
      },
      "cell_type": "code",
      "source": [
        "import random\n",
        "import numpy as np\n",
        "np.set_printoptions(precision=1, suppress=True)\n",
        "\n",
        "# Track 2 - Q1\n",
        "\n",
        "# Simple Grid World\n",
        "#\n",
        "#  _  _  _  G           G = Goal (+1), W = Wall, P = Pit (-1)\n",
        "#  _  W  _  P\n",
        "#  _  _  _  _\n",
        "\n",
        "env_model = {\n",
        "    'cell_0_0': {'UP': [(1., 0, 'cell_1_0')],\n",
        "                 'RIGHT': [(1., 0, 'cell_0_1')]},\n",
        "    'cell_0_1': {\"RIGHT\": [(1., 0., \"cell_0_2\")],\n",
        "                 \"LEFT\": [(1., 0., \"cell_0_0\")]},\n",
        "    'cell_0_2': {\"UP\": [(1.0, 0., \"cell_1_2\")],\n",
        "                 \"RIGHT\": [(1.0, 0., \"cell_0_3\")],\n",
        "                 \"LEFT\": [(1.0, 0., \"cell_0_1\")]},\n",
        "    'cell_0_3': {\"UP\": [(1.0, 0., \"pit\")],\n",
        "                 \"LEFT\": [(1.0, 0., \"cell_0_2\")]},\n",
        "    'cell_1_0': {\"UP\": [(1.0, 0., \"cell_2_0\")],\n",
        "                 \"DOWN\": [(1.0, 0., \"cell_0_0\")]},\n",
        "    'cell_1_2': {\"UP\": [(1.0, 0., \"cell_2_2\")],\n",
        "                 \"RIGHT\": [(1.0, 0., \"pit\")],\n",
        "                 \"DOWN\": [(1.0, 0., \"cell_0_2\")]},\n",
        "    'pit': {\"EXIT\": [(1.0, -1., \"terminal\")]},\n",
        "    \"terminal\": {\"\": [(1., 0., \"terminal\")]},\n",
        "    'cell_2_0': {\"RIGHT\": [(1.0, 0., \"cell_2_1\")],\n",
        "                 \"DOWN\": [(1.0, 0., \"cell_1_0\")]},\n",
        "    'cell_2_1': {\"RIGHT\": [(1.0, 0., \"cell_2_2\")],\n",
        "                 \"LEFT\": [(1.0, 0., \"cell_2_0\")]},\n",
        "    'cell_2_2': {\"RIGHT\": [(1.0, 0., \"goal\")],\n",
        "                 \"LEFT\": [(1.0, 0., \"cell_2_1\")],\n",
        "                 \"DOWN\": [(1.0, 0., \"cell_1_2\")]},\n",
        "    'goal': {\"EXIT\": [(1.0, +1., \"terminal\")]}\n",
        "}\n",
        "GAMMA = 0.9\n",
        "ETA = 0.1\n",
        "\n",
        "def get_initial_policy(model):\n",
        "    \"\"\" Returns an initial policy that selects the first available action \"\"\"\n",
        "    policy = {}\n",
        "    for state in model:\n",
        "        policy[state] = [action for action in model[state]]\n",
        "    return policy\n",
        "\n",
        "def get_transition_matrix(model, policy):\n",
        "    \"\"\" Computes a transition matrix for a deterministic policy \"\"\"\n",
        "    nb_states = len(model)\n",
        "    T = np.zeros((nb_states, nb_states), dtype=np.float32)\n",
        "    list_states = sorted([state for state in model])\n",
        "\n",
        "    for state in model:\n",
        "        actions = policy[state]\n",
        "        for action in actions:\n",
        "            for prob, reward, next_state in model[state][action]:\n",
        "                s, s_ = list_states.index(state), list_states.index(next_state)\n",
        "                T[s, s_] = prob / len(actions)\n",
        "    return T\n",
        "\n",
        "def get_sr_td(model, policy, nb_trajectories=1000):\n",
        "    \"\"\" Computes the SR matrix using TD(0) \"\"\"\n",
        "    nb_states = len(model)\n",
        "    M = np.eye(nb_states)\n",
        "    list_states = sorted([state for state in model])\n",
        "    diag = np.eye(nb_states)\n",
        "\n",
        "    # Generating trajectories and performing TD(0)\n",
        "    for _ in range(nb_trajectories):\n",
        "        state = 'cell_0_0'\n",
        "        while True:\n",
        "            actions = policy[state]\n",
        "            action = random.choice(actions)\n",
        "            prob, reward, next_state = random.choice(model[state][action])\n",
        "\n",
        "            s = list_states.index(state)\n",
        "            s_ = list_states.index(next_state)\n",
        "            M[s, :] = M[s, :] + ETA * (diag[s] + GAMMA * M[s_, :] - M[s, :])\n",
        "\n",
        "            if state == 'terminal':\n",
        "                break\n",
        "            state = next_state\n",
        "\n",
        "    # Return M\n",
        "    return M\n",
        "\n",
        "def get_sr_calc(model, policy):\n",
        "    \"\"\" Computes the SR matrix using the formula \"\"\"\n",
        "    nb_states = len(model)\n",
        "    I = np.eye(nb_states)\n",
        "    T = get_transition_matrix(model, policy)\n",
        "    sr = np.linalg.inv(I - GAMMA * T)\n",
        "    return sr\n",
        "\n",
        "initial_pi = get_initial_policy(env_model)\n",
        "sr_calc = get_sr_calc(env_model, initial_pi)\n",
        "sr_td0 = get_sr_td(env_model, initial_pi, nb_trajectories=10000)\n",
        "\n",
        "print('TD 0')\n",
        "print(sr_td0)\n",
        "print()\n",
        "print('Calc')\n",
        "print(sr_calc)\n",
        "\n",
        "## ==============================================================\n",
        "#\n",
        "# Discussion\n",
        "#\n",
        "# From https://papers.nips.cc/paper/5340-design-principles-of-the-hippocampal-cognitive-map.pdf,\n",
        "# we have established that the TD(0) update rule is as follows:\n",
        "#\n",
        "# M(s,j) <-- M(s,j) + eta * [I{s=j} + gamma * M(s',j) - M(s,j)]\n",
        "#\n",
        "# We have implemented the TD(0) update rule and reach a matrix M that is very close to the exact solution.\n",
        "# \n",
        "# One of the reason for divergence might be that our policy is stochastic (i.e. it randomly samples an action from\n",
        "# the list of available actions), therefore it would take a lot of iterations to converge to the exact solution.\n",
        "#\n",
        "## ==============================================================\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "TD 0\n",
            "[[ 2.2  1.2  0.8  0.2  1.4  0.3  0.7  0.4  0.3  0.1  0.2  2.4]\n",
            " [ 1.   1.9  1.4  0.4  0.5  0.5  0.3  0.2  0.2  0.1  0.3  3.3]\n",
            " [ 0.4  0.7  1.9  0.5  0.3  0.8  0.2  0.2  0.3  0.1  0.4  4.4]\n",
            " [ 0.3  0.4  0.9  1.3  0.2  0.2  0.1  0.1  0.1  0.   0.6  5.7]\n",
            " [ 1.3  0.7  0.5  0.2  2.3  0.2  1.3  0.6  0.4  0.1  0.2  2.3]\n",
            " [ 0.2  0.2  0.6  0.2  0.1  1.3  0.1  0.2  0.5  0.1  0.5  6. ]\n",
            " [ 0.9  0.5  0.3  0.1  1.4  0.3  2.   1.   0.6  0.2  0.1  2.7]\n",
            " [ 0.3  0.2  0.2  0.1  0.4  0.4  0.7  1.7  1.1  0.3  0.2  4.4]\n",
            " [ 0.2  0.2  0.3  0.1  0.2  0.6  0.3  0.6  1.5  0.4  0.2  5.5]\n",
            " [ 0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   0.   9. ]\n",
            " [ 0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   9. ]\n",
            " [ 0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  10. ]]\n",
            "\n",
            "Calc\n",
            "[[ 2.1  1.2  0.7  0.2  1.3  0.3  0.8  0.4  0.3  0.1  0.2  2.4]\n",
            " [ 1.2  1.9  1.1  0.3  0.7  0.4  0.5  0.3  0.2  0.1  0.3  3.1]\n",
            " [ 0.5  0.7  1.7  0.5  0.3  0.6  0.2  0.2  0.3  0.1  0.4  4.5]\n",
            " [ 0.2  0.3  0.8  1.2  0.1  0.3  0.1  0.1  0.1  0.   0.6  6.1]\n",
            " [ 1.3  0.7  0.5  0.1  2.2  0.3  1.3  0.7  0.4  0.1  0.1  2.3]\n",
            " [ 0.2  0.3  0.6  0.2  0.2  1.3  0.2  0.2  0.5  0.2  0.5  5.7]\n",
            " [ 0.8  0.5  0.3  0.1  1.3  0.3  2.1  1.1  0.6  0.2  0.1  2.7]\n",
            " [ 0.4  0.3  0.3  0.1  0.7  0.4  1.1  1.8  0.9  0.3  0.1  3.7]\n",
            " [ 0.2  0.2  0.3  0.1  0.3  0.5  0.4  0.6  1.4  0.4  0.2  5.5]\n",
            " [ 0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   0.   9. ]\n",
            " [ 0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   9. ]\n",
            " [ 0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  10. ]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "ARnPJIPCG-TX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Q2.2 - Value Prediction with SR"
      ]
    },
    {
      "metadata": {
        "id": "shxuommF8xeP",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "base_uri": "https://localhost:8080/",
          "height": 125
        },
        "outputId": "63fd47dd-8674-4e3b-cb71-ea608e228bd8",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1523236189699,
          "user_tz": 240,
          "elapsed": 305,
          "user": {
            "displayName": "Philip Paquette",
            "photoUrl": "//lh6.googleusercontent.com/-DGrBTu2a2aw/AAAAAAAAAAI/AAAAAAAAANs/SZizZJUV1nA/s50-c-k-no/photo.jpg",
            "userId": "101375949079620784780"
          }
        }
      },
      "cell_type": "code",
      "source": [
        "# Q2\n",
        "\n",
        "# Computing Value Iteration\n",
        "def policy_evaluation(model, policy):\n",
        "    \"\"\" Evaluates a policy and returns the value at each state \"\"\"\n",
        "    diff = float('inf')\n",
        "    values = {state: 0. for state in model}\n",
        "\n",
        "    # Evaluating policy until convergence\n",
        "    while diff > 1e-5:\n",
        "        diff = 0\n",
        "\n",
        "        # For each initial state, selecting action based on policy and re-evaluating\n",
        "        for state in model:\n",
        "            state_value = 0\n",
        "            for action in policy[state]:\n",
        "                prob_action = 1. / len(policy[state])\n",
        "                for prob, reward, next_state in model[state][action]:\n",
        "                    state_value += prob * prob_action * (reward + GAMMA * values[next_state])\n",
        "            diff += abs(values[state] - state_value)\n",
        "            values[state] = state_value\n",
        "\n",
        "    # Returning values\n",
        "    return values\n",
        "\n",
        "sorted_states = sorted([state for state in env_model])\n",
        "reward_vector = [0., 0., 0., 0., 0., 0., 0., 0., 0., 1., -1., 0.]\n",
        "gvf_values = np.dot(sr_calc, reward_vector)\n",
        "pol_values = policy_evaluation(env_model, initial_pi)\n",
        "pol_values = [pol_values[state] for state in sorted_states]\n",
        "\n",
        "print('-' * 80)\n",
        "print('GVF')\n",
        "print(np.array(gvf_values))\n",
        "print()\n",
        "print('Policy Evaluation')\n",
        "print(np.array(pol_values))\n",
        "\n",
        "## ==============================================================\n",
        "#\n",
        "# Discussion\n",
        "#\n",
        "# We have implemented a very simple reward function, \n",
        "# 1) We get a reward of +1 when exiting from the goal\n",
        "# 2) We get a reward of -1 when exiting from the pit\n",
        "# 3) We get a reward of 0. everywhere else.\n",
        "#\n",
        "# We have implemented policy evaluation to compute the value of every state if we follow our random policy.\n",
        "#\n",
        "# We have also computed the value of every state using the exact SR method calculated in the previous question.\n",
        "#\n",
        "# As expected, both methods converge to the exact same value function.\n",
        "#\n",
        "# i.e. +1 for goal state, -1 for pit, and values between -1 and +1 for the other states.\n",
        "#\n",
        "## =============================================================="
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--------------------------------------------------------------------------------\n",
            "GVF\n",
            "[-0.1 -0.2 -0.3 -0.6 -0.  -0.3  0.   0.1  0.2  1.  -1.   0. ]\n",
            "\n",
            "Policy Evaluation\n",
            "[-0.1 -0.2 -0.3 -0.6 -0.  -0.3  0.   0.1  0.2  1.  -1.   0. ]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "pFQ9U-IiHDnC",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Q2.3 - Tracking with SR"
      ]
    },
    {
      "metadata": {
        "id": "pysY9hxa8z91",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "base_uri": "https://localhost:8080/",
          "height": 366
        },
        "outputId": "8361d567-f2c3-45bf-81af-e3bb1a1fcb3f",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1523237240798,
          "user_tz": 240,
          "elapsed": 870,
          "user": {
            "displayName": "Philip Paquette",
            "photoUrl": "//lh6.googleusercontent.com/-DGrBTu2a2aw/AAAAAAAAAAI/AAAAAAAAANs/SZizZJUV1nA/s50-c-k-no/photo.jpg",
            "userId": "101375949079620784780"
          }
        }
      },
      "cell_type": "code",
      "source": [
        "# Q3\n",
        "\n",
        "def plot_error_policy_eval(model, policy, reward_vector_1, reward_vector_2, sr_calc):\n",
        "    \"\"\" Plots the rewards error \"\"\"\n",
        "    list_states = sorted([state for state in env_model])\n",
        "    values = {state: 0. for state in model}\n",
        "    errors = []\n",
        "\n",
        "    nb_iters = 0\n",
        "    reward_vector = reward_vector_1\n",
        "    ground_truth = np.dot(sr_calc, reward_vector)\n",
        "    while nb_iters <= 1000:\n",
        "\n",
        "        # For each initial state, selecting action based on policy and re-evaluating\n",
        "        for state in model:\n",
        "            state_value = 0\n",
        "            for action in policy[state]:\n",
        "                prob_action = 1. / len(policy[state])\n",
        "                for prob, _, next_state in model[state][action]:\n",
        "                    reward = reward_vector[list_states.index(state)]\n",
        "                    state_value += prob * prob_action * (reward + GAMMA * values[next_state])\n",
        "\n",
        "            values[state] = state_value\n",
        "            nb_iters += 1\n",
        "            estimate = [values[state] for state in list_states]\n",
        "            errors += [np.sum(np.square(np.array(estimate) - ground_truth))]\n",
        "\n",
        "    # New value function\n",
        "    nb_iters = 0\n",
        "    reward_vector = reward_vector_2\n",
        "    ground_truth = np.dot(sr_calc, reward_vector)\n",
        "    while nb_iters <= 1000:\n",
        "\n",
        "        # For each initial state, selecting action based on policy and re-evaluating\n",
        "        for state in model:\n",
        "            state_value = 0\n",
        "            for action in policy[state]:\n",
        "                prob_action = 1. / len(policy[state])\n",
        "                for prob, _, next_state in model[state][action]:\n",
        "                    reward = reward_vector[list_states.index(state)]\n",
        "                    state_value += prob * prob_action * (reward + GAMMA * values[next_state])\n",
        "\n",
        "            values[state] = state_value\n",
        "            nb_iters += 1\n",
        "            estimate = [values[state] for state in list_states]\n",
        "            errors += [np.sum(np.square(np.array(estimate) - ground_truth))]\n",
        "\n",
        "    # Returning\n",
        "    return errors\n",
        "\n",
        "def plot_error_sr_td0(model, policy, reward_vector_1, reward_vector_2, sr_calc):\n",
        "    \"\"\" Plots the rewards error \"\"\"\n",
        "    errors = []\n",
        "\n",
        "    nb_states = len(model)\n",
        "    M = np.eye(nb_states)\n",
        "    list_states = sorted([state for state in model])\n",
        "    diag = np.eye(nb_states)\n",
        "\n",
        "    # Generating trajectories and performing TD(0)\n",
        "    nb_iters = 0\n",
        "    reward_vector = reward_vector_1\n",
        "    ground_truth = np.dot(sr_calc, reward_vector)\n",
        "    while nb_iters <= 1000:\n",
        "        state = 'cell_0_0'\n",
        "        while True:\n",
        "            actions = policy[state]\n",
        "            action = random.choice(actions)\n",
        "            prob, reward, next_state = random.choice(model[state][action])\n",
        "\n",
        "            s = list_states.index(state)\n",
        "            s_ = list_states.index(next_state)\n",
        "            M[s, :] = M[s, :] + ETA * (diag[s] + GAMMA * M[s_, :] - M[s, :])\n",
        "            nb_iters += 1\n",
        "            estimate = np.dot(M, reward_vector)\n",
        "            errors += [np.sum(np.square(np.array(estimate) - ground_truth))]\n",
        "\n",
        "            if state == 'terminal':\n",
        "                break\n",
        "            state = next_state\n",
        "\n",
        "    # Changing reward fn\n",
        "    nb_iters = 0\n",
        "    reward_vector = reward_vector_2\n",
        "    ground_truth = np.dot(sr_calc, reward_vector)\n",
        "    while nb_iters <= 1000:\n",
        "        state = 'cell_0_0'\n",
        "        while True:\n",
        "            actions = policy[state]\n",
        "            action = random.choice(actions)\n",
        "            prob, reward, next_state = random.choice(model[state][action])\n",
        "\n",
        "            s = list_states.index(state)\n",
        "            s_ = list_states.index(next_state)\n",
        "            M[s, :] = M[s, :] + ETA * (diag[s] + GAMMA * M[s_, :] - M[s, :])\n",
        "            nb_iters += 1\n",
        "            estimate = np.dot(M, reward_vector)\n",
        "            errors += [np.sum(np.square(np.array(estimate) - ground_truth))]\n",
        "\n",
        "            if state == 'terminal':\n",
        "                break\n",
        "            state = next_state\n",
        "\n",
        "    # Return errors\n",
        "    return errors\n",
        "\n",
        "reward_vector_1 = [0., 0., 0., 0., 0., 0., 0., 0., 0., 1., -1., 0.]\n",
        "reward_vector_2 = [0., 0., 0., 0., 0., 0., 0., 0., 0., 1.1, -2., 0.]\n",
        "\n",
        "errors_no_sr = plot_error_policy_eval(env_model, initial_pi, reward_vector_1, reward_vector_2, sr_calc)\n",
        "errors_sr = plot_error_sr_td0(env_model, initial_pi, reward_vector_1, reward_vector_2, sr_calc)\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "plt.plot(errors_no_sr, label = \"without SR\")\n",
        "plt.plot(errors_sr, label = \"with SR\")\n",
        "plt.legend()\n",
        "\n",
        "## ==============================================================\n",
        "#\n",
        "# Discussion\n",
        "#\n",
        "# It's not clear if we have to do this question or not, as the questions changed after being posted.\n",
        "#\n",
        "# We computed the value of every state using policy evaluation and SR TD(0)\n",
        "# As requested, we abrutely changed the reward function after 1,000 steps\n",
        "#\n",
        "# From the graph below, we can see that policy evaluation (without SR) has a huge peak after the change of \n",
        "# reward function, but SR TD0 stays relatively constant.\n",
        "#\n",
        "# This is the expected behaviour.\n",
        "#\n",
        "# It can be explained by the fact that SR computes the expected (discounted) occupancy of future states\n",
        "# and is independent of the reward function.\n",
        "#\n",
        "# This is one reason why SR is very useful, especially for multi-task learning.\n",
        "#\n",
        "## =============================================================="
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f134d559c18>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAFKCAYAAAAqkecjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XlgE2XCBvBnkjTpkfROb24K5b7l\nkoIIqKgrXqAuiruouB/iuourfui34LGoLLLrqt+nIu6hu4oi7nqgeACKUO77LDc9oE3vpGnaJpnv\njzTThiY96JGZ4fn9sUuuyduMM8+877yHIIqiCCIiIup0mmAXgIiI6ErFECYiIgoShjAREVGQMISJ\niIiChCFMREQUJAxhIiKiINF19hdaLNZ23V5MTDhKS+3tuk26fNwf8sF9IR/cF/IRrH1hNpv8Pq/4\nmrBOpw12EagB7g/54L6QD+4L+ZDbvlB8CBMRESkVQ5iIiChIGMJERERBwhAmIiIKEoYwERFRkDCE\niYiIgoQhTEREFCSdPlkHERFRSxQXF2HVqrfwxBNPY9++PejWrTtiYmJxxx034x//WI3w8PDL3vam\nTd9j0qRrfZ6rrLThxRefR2lpCdxuF6KiovH008/CZDLhjjtuRkJCIjQaDURRhMEQikWLfo/4eHOb\n/kbWhImISJbi4uLxxBNPAwC+/PIzlJaWtMt2L1zIx3ffrW/0/OrV/0L//gPwxhsr8X//9y4yMvrj\nm2++kl5fvvwveP31t/HGGysxZco0rFz5f20uC0OYiIiC6p57bofL5YLT6cTUqZk4duwIAOC3v30E\nd975M+zcuQ2bN2/C0qXP4eLFiwCATz75CPPnP4iHH/4l7PZKOJ1OLF36LB555CE89ND92LFjGwDg\njjtuht3umaby9df/jLVr12LFipexb98e/PWvK33KYbNZYbPZpMf33/8Abr99pt8y9+8/ELm5OW3+\n2xXdHF1d48KXP51GZWU1xg9Khj5EXtOREREpyUcbTmLnscJ23eaojATMnNy7yff07dsPp0+fgtNZ\ni4yMfjh06AD69MnAkSOHkJraBaNGjUHv3n3w298+gaSkJABAz569cO+992Px4kXYtWsnKitt0Ov1\neP31t1FUZMEjj8zDhx+u9ft9d999L9au/Qi/+MWDPs/fdttM/OY3j2Dbti246qqxuPbaaUhP7+N3\nG5s2fY8+fTIu4xfxpegQPpFbhjc/PQgAMIXrMTIjIcglIiKi1ho6dDgOHz6Imppq3HHHLPzww0YM\nGXISffpkoLKy0u9nBg8eCgAwmxNQWWnD8eNHMWzYCABAfLwZen0IKirKW1WOtLQu+OCDT7Bnzy5s\n356Fxx77FX71q0dx0023AAAef/xRaDQa5OfnYfDgoXjiiUVt+Ks9FB3C/XvEYvLILtiwKwe1Lnew\ni0NEpGgzJ/duttbaEYYNG4H33/8bqqsduOmmW/Dll5/j4MH9GD58JDZv/sHvZ7Ta+pZPURQBCHX/\n71FbWwtB0EAQBOk5p9PZZDmqqx0wGEJx1VVjcNVVY3D11Zl49923pRBevvwvCA8PxyefrEZOTg7C\nwyPa8Fd7KPqesEYQ0L9HrOeB2PR7iYhInrp27YaCggLYbJUID49AXFwcNm/ehGHDRkrv0Wg0cLlc\nAbfRr19/7NmzCwBQUHARGo0GJpMJ4eERKC4ugsvlwuHDB5vc1mOPzcfOndulxxZLIVJSUhu975Zb\nbsfevbtx4kT2Zf/NXooOYQ+h+bcQEZGsxcTESPd7+/cfiAsXLiAhof4W49Chw/HMM0/i9OlTfj9/\n7bXT4Ha7sWDBPCxZsgi/+52nqfj222fiySd/g6ef/h169OgJAOjWrQeOHz+Gv/zlFZ9tLFq0GP/6\n1z8wf/6DePTRh7Fp0wbMmze/0XfpdDrMn/9rrFjxkk/t+3IIYlu30EoWi7Vdt7fnVAle/3gfHrip\nH8YNTG7XbVPrmc2mdt/HdHm4L+SD+0I+grUvzGaT3+cVXxMWWBEmIiKFUnwIe3VufZ6IiKjtFB/C\nrAgTEZFSKT6EiYiIlErxIcx7wkREpFSKD2Ev3hMmIiKlUfSMWR6sChMRqdHlLmXodDqxYsXLOH36\nFLRaLbRaLRYtWoKkpCTce++9qKiwITQ0VHr/woVPSWOIO5sKQthD5JRZRESqculShnffPRsxMbHN\nfu7bb7+GRqPFm2++CwD46qsv8OmnH+NXv1oAAFi06Pfo2dMzPeeePbvw5z//Ea++2vZlCS+H4puj\neU+YiEjZ2mMpw4asViuqquqfu+GGm6QAvtSAAQORk3O+g/6y5qmmJsyKMBFR26w9+QX2Fh5s120O\nSxiE23rf1OR72mMpw8zMSdL2rrvuBnz11ee4++7bMHbseEyceC2GDBnq97s3bvweffu2fUnCy6X4\nEGZNmIhI2dpjKcOGoqKi8e67/8SBA/uwY8c2PPvs07jxxp9h7tx5AIClS59DaGgoioqKkJKSgkWL\nlnTo39cUxYewFyvCRERtc1vvm5qttXaE9lnKsF5tbS20Wi2GDBmGIUOG4eabZ2DBgnlSCHvvCW/Z\nshmff/4p4uPjO+6Pa4bi7wmzdzQRkbK1x1KGDb344nP48svPpMeFhQV+lyQcP34CampqsHXrT23/\nIy5Ti2rCy5Ytw+7du+F0OjFv3jxMmzZNem3y5MlISkqSrkqWL1+OxMTEjiktERGpUkxMDCIiIgB4\nljLcu3eP36UMX3zxlUCbkCxY8Fv88Y9LsW7d59Dr9dBqdVi48KmA71206HGMGDEKBoOhff6YVmh2\nKcNt27Zh1apVWLlyJUpLS3Hrrbdi06ZN0uuTJ0/G559/Lv14zWnvJaQOnS/Din/twf03ZCBzSEq7\nbptaj0u2yQf3hXxwX8iH3JYybLYmPGrUKAwePBgAEBkZiaqqKrhcLp/2eDno5GWRiYiI2qzZENZq\ntdKsJGvWrEFmZmajAF68eDHy8vIwYsQILFy4EEIndlnmHWEiIlKqFveO/u6777BmzRq8++67Ps8/\n+uijmDBhAqKiojB//nysX78e119/fcDtxMSEQ6drx1r0+TIAgNEYGrC6T52L+0E+uC/kg/tCPuS0\nL1oUwps3b8abb76Jd955ByaTb+FnzJgh/TszMxPZ2dlNhnBpqf0yixpAXa3banXwnosM8N6XfHBf\nyAf3hXzI7Z5ws0OUrFYrli1bhrfeegvR0dGNXps7dy5qamoAADt37kR6eno7FLfl2BxNRERK1WxN\neN26dSgtLcVjjz0mPTd69Gj07dsXU6dORWZmJmbNmgWDwYD+/fs3WQvuSOyWRUREStNsCM+aNQuz\nZs0K+PqcOXMwZ86cdi1Ua3DaSiIiUioVzJhVh0OUiIhIYRQfwgLvChMRkUIpPoS9WA8mIiKlUX4I\nsyJMREQKpfwQrsNbwkREpDSKD2H2jiYiIqVSfAgTEREpleJDmL2jiYhIqRQfwl5cypCIiJRG+SHM\nijARESmU8kO4DuvBRESkNIoPYVaEiYhIqRQfwhJWhYmISGEUH8IcJ0xEREql+BD2YkWYiIiURgUh\nzKowEREpkwpCuA7HCRMRkcIoPoR5T5iIiJRK8SHsxXowEREpjeJDmBVhIiJSKsWHsBdvCRMRkdIo\nPoQF3hQmIiKFUnwIExERKZXyQ5gVYSIiUijlh3Adkf2jiYhIYRQfwqwIExGRUik+hCWsCBMRkcIo\nPoTZO5qIiJRK8SHsxYowEREpjWpCmIiISGlUE8Iip8wiIiKFUXwI85YwEREpleJDmIiISKkUH8IC\nRwoTEZFCKT6EvXhLmIiIlEb5IcyKMBERKZTyQ7gOK8JERKQ0ig9hVoSJiEipFB/CEt4UJiIihVF8\nCHOcMBERKZXiQ9iL9WAiIlIaXUvetGzZMuzevRtOpxPz5s3DtGnTpNe2bt2KFStWQKvVIjMzE/Pn\nz++wwvrDccJERKRUzYbwtm3bcOLECaxevRqlpaW49dZbfUL4hRdewKpVq5CYmIjZs2fjuuuuQ+/e\nvTu00H6xKkxERArTbAiPGjUKgwcPBgBERkaiqqoKLpcLWq0WOTk5iIqKQnJyMgBg4sSJyMrK6twQ\nZkWYiIgUqtl7wlqtFuHh4QCANWvWIDMzE1qtFgBgsVgQGxsrvTc2NhYWi6WDito0VoSJiEhpWnRP\nGAC+++47rFmzBu+++26bvjAmJhw6nbZN22joQrkDABARrofZbGq37dLl436QD+4L+eC+kA857YsW\nhfDmzZvx5ptv4p133oHJVF/4hIQEFBUVSY8LCgqQkJDQ5LZKS+2XWdSm2SqrYbFYO2Tb1HJms4n7\nQSa4L+SD+0I+grUvAgV/s83RVqsVy5Ytw1tvvYXo6Gif19LS0mCz2ZCbmwun04mNGzdi/Pjx7VPi\nFmLvaCIiUqpma8Lr1q1DaWkpHnvsMem50aNHo2/fvpg6dSqWLFmChQsXAgCmT5+OHj16dFxpiajF\nftqfhz9/sBfGsBAs+eUoRISGBLtIRHSJZkN41qxZmDVrVsDXR40ahdWrV7droVqFFWEiv46fK0V1\nrQvVtS4UlTkQkcQQJpIb9cyYxe7RRAGJHD9AJEuKD2FWhImISKkUH8JevM4n8tWwdYgtRUTypPgQ\nFqRllHiWISIiZVF8CBMRESmVakKYzW1ERKQ0ig9hgT2ziPxq2COaF6lE8qT4ECYiIlIqxYcwp60k\nah7HCRPJk+JD2IvNbUREpDSKD2HeEyYKQAzwbyKSDcWHsBeb24iISGmUH8KsCRM1i5eoRPKk/BD2\n4lmGiIgURvEhzIowkX9iwAdEJBeKD2EvnmOIiEhpFB/CArtHEzWLHReJ5EnxISzhOYaIiBRGPSFM\nRD5EkXNHE8mdakKYzW1ERKQ0ig9h3hImIiKlUnwIe7G5jegSPCaIZE/xIexdRel8gdXnHhgR1eOx\nQSRPig/hEJ3nTzh2vgxHzpUGuTREREQtp/gQ7ppkQtcEIwDAaq8JcmmIiIhaTvEhLAgCJg1L9Txg\nixuRhIcDkfwpPoQBcAJpombwljCRPKkjhOvwPENEREqiihBmRZioabxAJZInVYSwhGcaIgmHJRHJ\nnypCmCspETWDgUwkS6oIYS/OH01EREqiqhAmIv94eUokT6oKYba4EdXj4UAkf6oIYd4RJmoaA5lI\nnlQRwkREREqkjhBmVZioaawKE8mSOkK4Du8JEzXA44FI9lQRwgKrwkRN4vA9InlSRQh78URDRERK\noooQ5oRZRM3g9SmRLLUohLOzszFlyhS8//77jV6bPHky7rnnHtx777249957UVBQ0O6FbDGeaIgk\nPByI5E/X3Bvsdjuef/55jB07NuB7Vq5ciYiIiHYtGBERkdo1WxPW6/VYuXIlEhISOqM8bcIrfyL/\neGwQyVOzNWGdTgedrum3LV68GHl5eRgxYgQWLlzY6asa8Z4wUWNcypBI/poN4eY8+uijmDBhAqKi\nojB//nysX78e119/fcD3x8SEQ6fTtvVrfZhMYQAAozEUZrOpXbdNrcd9ID+RkWHcL0HG318+5LQv\n2hzCM2bMkP6dmZmJ7OzsJkO4tNTe1q/0YTabYLM6AAA2mwMWi7Vdt0+tYzabuA9kqLy8ivsliHhc\nyEew9kWg4G/TECWr1Yq5c+eipqYGALBz506kp6e3ZZNtwuY3okB4bBDJUbM14UOHDuHll19GXl4e\ndDod1q9fj8mTJyMtLQ1Tp05FZmYmZs2aBYPBgP79+zdZC+4wvCdMREQK1GwIDxw4EO+9917A1+fM\nmYM5c+a0a6EuF6/1ifxjIxGRPKljxqxgF4CIiOgyqCKEJbzaJ/KLhwaRPKkjhFkVJmqETdBE8qeO\nEK7Dcw5RADw4iGRJFSHM9YSJiEiJVBHCEra/EfnFtbaJ5EkVIcy5o4ka4+Q1RPKnihD24imHKAAe\nHESypKoQJiIiUhJVhTBb34j846FBJE+qCOHOXr+YiIioPagjhINdACKZYysRkTypIoS92BuUiIiU\nRFUhTESB8AKVSI5UFcI8zRDVY8MQkfypIoTZL4uoaQxkInlSRQhLeKIhIiIFUUkIsypMdCnOF00k\nfyoJYQ+ecoiISElUEcK8J0zUNF6gEsmTKkJYwt4nRESkIKoIYVaEiRpreE3KiWyI5EkVIezF0wwR\nESmJOkKYVWEiIlIgdYRwHba4ERGRkqgihAVWhYmaxAtUInlSRQgTEREpkTpCmBVhoiZx9iwieVJH\nCNfhiYaIiJREFSHMijBRYz5jg3l9SiRLqghhCU80RESkIKoIYc4dTdQ0Xp8SyZMqQtiLJxoiIlIS\nlYQwq8JElxIDPiAiuVBJCHtwknoiIlISVYQw7wkTNY3D94jkSRUhTEREpESqCGFWhIn88FlPOHjF\nIKLAVBHCXjzREBGRkqgjhFkVJiIiBVJHCNdhRZioHluGiOSvRSGcnZ2NKVOm4P3332/02tatW3HH\nHXdg1qxZeOONN9q9gC3B9YSJmsZAJpKnZkPYbrfj+eefx9ixY/2+/sILL+C1117DBx98gC1btuDk\nyZPtXsgW45mGiIgUpNkQ1uv1WLlyJRISEhq9lpOTg6ioKCQnJ0Oj0WDixInIysrqkII2iRVhoiZx\nnDCRPDUbwjqdDqGhoX5fs1gsiI2NlR7HxsbCYrG0X+laiacZonoMXiL503X2F8bEhEOn07bvNqPC\nAQAREQaYzaZ23Ta1HveB/JiModwvQcbfXz7ktC/aFMIJCQkoKiqSHhcUFPhttm6otNTelq9sxGw2\noazcs81KWzUsFmu7bp9ax2w2cR/IkNXq4H4JIh4X8hGsfREo+Ns0RCktLQ02mw25ublwOp3YuHEj\nxo8f35ZNXhbeEiZqGhumieSp2ZrwoUOH8PLLLyMvLw86nQ7r16/H5MmTkZaWhqlTp2LJkiVYuHAh\nAGD69Ono0aNHhxc6EJ5oiBpocED87atjGNM/EfqQ9r0VRERt02wIDxw4EO+9917A10eNGoXVq1e3\na6FajcsoETXr9U8P4rczhwa7GETUgLpmzOI4YaJGHrltEADg6NnSIJeEiC6lihBmPZgosJ4pkeid\nGsW5bIhkSBUhTESN+WSuwHHDRHKkihDmLWGiwATUHejMYCLZUUUIe7G5jSgAQWAGE8mQqkKYiPzT\n1LUWsfMikbyoKoR5eiGq5y9wmcFE8qKKEBZ4U5goMEGQjhF2ziKSF1WEcD2eYIj8EaTm6OCWg4h8\nqSyEicgfb1sRQ5hIXlQVwjzBENVreDjU37LhQUIkJ6oIYd4SJgpMkP4HcDODiWRFFSFMRE0TvCnM\nECaSFVWEsMDZo4kaaxC4UscspjCRrKgihL14T5jIP3bMIpIndYQwr/KJAhPqO2YxhInkRRUhzMZo\noqbVd15kChPJiSpCWMLzC5HEX8sQe0cTyYs6QphVYaImaTiOj0iW1BHCdXiRT9SY7zhhHiVEcqKK\nEOYQJaKm8ZYwkTypIoQlPMEQSUSfccLeVZSISE4UH8IutwveUwub2oj8q19FiccIkZwoOoTPlJ/D\nfZ88hg/P/R0A8M3OHDhqnEEuFZG8NFxvmxlMJC+KDuFoQxRq3U7kVeXCWxsut9UEt1BEMqSRJutg\nChPJiaJDOCY0GsOTBwIAxg8xA+A9LyK/2HeRSJZ0wS5AW4WHhAEA8nR7oUt1YOMFO0xlIYgNjcWY\n5BHQCIq+ziBqF94RBKwIE8mL4kPYHBEHACjQHkZIKvBTwSnptZjQKHQzdcG6M9+ipLoMgLdCICBS\nb8RNPa9DREh45xeaqJOxYxaRPCk+hO8ccCN6hvfEV9vPYu+JIsy9sR9yak7ih9wt2JCzGUeKjwf8\n7K6CfRiXchXcohuiKMIlutEtMg1jkkd24l9A1PGkVZSCWgoiupTiQ1in1aFnVHdEoRpuqxtdIrqh\nW3wcfsjd4hPA/zN6IYx6IyACFTVW/GHHCtidVfju/A8+29PkazA6aYRPj1IiJWpY6+U4YSJ5UnwI\nezW855ViTMLvRz+OSqcdudYLiA2NRlJEovReoz4CS8Y8CWutFQI00AgCNIIWHx5fi7MV5+EW3dAK\n2mD9KUTtShBQv9wnm6OJZEU9IXzJSSYxIgEA0DOqu9/3m8PjYEacz3PhOk8nL5fohhYMYVIPDduj\niWRJPV2HpRC+/E1oNZ6fwyW62qFARHLiOUA4qxyRvKgmhNtjqTZvEzRDmNTAd+7o4JWDiAJTTQh7\nteVKXwpht7u9ikMUdAIadMxiRZhIVlQTwu1RE9ZINWHOP03qIt0SZgoTyYpqQrg9Fi2X7gn7qQnX\nup34/vyPePfQP3G4+NhlfwdRUNQdH4WlVcEtBxH5UF3v6Lb0/mx4T7iy1o6/Hv4XCu0WuEQ3yqrL\npfftLtyP5ZnPIUwX2oYSE3Uel8tzYNiqaoNcEiJqSD0h3A5z43pD+L2jHyEuNAZHS7IhQEBcaEyj\n91bW2hnCpBAC+nSJwo/78zlCiUhm1BPC3iFKbTjNRBkiAQBnK87jbMV5CBCwZOwTiA+LQ3m1FXm2\nfOyzHMSW/B1wuXnfmJTD22fC7WYME8mJ+kK4DeeYad0moXtkF5wqO4PEiAQkhpsRH+aZ0CPKYEKU\noS8OFR8FADg5jIlkzneIEtcTJpKjFoXw0qVLsX//fgiCgEWLFmHw4MHSa5MnT0ZSUhK0Wk9T7vLl\ny5GYmBhoUx2mvjn68k8yGkGDjNh0ZMSmB3yPTvD8ZE7WhElBNBrvZB1BLggR+Wg2hHfs2IFz585h\n9erVOHXqFBYtWoTVq1f7vGflypWIiIjosEK2RHvUhFtCq/Gd0ONk2RmcKT8Hp9sJl+jCiMShSI7o\n/IsQokAEoX7aSjZHE8lLsyGclZWFKVOmAAB69eqF8vJy2Gw2GI3GDi9ca3TWKjG6us5bFyst+Oj4\nv5Fjy/d5Pceah18N+WUHl4KodTRsjiaSpWbHCRcVFSEmpr53cGxsLCwWi897Fi9ejLvvvhvLly8P\n2kHeWZMR6DSe65Z/HvtYCuCpXSdhwdAHAQAX7ZaAnyXqTA07KXovUlkRJpKXVnfMujTkHn30UUyY\nMAFRUVGYP38+1q9fj+uvvz7g52NiwqHTte8KRWazCUajAQAQFRUGs9nUrttvKLrEt9n9tv43YOaA\nm6DRaPDRSTNqXbUd+v1KcKX//XITH29EobUGABAWruf+CRL+7vIhp33RbAgnJCSgqKhIelxYWAiz\n2Sw9njFjhvTvzMxMZGdnNxnCpaX2yy2rX2azCRaLFZV2z0mmtMwOi8Xart/RUM/Q3hgSPwC1ohPX\nd7sWvaK7o7i4EgCgETWorq3p0O+XO+/+IPkoKrLBWuGZKctmc3D/BAGPC/kI1r4IFPzNNkePHz8e\n69evBwAcPnwYCQkJ0v1gq9WKuXPnoqbGE4A7d+5EenrgnsUdqbPWSzWHx+GhwXMwf8hc9Iru7vNa\niEaHWjdnJCL5EThOWFGKq0rx0s5XcaT4OI4UH8cb+1f5zNpH6tFsTXj48OEYMGAA7rrrLgiCgMWL\nF2Pt2rUwmUyYOnUqMjMzMWvWLBgMBvTv37/JWnBnCOY5JkQTglq3E6IoSic9omBpeOdI6h3NDFaE\nrRd2IMeahzf2r5KeW3fmW9yTcUcQS0UdoUX3hB9//HGfxxkZGdK/58yZgzlz5rRvqS6Dpj0mj26j\nEE0IRIj49OSX6BvbGwPiMpr/EFEn8I4TZu9oZYgNjW70XLWrJggloY6mmhmzIIMr/Zi6A+f7nB/x\nfc6PmNp1EkJ1BhRVlcAYEoHkiEQMSxgMvTYkeIWkK44gNOwdzRBWgn8d+wQAcH//u2HQ6vHWwb/D\n4XRAFEWUVZcj2hDF1jaVUE0Ie2fMCuYM9bP63ooJqWOwMecn7CzYi2/Pb2r0ntXZn6KbqQseHvIL\nGLT6zi8kXZHq544OckFUxC26satgH4aZByGkgy6snW4nRiYOhQABF+0WvLBjBS5WFuBXg3+BgfH9\nOuQ7qXOpJ4SlGbOC2RytQ7fILvh5vzsxJnkk3KIbgiDAYi+GVqNBdukp7CrYh+yyUzhbfh59Y3sH\nrax0ZRHqumCyObr9bM7bho+y/42DCUcwd+DsDvkOp+iEIAiINkShqKpYej7Xls8QVgkVhXDnzJjV\nEiEanc/80/1iPf8/PmU0MmLS8f6xj1HsKA1S6ehKI0CorwkzhNtNqaMMALDPcqjDvkNbN1f92JRR\nWHfmW+l5a42tw76TOpd6Qrju/+V+pW/QeSYVqWEnC+okBfZCVDo97dBsjm4/nXFPdlTiUADAjT2m\n4rpu16DQXoQ/7FgBt8gdqRbqCeFOWsChrfQaz70jhjB1tDLdKYRd9RNe3vN13TPX4dtdOZg0LAXJ\nccFdcEUNXG7PIi7e+eTbU4whGhpB8LnXrNPooK27r+BiCKtGs5N1KIWcmqObYtB6asIVNZw9hzpW\nSchxn8eC3jNr1srPjwSjOKrjXVPcu7JaS4miiNXH/40dF/f4ff2A5TBKq8ug0zTu7KWpC3zWhNVD\nNSHslX2+LNhFaFJoXXP0xtyfkGvNb+bdQImjFPM3PIHvz//Y0UUjlZFGDNSZd2cvAMDZi1YUtvP0\nsVeaU2Vn8UPuFgCAtoU1YVEUsSVvOx7Z+CR+zNuKvx/50Od1a40Nf9n7Nt46+HcAQIoxqdE26u/t\nM4TVQjXN0aZwz1WjVivvsXOpxmT0ie6F7LJTOFqSjTRTis/rVc4qbM3f6RneJALWWk8HjLUnv8C1\nXTODUGJSKhEiRBG4ped0fHZmHQyhLvTtEo3jOWWoqeVJ/HIU2i1YuuNPqHU7pef8TazhzxdnvsHX\nZ7/3ea7h7Hqb87JwvPSk9NptvW9stA2N1BztanXZSZ5UUxNOig0HIP+5cTWCBren3wwAKHE0rrVv\nytmCtSe/gLXGJgUwAHQxpjR6L1GzRAHhIZ5jo7LWji6JnnnfXTI/TuRqS/4OnwAGAJO+ZSvyXKws\naPScw+WQ/m2rrZT+/buRjyA2NKbR+73N0XLvgEotp5oQ1tZNy6eEk0uYLgwAUOV0NHrtizPfAABu\n6XkD/jTxBQyK7+/5TN2JlKjlPMeCtx9CtatGOk44VMnDLbqxu2B/wPuzlyqpG1r46NCHsHjM76Rt\ntITDWQ0AeHXSUoxOGgHAc2Hkz4cGAAAgAElEQVTkZa/13LN/ftx/o3tkV7/baI+OWYX2InYMlRHV\nNEdrtXX/cSpgDEaYLhQAYKkqwoaczdhXeAgGrR49ouoPvCndJkIjaPDw4PvxyIYn4bzk6puoOZ6Y\nFWCo6+BT7aqR5pCWe4tRZ9lTeAB/PfwvAIBJb0S/2D5Nvt8blD2ju0NTd8+9pYFY5XQgRKODTqND\nRN1F9eKsl6HT6HyO73Bd4Atub3O0W3TDXmvH01v+gAHx/fBACycLKa+24tltywAAf560FCEa1USA\nYqlmD2gUVBMO1RkQognB2YrzOFtxXnr+SImnN+uYpJHSwQZ4Jv9wcolEai1BBERAXzc96oacHwF3\nKKAbqojjpDOct+ZK/z5Rehr9YvvgbMV5HC3OxnXdJ/sch0DdhYyg8RmW5G7h/VmHy4FQrecCPLJB\nE3bDAL6pxzSp86Y/9SHsQn5lAWrctdhbeAAVNVafbQZS2aDJe3PuVkxuYT8Tt+jGu4f/hX4x6Rif\nOrpFn6GWUU0I6xQUwhpBg/lDfonz1jzUup0w6SPgFt348PinANBoTeIQTQhstXbstxxGlbMKpY4y\nHCw+itSIJIxLuQo9oroF488guRMBCILUHO1p+rTDMGAr3O4r+0S64+IefHbqa5RW1/fL8Aby8l1v\nQISIrpFdMCCur/R6cVUJzlScA1A/JFIjaFrUtH/UcgIFdgvMYXEAgDHJI1FZa0dMaDRSjckwaA0w\nh8UitK6VLBBNg+bohk3Kp8rOYljCoGbL0bBDV0FVUbPv96qstWNv4QHsLTzAEG5nqglhrcKa2dJj\neiE9ppfPc94QHpU0zOf5UJ0BxY5SvF03dMHrXEUOzlScxzOjF3ZsYUmZBM+xkBhu9n1a67qi7wk7\nnA68d/QjuEU3THojJqSOxbfnNkpNzWJdQ35FdYXP5z4//U2jbXlCuOnm6M9PfY2vz23wbLNufgCT\n3ogZvae3uuzee8KiKKKmwcV6S6exbHiB73S1/BYXb4d1HNWEsNQc7VLuyeXBQffhgu2i1BnLa3a/\nO3Gq7BwM2hCE6kIRoglBldOBjbmbcaGyAC63q9UTBpD6iRABUYBeG4JfDLhHuvfptkUr5mK1I1iq\nSuAW3RgSPwAPDroPgiBgc24WHC6HT5hdekyFh4Q12pZG0OBsxXks2/UazlXkAADSjCn476sek96z\nq3C/9O8aV9tuK3lr4AV2i09NuMRRCmuNDREh4Y2a0L2qnFU4XlI/BKo1nbtqeDusw6gmhLUab8cs\n5Z5chpoHYqh5YKPn+8T0Rp+YxisuHS3JRqG9CNWuaoRr2HuaAhuZOBR9Y3rjv396AYKuRtHHSVs5\n6kYlJEckSqGm0+hQYLfgqZ+eC/g5sS60HhnygPScd+pKbwADnhWOvON/XW6Xz+pHbR3rrxE0CNHo\nUFpd5jPZx7fnN+Hb85tgDovDkrFP+v3sB8fWYneDC4KW3ssGgNo2XjxQYOoZolQ3SUeuxXbFjKHz\nduCw+xnqRFR3U1h6ZNIbESIYAF0t8qrOSwFypbloLwRQv5gKAMSHxTZ636W/T37lRQCAOTxeem50\n0gjoNSEYkTAE3Uxd6j8rumCtseHpLX8AAIzrOhJPjFyAGb1a3wR9qZt7Xh/wNUtVsd/xyABQeMk9\n4JYOrSqwW/B/B/7a8gLWcbqdyC49iQOWw63+7JVEPSFc1xxdVO7Ad7tzm3m3OniHOi3Oegk51rwg\nl4bkRvQzk7oWWmhC7fjS8iE+OflFEEoVPOcqcvDE5iX48PhaAL6LqMwdOBvzBs3BzT2vl8bxe+eG\nzrHmYf6GJ3Cy7AwAQN9gUYWf97sDKya+gF8O/DmeGLUAA+MyPJ91u3Cq/CystTaY9EY8NOIedIvs\n0i4rL13aAeu1a17Ca9e8hGhDFADg1b1v+7zucruw4+Ie6RyRakwG0PIQ/urM9yirLm91OZdkLcOr\nddNwXqwsbPXnrxSqCWGdVoMpI9IAAEVlV0bNcHjCEGl+4P0duKYpKZgINKwNJ4bUj0X/MXfrFdXh\n5ovT30iTY0QbojDUXB9mJr0Rg80DcH33ybgn43YAntqs0+3Ex9n/8dmON6S9Ggartm7crUt0ScE1\ns88MhOsb30++XBEh9Stg/WniC9AIGmgEjdSTu6LGilf3vIXDxcdRVl2OxVkvS03XGTHp+O3wX0ll\nbIq91o4vTn+DnQW+E5m0pKWx1u306XnecGYw8qWae8IAMGlYKr7bnYtal/wn7GgPvaN7YMnYJ7E4\n6yW/U2DSlc63ORoARkZMwbFdcYgbug+VbissVcVIjkjssBK43C5kXdiJfrF9EOenybczVdRYoRE0\neO2al5p8n7cHstvtwrYLu3Cq/CwAz7Ci9OieTU5w4R0/7HS7pBmywrRNDztqLb0mBBkx6UgxJklj\nwAFP2G/J3wEAyC47hRNlpzGj93QpDMenjMbNPa+Tpr4M1DFLFEWs2PO/OF1+zuf5PjG9kV16Em7R\n3eyiFfsKD/o8dl6htz5aQlUhrNd5Dp7a2itnhxvrZt7Jr7yIrfk7UOV04EJlAbIu7MRdfW/FhNSx\nQS4hBYso/U+9MIMOoiMCKSG9caJ6b6PpCx3OaoRodAF724uiiFp3rc/Jv6EqZxV2XtyLjTk/+dyD\nNIUY8dKE37flz7lsoijC7qxCldPRogkttA1CqrSuNjsxbTxm9rml2c/q6gK62FGMz0971nE2NDH5\nxuUQBAELhj3o97vnD5mL3YX7kWe7gBxrHj49+SUAYP6QuehfV1P23usO1By9aMsL0lAqvSYEkXoT\nbu55HbbXTe3pEt04X56LuLDYgL/noeKjPo+vpBaX1lJVCIfUhXCN88qoCQOeeYENWj1yrHn457E1\nPq99ePxThvAVrXFN2BjmuZ9pqxQBnWcGKFEUUeIow57C/fj3qXUAPPcZHU4HDhYdxaikYdKwlxe2\nv4KL9kIsHPFf6BnVHUVVJfj67PewVBVhTPIovH/0I78lsdbaWjyrU3updtXg/aMfYU/hAem5pBbU\n+r0hbKkqxvm6Xs9jk0e26Du9n/2oQRO23s+6wB2lf1xf9I/ri7LqcqlT2FDzQGTEpkvvaTj1JeC5\nSMm1XUCxowRfnv5GCuBfDvg5RiQOkT63q3AfAM9wqOW730BESDiWTVjitxw6wRMt41NGY0v+doZw\nE1QWwp4DYOexQjzgdEmP1UwQBMwbdD/yKy8iXBeGUJ0BLtGNVYfeb9dmRmuNDV+f/R4jEoeiZ1Q3\niKKIbRd2weGqRnp0z0ZLMpJ8NOwLFBfpaRotr3ABscCre9/y+5k82wV8e24Tdhfuh8NVjYlp45Bn\nuyD1LP767Abc128WFmfVN+16Oy4BkJbrbKjUUdapIfz7rS/6rEwEAIMvGYPvj3e445b87dJz3lnH\nmqOra0HwdoK6q++tQTk2og1RWDhiPnKteRibcpXP2GFBEKARNHCJbthqKvHkT882+nyvqO4+AQzU\nX2A8v305AM8sWlXOqkb3yAGg2uVpivfOke3k0osBqSqEDfr6/9AOny3F0N7xTbxbPfrG9kbfWN9x\nxGv0pjYPQTldfhafn/4GZY4yqWlxU+4WhOnCUOWskt4nQMCTo36NLgximREhir414eQ4T6eesjIR\n+ga3aE16I0YlDsOGnM2e16vLsdfiua93vPQkJqaNw66CfdL7j5ecwMcn6mt7Y5NHIevCTgDAff1m\nYXTyCBwrOYF/HFmNHlHdsM9yEGXVFeisCVarnA4pgGf2mYFvzm1Ej6huuKnHtGY/mxBuhkGrR3WD\npvqwZqaT9BoY3x8/5mUBAIaaBwW1JapnVDf0DDClrSeEXY1m4fO6dNY+AIjURzZ6zl7raBTCx0pO\nSP/teEN4Y85m7C7YhzzbBTwxckGz03NeSVQVwlqNBjeO7YYvs86huubKvvLSa/UorPIsWRbo/l1z\nsvJ3IrvBIuNeDQMY8AyFOVF2CmnGZPx97xocvngCZyvOY3a/mS1uxqOO4L8Xa/ckE84WJiEmQYvr\nRvTC+AY1JW8I59kuSs2V5ypy4Bbd+ObcRgCeaTAL7BYplBeOmI+y6nIphLuYUgEAGbHpWHr1M9ia\nvwP7LAelSTJaqqy6HNYaGwxaPRIumXrTH5fbhaMl2ThbcR4/NajFTkgdg4lp41r8vdGGKCzPfA4C\nBHx84jM4nA6Y9MYWfXZAXF/8edJSbM3f0aJad7BoIPhMMLJ0/DMotFuQFJGIoyXZGJ4wuNFnbut9\nIzJie+Ptg/+Qnqt2VaO8ugI/5G7FroK9SAxPkBaiAep7cjdsJSm0F6FrZFpH/FmKpKoQBgBztOeq\nTAlLGnYkY0gELFXFWH3837i3/8xm3+8W3fj3yXW4YC9AtbMGDpcDebYLAIDnxj6FHFs+Vh78B7pH\ndsXjI+Zj7ckvsCFnM27sMRVfnvkWn5z4HPsKD+FUef3B9v7Rj9DVlCqNS6TOFWggyZzrM/Ds33Yi\n/0gyogem+zRVPjXqMby0889SpyLAE4YLNj4lPc6ITUeB3SI97hnVzWc9Xu94VS9vrWdP4X6MTBza\n7BSr9lo7frd5ic9z13S5Gnek/6zJz/2Uvx0fZf/b57mRiUMDTuPYFO9nWtIZ61IhGl2rQj8YIg2R\n0kxecaExiNSbEGXw1HSvShru9zMh2hAMMQ/EK5nP4dltf0RFjRV/2LHC5z3FdestA8B/Dfklukd2\nxeQuE+B0O3Gw6ChKq8tgv+Qi/kqnuhDWqmAO6fbw83534oXtr0irvthr7ThSkg1bbSUqayphq63E\nPsshqbOMtzMGAGnscZQ+En1ieiMuLBZxYbF4ZOgD6GpKgyAIuD39ZtyefjNq3U78mJsFa60Np8rP\nwKAzYE6/WdLVcqG9iCEcLIJn7uhLdUsyYdqoLvhmZw5KKnxrp4nhZhhDImCrrcTwhMEorirFOWt9\njenGHlMxOH4AfsjdCgB4ePD9AODT/+DSptvwuubKQ8XH8J9TX+G29JsCFtktuqUANoZEYETiUOwp\n3I9NOVtwY4+pPk2fG87/iDRTijSlq/c+bJoxBcMTBiO/8iKm95ja5E90pVow9AEsznoZALB4zBOt\nmkQkVBeKqd0m4ZMTn0vPxYfFoW9ML1S7arCrYB8WjpgvNYXfnn4zACAxPAEfn/gPQ/gS6g3hK3hu\nXMBzUuxm6oI8Wz5Ol5/FOwffQ3mDoG3IXjeBAQDc0vMGTO02ye9B6W/B8xCNDs+OewoOZzVcohO9\nUlNRWmzHXX1vxYfHP4XL7YRbdGPNic8xOL6/Ty9N6gz+T659u0bjm505qKr2vW2j14bguXH/DZ2g\nhVajRXl1BRZteUF6PSY0BmmmFPxywM9RaLdIi410MaXi0aEPIcpgavTfTq+o7ugf2xdHSo7j+5wf\nMb3HlID3BN9r0Lt68ZjfITwkHDpBi+9zfsSFykLpxG6tsUkzfr0x2bNIvbcz0IOD7kV83ZKB5F98\nWByeGb0Qek3IZS3+cnXKaIRpQ3Gg6Ahu630TzOH1v/cvBtzj9zPeBTAanm9IjSGsVf5CDu0lPiwW\n56w5eGX3/0rP/XLAzwGISAg3o8rpQLQhEtGGaM+C7xBwderoVk+tZ9DqYai77+ztHertTbr94h5U\nuarxQ+4W/JC7RTphtpZ3QnxqOdHPECWvcIPn0LdXNx46YmjQhyDKEInfj/kdPj35BWJDYzEqcSgA\nNOo5C6BR50CvEG0I5g+di/kbngAAFFWVBOwx7G3WvrZrJsLrOvV4J/koqSqRQrjhbEze7QKeZmR/\nHYiosbaMntBr9RibMgpjU0a1+DPeFpEPjq9FrduJa7pcfdnfrybqC2GpOfrKvicMANd1nwyj3iiN\nU0wxJvk9eQLA9d2vbdfv9vaKPFJy3KejRiCiKKLaVQ2toIVOo/MJ3C9Or8dXZ7/HwLh+uDvjtkb3\nHCkQT3O0v2uXsLoQrvITwpdKDDfj4cG/aHNpZvSajn+fWodPT36JKEMkNIIGIxOHIiM2HbbaSnx9\n9nuf93p5O0Wds+YiMSIRte4aHCnO9vsdbtHtM7czyYf3nAAAa058BlNIBEYmDUOpowybcrfgum7X\nSBdeVxL1hjBrwkg1Jl9Wx5L20DemN27tfSMcTk/vya0XPNPp7bi4x3NRIAgodZQhIiQcttpKbM7N\nkoZBhelCMbPPDIxMHIoVu/8XZyrOA/DMwvP+0Y/xyNAHAn4vXarpmnBLQri9xIRGAwCOlZ6QnttZ\nsBd/GPe0z1jVG7pP8elMZazrYbshZ7PUe9srOSIRV6eM8RkuRfLU1ZSGCaljsbluCNfak19iZNIw\nrNjzfyhxlKK82or7B9wV5FJ2PvWFcN2Shk6GcFDpNDpM6TpRelxaXYajJdk+a6D6kxGTjmOlJ7Dh\n/I+IMURJAezVsFcuNSfwMRAW6jn0bVWdt05sj8huCNUa4HBV466+t2Jz3jbk2S7gnUPvSe/5n9GP\nI/GS4UjJEYkwhkTAoNVjUHx/aDXauscGjE+5CjqNDlVOB744sx7zh8zttL+HWker0eKuvrfiZz2v\nx+82L0aUIRLvHHofJXU9qgvsV+ZKS+oL4brZbtgcLS+39r4R6UWeye+dbhcEQUBcWCwqayvx4fFP\npfctGPYglu16DecqcvC/dWuYzug1HVO7TcJvf3hG6rlNgS3d8SckRyTWzR3t//cK0+ugD9HgwKli\n7M22YFif5sfhtlVcWAyWZz6HalcNQnUG2GurkGe7gBNlpwEADw66D0kRCY0+Z9Ib8eLV/wMBQsB+\nAdd3n4xxKaOkYTYkX+EhYYgPjcV5ay7OW+uXnT1vzcWbB/6GXw6457LnNlAi1YWwzlsTvsKHKMlN\nqjE54FCl7pFd8fbBf+BXdfcd+8f28UwQ4XahR2Q3jEu5CgAQrgtHsaMEL+74Mx4f+UiTq9nIUY2r\nFtUuzwIJpXUTUeg0OvSI7NrmTmcljlJszd+BA0VHkGe74BnjrQGAEL8XLhqNgElDU/HNzhy8tvYg\nHrtzMAb36vgZ5gRBQGjdggYT08bjswbjkZPCGwewVN5mxvoKgsAAVpD4sDgUOUoAAAPiMnC4+BgA\n4GDREWSXnsLA+H7BLF6nUtZZrAVM4Z4rqP2ninDL1d2viPmjla6LKRXPj/tv6fGNPaZhcpdMhOlC\nfcLpjj4/w8qD/0CuLR8fZ/8b8aFxcIkuhGg9K70U2otgq62EtcaKLqY0XNs1s9OD2i26sfLgexAh\nQidocbjkeKOVivz5n9GP+60FtsTmvG3SQvWXEkIr/T4PALMm90ZcZCg++P4Edh2zdEoINxSqMyBM\nF4qqupm04oO81CF1nquShkt9AyakjsEN3adg+e7XAeCKG0esXbJkyZLO/EK7vfkTUmtERBh8tqnT\nCliXdQ4V9loUlzswou/lndjo8ly6Py6HIAgI0YY0qh0mRSQgOSIRewsPIMeah+OlJ5FddgrHSk5g\nv+UQTpadxnlrLi7aC5FdehL7LYeQ2ckzF23K3YKNuT+h0G7BRXuhtHB6UkQi0owpSDOlwKiPQKox\nGX2ie+F83QQTMaFRSI5IhN1ZhWpXDfJsFxBb15HJn2MlJyDAU6v8467XpOefvuq3iNKbpCZeQQCm\nd5/qt6YtCAK6JZmwbts5aLUCMod0/tzfGTGentG/HvaQNI5UjdrjuFCTNFMKrkkbj+6RXTAwrh9i\nw2KQFG7GXstB7LccQklVKbqaUhtN/HK85CQ25f6EN/avQlFVMWIMUa0eLRGsfRER4X8RENXVhEP1\nOjz0swF467PDyDpcgNsn9kJsJCcLV4sh8QNwd9/b4BZFRBlM0Gv1eH3fOwA8UxSOTByKLfk7cLDo\nCC5UFmDVofcxd+DsDi1TjjUf2y/uglsU8UPuFgBAenRPdDWlISY0Gt0ju6JHVFe/nx2dPAKv7P5f\n/OfUV/jPqa98Xntw0H0Yah7Y6DMXKwvx2r6VAIDp3adIz/9p4gvQa/VIMSYhzZSCNw/8De7Kppto\ndVoNkmLDkWuxBWUsdtfINDw06L5O/U6Sh/CQcAxNGCQ99s4zDQDbLu6CXqvHrL4zpOdEUcRf9r0t\nPd5xcQ/2Ww5hTv+7kGpMVuwELaoLYQAY3T8Rn205gwvFdvy4Px8zJvQMdpGonWg1WlydOsbnuSdG\nLkCB3SLNeTsovj8+OfE5NuRsxp7CA7jP7USJoxSRelOLV8MJpKiqGCfLzqDUUYYSRxkqaqw4UnLc\nZ4H0XlE98Njwh1u0vS7GVGTEpKPGXQtjSARcoku6P/b3wx/gQMJg7CzYi3BdGJxuJ2rdTql2DQDr\nzn4HAPj1sHk+nVkGxGUgteJanDzlAgLPEukpQ4IReUWV+CLrHG4e172FvwRR+0qP7ombe16Prfnb\nUewoxZ7C/bgj/WaIEPHe0Y98VvGa1WcGVmf/G9WuGrx98B/oFtkF9/e/GxEh4T7jkZVAEEWxU3sw\nWSz+p068XGazye82z1204tm/7cQ1w1Nx77S+7fqdFFig/REMy3e9Ic2d7TV34GykR/eErbYSOda8\nunHM5ZjU5WqY9EbkWvOxu3A/alw1GBjfDwatHln5u6RxzoH8rOf16BfXB8aQCMQYottUo9yavxP/\nPPaxz3MmvRFR+khoNVo43U5Uu2owKnEokiISkRhullYuaujPaw7gwMkivPPkNdA0UZ6TeeV4+Z97\n4HKLyBySgvtvyLjsspN/cjoulGDRTy+gvKbC72vTe0zFjT2m4ljJCXyU/Z9GQ5uGJQzG7b1vksal\nXypY+8Js9r+WtiprwgAQGeGpFdgdnTcZAclLZtpYFJ0sRqTeBINWj9Pl57Dq0Pt+3/v1uQ0YmTgU\nh4uPS0s1bqprWm4oRBOCEQlDMCA+A4V2C8qrregZ1c3v+quXa1zKKCRFmHG6/ByOlZxASkQSbu19\nY6uDvaWX171To/DbmUPwxw/34cf9+bhjUi8YwzjrFAXPvMFz8O6hf0o9qAHgnr63Y5C5PyL1njDL\niE3H78c8jh0X9yArfyfOW3PhcFVjb+EB7C08gDBdGBxOB/rF9sG9/WdKn2uOtcYGh7PaZz7sjqTa\nmnCt042HX9mEqAg9npo9AgnR6u30ISdyvuL/5txGfHf+B9S4ahATGo1eUT2QakzGmhOf+bxviHkg\nuhhT8cWZ9dJzIxKG4K6+typqWr0/fXwAB081XxP2+nzLGXy6+Qzio0Ix75YB6JXC6UHbi5yPC7my\n11ZhV8E+FDtK0D2yK4Y1uH/sjyiK+OD4J9iS37jVKj4sDs+OfRJA/b5wuV0+i1fUup04YDmMvx7+\nFwDgxav/p8XrSLdEoJpwi0J46dKl2L9/PwRBwKJFizB4cP2Cz1u3bsWKFSug1WqRmZmJ+fPnN7mt\nzgphAHj+7ztx5oLntdsye+Im3u/qcHI/2YiiCKfbiZAG8wvbaipRUl0KU4gRBq1B6qWbY81Dod2C\nEXWLFiiNN4RXPXlNi2rRVdVO/H7VDhRXOCAIwB0Te+H60W0fw0zyPy7UpMZViwJ7IdKMKSisKsJz\n2/4IADCHxcGkN8FaW4HyahtqXDUI1Rpg1BtRWVspDZUDgNFJI3Bvv5nt+t/+ZTdH79ixA+fOncPq\n1atx6tQpLFq0CKtXr5Zef+GFF7Bq1SokJiZi9uzZuO6669C7t//VVDrbgtsH46cDF7D2x9P4Musc\nkuPCYY4OQ9fEljVLkPp4hz81ZNRHwKiPaPTeLqZUv/da1SrMoMOL88bgpwMX8I/1x/HxplP4avt5\n9O8egwE9YtErJQoJMWHQaZueOIMomPTaEOm4jTHU3xe2VBXDUlWMEI0OIZoQ1ABwuKrhqPIsgdnF\nlApRFHFd98kYah7YaRefzYZwVlYWpkzxDIPo1asXysvLYbPZYDQakZOTg6ioKCQne2ZCmjhxIrKy\nsmQTwtFGA24a1x2FpVX46eAFvPHpIQBAlFGPcIMOYQYddFoNdFoBGo3vD37pLENN7Y9LX2pq5zW3\nXy/9rBDwgZ+p+WVQYzEYdKjuxEUBKLBzF/13bGmKTqvBpGGp6JEcifU7z+P4+TLsOFqIHUfrO7+E\nG3QID9UhVK+FTquBVitAp6n7f60mYNN3a/7zDHQM+X02wHYDfl0QjhMeF8HTD/fAKTjgEmoAuBEV\nkoCaas9ohiHp0biqXxI0EC5rXeX20GwIFxUVYcCAAdLj2NhYWCwWGI1GWCwWxMbG+ryWk5PT5PZi\nYsKha+dZrAJV870evXs4Rg3MR1GZA/tPWFBYaoetyglLmQNOzjFNKpZqNsJsNrX6qt5sNmHkoBSI\nooicAiv2nbDgTF4FCkrsqKisRmVVLUqt1XC6RTidbq5aRgpyUfqXXq/FjEnBnSKz1b2j29qPq7TU\n3qbPX6ql91oGdYsBugHXDPGdv1gURbjcItwNTiKN/kLx0ocN3tvMz+H7uu+bL/1oU9u69HdvrozB\nEhdnRHGxLdjFIHj2RVWlA0VFbdsfYVoBYzMSMDYj8Oxz3uPI6XLD7fe61v9/oP6ebc0pJtD5KOAm\ngnSc8LiQj4b7whge0mn36i/7nnBCQgKKioqkx4WFhTCbzX5fKygoQEKCsqaJFATBs+gDp5huF9Em\nA2odnJ5PDjpzX3iPI94v9o/HhXzIbV80e8SMHz8e69d7hmocPnwYCQkJMBo93bbT0tJgs9mQm5sL\np9OJjRs3Yvz48R1bYiIiIpVotiY8fPhwDBgwAHfddRcEQcDixYuxdu1amEwmTJ06FUuWLMHChQsB\nANOnT0ePHj06vNBERERqoNrJOig4uD/kg/tCPrgv5ENu01byBg4REVGQMISJiIiChCFMREQUJAxh\nIiKiIGEIExERBQlDmIiIKEgYwkREREHCECYiIgqSTp+sg4iIiDxYEyYiIgoShjAREVGQMISJiIiC\nhCFMREQUJAxhIiKiIGEIExERBYku2AVoi6VLl2L//v0QBAGLFi3C4MGDg10k1du+fTt+/etfIz09\nHQDQp08fPPDAA3jiiaHA08QAAASaSURBVCfgcrlgNpvxxz/+EXq9Hp999hn+/ve/Q6PRYObMmbjz\nzjuDXHp1yM7Oxn/913/h/vvvx+zZs3HhwoUW//61tbV46qmnkJ+fD61WixdffBFdunQJ9p+kWJfu\ni6eeegqHDx9GdHQ0AGDu3LmYNGkS90UnWLZsGXbv3g2n04l58+Zh0KBByjguRIXavn27+NBDD4mi\nKIonT54UZ86cGeQSXRm2bdsmLliwwOe5p556Sly3bp0oiqL4yiuviP/85z/FyspKcdq0aWJFRYVY\nVVUl3njjjWJpaWkwiqwqlZWV4uzZs8VnnnlGfO+990RRbN3vv3btWnHJkiWiKIri5s2bxV//+tdB\n+1uUzt++ePLJJ8UNGzY0eh/3RcfKysoSH3jgAVEURbGkpEScOHGiYo4LxTZHZ2VlYcqUKQCAXr16\noby8HDabLcilujJt374d1157LQDgmmuuQVZWFvbv349BgwbBZDIhNDQUw4cPx549e4JcUuXT6/VY\nuXIlEhISpOda8/tnZWVh6tSpAIBx48Zxn7SBv33hD/dFxxs1ahReffVVAEBkZCSqqqoUc1woNoSL\niooQExMjPY6NjYXFYgliia4cJ0+exMMPP4y7774bW7ZsQVVVFfR6PQAgLi4OFosFRUVFiI2NlT7D\n/dM+dDodQkNDfZ5rze/f8HmNRgNBEFBTU9N5f4CK+NsXAPD+++/jvvvuw29+8xuUlJRwX3QCrVaL\n8PBwAMCaNWuQmZmpmONC0feEGxI5+2an6N69Ox555BHccMMNyMnJwX333QeXyyW9Hmg/cP90jtb+\n/twv7euWW25BdHQ0+vXrh7fffhuvv/46hg0b5vMe7ouO891332HNmjV49913MW3aNOl5OR8Xiq0J\nJyQkoKioSHpcWFgIs9kcxBJdGRITEzF9+nQIgoCuXbsiPj4e5eXlcDgcAICCggIkJCT43T/NNdvR\n5QkPD2/x75+QkCC1SNTW1kIURam2QG03duxY9OvXDwAwefJkZGdnc190ks2bN+PNN9/EypUrYTKZ\nFHNcKDaEx48fj/Xr1wMADh8+jISEBBiNxiCXSv0+++wzrFq1CgBgsVhQXFyM2267TdoX33zzDSZM\nmIAhQ4bg4MGDqKioQGVlJfbs2YORI0cGs+iqNW7cuBb//uPHj8fXX38NANi4cSNGjx4dzKKrzoIF\nC5CTkwPAc68+PT2d+6ITWK1WLFu2DG+99ZbUM10px4WiV1Favnw5du3aBUEQsHjxYmRkZAS7SKpn\ns9nw+OOPo6KiArW1tXjkkUfQr18/PPnkk6iurkZKSgpefPFFhISE4Ouvv8aqVasgCAJmz56Nn/3s\nZ8EuvuIdOnQIL7/8MvLy8qDT6ZCYmIjly5fjqaeeatHv73K58Mwzz+Ds2bPQ6/V46aWXkJycHOw/\nS5H87YvZs2fj7bffRlhYGMLDw/Hiiy8iLi6O+6KDrV69Gq+99hp69OghPffSSy/hmWeekf1xoegQ\nJiIiUjLFNkcTEREpHUOYiIgoSBjCREREQcIQJiIiChKGMBERUZAwhImIiIKEIUxERBQkDGEiIqIg\n+X9he1AYH71S+gAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f134d81abe0>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "BC77iNL2Gk68",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## NB : Discussion are at the end of each snippet of code"
      ]
    },
    {
      "metadata": {
        "id": "x1y3QwOFGmqg",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}