4-reuters.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "j1bsrXvzxBbl"
   },
   "source": [
    "# Classifying newswires with tf.keras and tf.data.\n",
    "\n",
    "In this notebook, you'll learn how to classify newswires from the [Reuters dataset](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/reuters) into categories. This tutorial was inspired by [this one](https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/3.6-classifying-newswires.ipynb). This notebook is similar to previous one. The principal difference is here, we will explore a multiclass text classification problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "fGBZG7rvwf2C"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_39NLiiGy9ey"
   },
   "source": [
    "\n",
    "## A first look at the dataset\n",
    "\n",
    "### Step 1) Downloading the dataset\n",
    "\n",
    "The dataset consists of a set of short [newswires](https://www.google.com/search?q=what+is+a+newswire) and their corresponding topics as published by Reuters in 1986 (the year Madonna's Papa Don't Preach came out, although I preferred You Can Call Me Al by Paul Simon 🎺)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "F-DuuRzay9KG"
   },
   "outputs": [],
   "source": [
    "(train_data, train_labels), (test_data, test_labels) = tf.keras.datasets.reuters.load_data(num_words=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "arCrVEMGzR7t"
   },
   "source": [
    "### Step 2) Exploring the dataset \n",
    "\n",
    "There are around 46 different topics. The set is divided into 8982 training examples and 2246 test examples. Each training example is a list of words represented as integers similar to the IMDB dataset, and the labels are integers up to 46."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "KL6sMDLVzNpF"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 2, 8, 43, 10, 447, 5, 25, 207, 270, 5, 3095, 111, 16, 369, 186, 90, 67, 7, 89, 5, 19, 102, 6, 19, 124, 15, 90, 67, 84, 22, 482, 26, 7, 48, 4, 49, 8, 864, 39, 209, 154, 6, 151, 6, 83, 11, 15, 22, 155, 11, 15, 7, 48, 9, 4579, 1005, 504, 6, 258, 6, 272, 11, 15, 22, 134, 44, 11, 15, 16, 8, 197, 1245, 90, 67, 52, 29, 209, 30, 32, 132, 6, 109, 15, 17, 12]\n",
      "3\n",
      "8982\n",
      "2246\n"
     ]
    }
   ],
   "source": [
    "print(train_data[0])\n",
    "print(train_labels[0])\n",
    "print(len(train_data))\n",
    "print(len(test_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "T9DdCQVQRlJQ"
   },
   "source": [
    "### Converting the integers to words\n",
    "\n",
    "We can get the dictionary from `tf.keras.datasets.reuters.get_word_index` which hashes the words to their corresponding integers. Let's try and convert a newswire from integers back into it's original text by first reversing this dictionary and then iterating over a newswire and converting the integers to strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "nVdKXJ7PRmUb"
   },
   "outputs": [],
   "source": [
    "# Dictionary that hashes words to their integer\n",
    "word_to_integer = tf.keras.datasets.reuters.get_word_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "b8iedZIuRoIG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['mdbl', 'fawc', 'degussa', 'woods', 'hanging', 'localized', 'sation', 'chanthaburi', 'refunding', 'hermann']\n",
      "the\n",
      "of\n",
      "UNK year to december 31 world group pre tax profit 2 63 billion marks vs 3 04 billion world group turnover 40 47 billion vs 44 38 billion world group investment in fixed assets 2 66 billion vs 2 46 billion parent company pre tax profit 1 97 billion vs 1 91 billion parent turnover 18 72 billion vs 20 46 billion parent domestic turnover 7 10 billion vs 8 14 billion parent foreign turnover 11 62 billion vs 12 32 billion parent investment in fixed assets 1 14 billion vs 884 mln reuter 3\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "print(list(word_to_integer.keys())[0:10])\n",
    "\n",
    "integer_to_word = dict([(value, key) for (key, value) in word_to_integer.items()])\n",
    "\n",
    "# Demonstrate how to find the word from an integer\n",
    "print(integer_to_word[1])\n",
    "print(integer_to_word[2])\n",
    "\n",
    "import random\n",
    "\n",
    "random_index = random.randint(0, 100)\n",
    "# We need to subtract 3 from the indices because 0 is \"padding\", 1 is \"start of sequence\", and 2 is \"unknown\"\n",
    "decoded_newswire = ' '.join([integer_to_word.get(i - 3, 'UNK') for i in train_data[random_index]])\n",
    "print(decoded_newswire)\n",
    "print(train_labels[random_index])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NZYEERu2R0y1"
   },
   "source": [
    "### Step 3) Format the data\n",
    "As before, we are going to multi-hot encode our newswire which is an array of integers into a 10,000 dimensional vector. We will place 1's in the indices of word-integers that occur in the newswire, and 0's for everything else."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "nbTtNYCIR4hs"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(8982, 10000)\n",
      "[0. 1. 1. ... 0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
    "    results = np.zeros((len(sequences), dimension), dtype=np.float32)\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
    "    return results\n",
    "\n",
    "\n",
    "train_data = vectorize_sequences(train_data)\n",
    "test_data = vectorize_sequences(test_data)\n",
    "\n",
    "print(train_data.shape)\n",
    "print(train_data[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MKo1m1ZrSELb"
   },
   "source": [
    "### Step 4) Format the labels\n",
    "\n",
    "We will also use [tf.keras.utils.to_categorical](https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical) to one hot encode our labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "b5Ew7zv-SJK-"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n",
      "[0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "LABEL_DIMENSIONS = 46\n",
    "\n",
    "print(train_labels[0]) # Before\n",
    "train_labels  = tf.keras.utils.to_categorical(train_labels, LABEL_DIMENSIONS)\n",
    "print(train_labels[0]) # After\n",
    "\n",
    "test_labels = tf.keras.utils.to_categorical(test_labels, LABEL_DIMENSIONS)\n",
    "\n",
    "# Needed later\n",
    "train_labels = train_labels.astype(np.float32)\n",
    "test_labels = test_labels.astype(np.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mupgnRsQSa6k"
   },
   "source": [
    "### Step 5) Create the model\n",
    "\n",
    "Our model is similar to the previous notebook, modified to work for a multiclass classification problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "VLsDpdIoSaIU"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 64)                640064    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 46)                2990      \n",
      "=================================================================\n",
      "Total params: 647,214\n",
      "Trainable params: 647,214\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "### Create a model\n",
    "model = tf.keras.Sequential()\n",
    "\n",
    "model.add(tf.keras.layers.Dense(64, activation=tf.nn.relu, input_shape=(10000,)))\n",
    "model.add(tf.keras.layers.Dense(64, activation=tf.nn.relu))\n",
    "model.add(tf.keras.layers.Dense(LABEL_DIMENSIONS, activation=tf.nn.softmax))\n",
    "\n",
    "optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.001)\n",
    "\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optimizer,\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E0KQorztSW0n"
   },
   "source": [
    "### Step 6) Validation set\n",
    "\n",
    "As before, we want to test our model on data it hasn't seen before, but before we get our final test accuracy. Here, we'll create a validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Ci6jFHB9TbWA"
   },
   "outputs": [],
   "source": [
    "VAL_SIZE = 1000\n",
    "\n",
    "val_data = train_data[:VAL_SIZE]\n",
    "partial_train_data = train_data[VAL_SIZE:]\n",
    "\n",
    "\n",
    "val_labels = train_labels[:VAL_SIZE]\n",
    "partial_train_labels = train_labels[VAL_SIZE:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CaNEaxWkTkAE"
   },
   "source": [
    "### Step 7) Create a tf.data Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Zx7aLE4vTnsS"
   },
   "outputs": [],
   "source": [
    "BATCH_SIZE = 512\n",
    "TRAINING_SIZE = partial_train_labels.shape[0]\n",
    "\n",
    "training_set = tf.data.Dataset.from_tensor_slices((partial_train_data, partial_train_labels))\n",
    "training_set = training_set.shuffle(TRAINING_SIZE).batch(BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "W8S-IkYMTsHm"
   },
   "source": [
    "### Step 8) Training your model\n",
    "Please be patient as this step may take a while."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "3OuvepxiTu7m"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000/1000==============================] - 0s 112us/sample - loss: 1.7716 - acc: 0.6540\n",
      "Epoch #1\t Training Loss: 1.94\tTraining Accuracy: 0.62\tValidation Loss: 1.77\tValidation Accuracy: 0.65\n",
      "1000/1000==============================] - 0s 75us/sample - loss: 1.3295 - acc: 0.7220\n",
      "Epoch #2\t Training Loss: 1.31\tTraining Accuracy: 0.73\tValidation Loss: 1.33\tValidation Accuracy: 0.72\n",
      "1000/1000==============================] - 0s 63us/sample - loss: 1.1428 - acc: 0.7590\n",
      "Epoch #3\t Training Loss: 0.98\tTraining Accuracy: 0.81\tValidation Loss: 1.14\tValidation Accuracy: 0.76\n",
      "1000/1000==============================] - 0s 63us/sample - loss: 1.0469 - acc: 0.7760\n",
      "Epoch #4\t Training Loss: 0.74\tTraining Accuracy: 0.86\tValidation Loss: 1.05\tValidation Accuracy: 0.78\n",
      "1000/1000==============================] - 0s 62us/sample - loss: 0.9862 - acc: 0.7910\n",
      "Epoch #5\t Training Loss: 0.57\tTraining Accuracy: 0.89\tValidation Loss: 0.99\tValidation Accuracy: 0.79\n",
      "1000/1000==============================] - 0s 65us/sample - loss: 0.9236 - acc: 0.7990\n",
      "Epoch #6\t Training Loss: 0.43\tTraining Accuracy: 0.93\tValidation Loss: 0.92\tValidation Accuracy: 0.80\n",
      "1000/1000==============================] - 0s 65us/sample - loss: 0.9247 - acc: 0.8090\n",
      "Epoch #7\t Training Loss: 0.35\tTraining Accuracy: 0.95\tValidation Loss: 0.92\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 69us/sample - loss: 0.8963 - acc: 0.8160\n",
      "Epoch #8\t Training Loss: 0.26\tTraining Accuracy: 0.97\tValidation Loss: 0.90\tValidation Accuracy: 0.82\n",
      "1000/1000==============================] - 0s 61us/sample - loss: 0.9123 - acc: 0.8150\n",
      "Epoch #9\t Training Loss: 0.22\tTraining Accuracy: 0.96\tValidation Loss: 0.91\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 69us/sample - loss: 0.9062 - acc: 0.8140\n",
      "Epoch #10\t Training Loss: 0.18\tTraining Accuracy: 0.97\tValidation Loss: 0.91\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 75us/sample - loss: 0.9397 - acc: 0.8100\n",
      "Epoch #11\t Training Loss: 0.16\tTraining Accuracy: 0.96\tValidation Loss: 0.94\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 65us/sample - loss: 0.9381 - acc: 0.8130\n",
      "Epoch #12\t Training Loss: 0.14\tTraining Accuracy: 0.96\tValidation Loss: 0.94\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 65us/sample - loss: 0.9686 - acc: 0.8120\n",
      "Epoch #13\t Training Loss: 0.13\tTraining Accuracy: 0.96\tValidation Loss: 0.97\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 80us/sample - loss: 0.9768 - acc: 0.8140\n",
      "Epoch #14\t Training Loss: 0.12\tTraining Accuracy: 0.96\tValidation Loss: 0.98\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 64us/sample - loss: 1.0036 - acc: 0.8120\n",
      "Epoch #15\t Training Loss: 0.12\tTraining Accuracy: 0.96\tValidation Loss: 1.00\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 69us/sample - loss: 1.0178 - acc: 0.8120\n",
      "Epoch #16\t Training Loss: 0.11\tTraining Accuracy: 0.96\tValidation Loss: 1.02\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 67us/sample - loss: 1.0400 - acc: 0.8060\n",
      "Epoch #17\t Training Loss: 0.11\tTraining Accuracy: 0.96\tValidation Loss: 1.04\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 68us/sample - loss: 1.0537 - acc: 0.8040\n",
      "Epoch #18\t Training Loss: 0.10\tTraining Accuracy: 0.96\tValidation Loss: 1.05\tValidation Accuracy: 0.80\n",
      "1000/1000==============================] - 0s 71us/sample - loss: 1.0798 - acc: 0.8050\n",
      "Epoch #19\t Training Loss: 0.11\tTraining Accuracy: 0.95\tValidation Loss: 1.08\tValidation Accuracy: 0.81\n",
      "1000/1000==============================] - 0s 63us/sample - loss: 1.0835 - acc: 0.8040\n",
      "Epoch #20\t Training Loss: 0.10\tTraining Accuracy: 0.95\tValidation Loss: 1.08\tValidation Accuracy: 0.80\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 20\n",
    "\n",
    "# Store list of metric values for plotting later\n",
    "tr_loss_list = []\n",
    "tr_accuracy_list = []\n",
    "val_loss_list = []\n",
    "val_accuracy_list = []\n",
    "\n",
    "for epoch in range(EPOCHS):\n",
    "    for newswires, labels in training_set:\n",
    "        # Calculate training loss and accuracy\n",
    "        tr_loss, tr_accuracy = model.train_on_batch(newswires, labels)\n",
    "  \n",
    "    # Calculate validation loss and accuracy\n",
    "    val_loss, val_accuracy = model.evaluate(val_data, val_labels)\n",
    "\n",
    "    # Add to the lists\n",
    "    tr_loss_list.append(tr_loss)\n",
    "    tr_accuracy_list.append(tr_accuracy)\n",
    "    val_loss_list.append(val_loss)\n",
    "    val_accuracy_list.append(val_accuracy)\n",
    "  \n",
    "    print(('Epoch #%d\\t Training Loss: %.2f\\tTraining Accuracy: %.2f\\t'\n",
    "           'Validation Loss: %.2f\\tValidation Accuracy: %.2f') % (epoch + 1, \n",
    "                                                                  tr_loss, \n",
    "                                                                  tr_accuracy,\n",
    "                                                                  val_loss, \n",
    "                                                                  val_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TR0thBUhUDTb"
   },
   "source": [
    "### Step 9) Plotting your loss and accuracy\n",
    "We are going to use `matplotlib` to plot our training and validation metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "FEekeJKbTqj-"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmUVOW19/HvZpZB5ogC0jhcZRSwBQ0aQI3B+aLIIMYhGoRojHqTNyw1xhhZrxpfJShXRa8jKDEax2jQG0nQONEgMogERcBGRECZRMWG/f7xnIaiqequ7hpOD7/PWmdVnbF2VVefXc9wnmPujoiISEXqxR2AiIjUDEoYIiKSFiUMERFJixKGiIikRQlDRETSooQhIiJpUcKQvDGz+ma21cwOzOa2cTKzQ8ws633TzexEM1uRML/UzI5LZ9sqvNb9ZnZNVfcv57g3mdlD2T6uxKdB3AFI9WVmWxNmmwLfAjui+UvdfXpljufuO4Dm2d62LnD3w7JxHDO7BDjP3QcnHPuSbBxbaj8lDEnJ3XedsKNfsJe4+/+m2t7MGrh7ST5iE5H8U5WUVFlU5fAnM3vczLYA55nZMWb2lpltNLM1ZjbZzBpG2zcwMzezgmh+WrT+JTPbYmZvmlnXym4brT/ZzP5tZpvM7E4z+5eZXZgi7nRivNTMPjSzL81scsK+9c3sDjPbYGbLgaHlfD7XmtmMMsummNnt0fNLzGxJ9H4+in79pzpWsZkNjp43NbNHo9gWA0eW2fY6M1seHXexmZ0RLe8F3AUcF1X3rU/4bG9I2H9c9N43mNkzZrZ/Op9NRcxsWBTPRjN71cwOS1h3jZl9amabzeyDhPd6tJnNi5avNbM/pPt6kgPurklThROwAjixzLKbgO3A6YQfH/sARwEDCKXXg4B/A5dH2zcAHCiI5qcB64FCoCHwJ2BaFbb9HrAFODNadzXwHXBhiveSTozPAi2BAuCL0vcOXA4sBjoBbYHZ4d8o6escBGwFmiUc+3OgMJo/PdrGgOOBr4He0boTgRUJxyoGBkfPbwP+AbQGugDvl9l2BLB/9Dc5N4phv2jdJcA/ysQ5Dbghen5SFGMfoAnw38Cr6Xw2Sd7/TcBD0fNuURzHR3+ja4Cl0fMewEqgQ7RtV+Cg6PkcYHT0vAUwIO7/hbo8qYQhmXrd3Z93953u/rW7z3H3t929xN2XA1OBQeXs/6S7F7n7d8B0womqstueBsx392ejdXcQkktSacb4f919k7uvIJycS19rBHCHuxe7+wbg5nJeZzmwiJDIAH4IfOnuRdH65919uQevAn8HkjZslzECuMndv3T3lYRSQ+LrPuHua6K/yWOEZF+YxnEBxgD3u/t8d/8GmAAMMrNOCduk+mzKMwp4zt1fjf5GNxOSzgCghJCcekTVmh9Hnx2ExH+ombV19y3u/naa70NyQAlDMvVJ4oyZHW5mfzWzz8xsM3Aj0K6c/T9LeL6N8hu6U217QGIc7u6EX+RJpRljWq9F+GVcnseA0dHzc6P50jhOM7O3zewLM9tI+HVf3mdVav/yYjCzC83svajqZyNweJrHhfD+dh3P3TcDXwIdE7apzN8s1XF3Ev5GHd19KfBfhL/D51EVZ4do04uA7sBSM3vHzE5J831IDihhSKbKdim9l/Cr+hB33xe4nlDlkktrCFVEAJiZsecJrqxMYlwDdE6Yr6jb7xPAiWbWkVDSeCyKcR/gSeD/EqqLWgEvpxnHZ6liMLODgLuB8UDb6LgfJBy3oi7AnxKquUqP14JQ9bU6jbgqc9x6hL/ZagB3n+buAwnVUfUJnwvuvtTdRxGqHf8f8JSZNckwFqkiJQzJthbAJuArM+sGXJqH13wB6Gdmp5tZA+AXQPscxfgEcKWZdTSztsCvy9vY3T8DXgceApa6+7JoVWOgEbAO2GFmpwEnVCKGa8yslYXrVC5PWNeckBTWEXLnTwkljFJrgU6ljfxJPA5cbGa9zawx4cT9mrunLLFVIuYzzGxw9Nq/IrQ7vW1m3cxsSPR6X0fTTsIb+LGZtYtKJJui97Yzw1ikipQwJNv+C7iAcDK4l9A4nVPuvhYYCdwObAAOBt4lXDeS7RjvJrQ1LCQ0yD6Zxj6PERqxd1VHuftG4CrgaULD8XBC4kvHbwklnRXAS8AjCcddANwJvBNtcxiQWO//CrAMWGtmiVVLpfv/jVA19HS0/4GEdo2MuPtiwmd+NyGZDQXOiNozGgO3EtqdPiOUaK6Ndj0FWGKhF95twEh3355pPFI1Fqp7RWoPM6tPqAIZ7u6vxR2PSG2hEobUCmY2NKqiaQz8htC75p2YwxKpVZQwpLY4FlhOqO74ETDM3VNVSYlIFahKSkRE0qIShoiIpKVWDT7Yrl07LygoiDsMEZEaY+7cuevdvbxu6LvkLGGYWWdCd7/9CH2np7r7H8tsY8AfCV3nthHG/pkXrbsAuC7a9CZ3f7ii1ywoKKCoqCh7b0JEpJYzs4pGK9gllyWMEuC/3H1edLXoXDN7xd3fT9jmZODQaBpA6KM9wMzaEPqaFxKSzVwze87dv8xhvCIiUo6ctWFEg5/Ni55vAZaw93ANZwKPRIOvvQW0ioZS/hHwirt/ESWJVyhnGGkREcm9vDR6W7inQV/2vOIUQgJJHEStOFqWanmyY481syIzK1q3bl22QhYRkTJy3uhtZs2Bp4Aro5Evs8rdpxKGp6awsFB9hEXy6LvvvqO4uJhvvvkm7lCkAk2aNKFTp040bJhqGLGK5TRhRIOMPQVMd/e/JNlkNXuOulk6euVqYHCZ5f/ITZQiUlXFxcW0aNGCgoICQh8WqY7cnQ0bNlBcXEzXrl0r3iGFnFVJRT2g/gdY4u63p9jsOeB8C44GNrn7GmAmcJKZtTaz1oT7BMzMRZzTp0NBAdSrFx6nT8/Fq4jUTt988w1t27ZVsqjmzIy2bdtmXBLMZQljIPBjYKGZzY+WXUM0dr+73wO8SOhS+yGhW+1F0bovzOz3hNFAAW509y+yHeD06TB2LGzbFuZXrgzzAGMyHp9TpG5QsqgZsvF3ylnCcPfXqeBmMNGd0S5Lse4B4IEchLbLtdfuThaltm0Ly5UwRET2VKeHBlm1qnLLRaT62LBhA3369KFPnz506NCBjh077prfvj29W2ZcdNFFLF26tNxtpkyZwvQs1VUfe+yxzJ8/v+INq6laNTRIZR14YKiGSrZcRLJv+vRQgl+1KvyfTZxY9dJ827Ztd518b7jhBpo3b84vf/nLPbZxd9ydevWS/zZ+8MEHK3ydyy5LWglSJ9XpEsbEidC06Z7LmjYNy0Uku0rbDFeuBPfdbYbZ7mjy4Ycf0r17d8aMGUOPHj1Ys2YNY8eOpbCwkB49enDjjTfu2rb0F39JSQmtWrViwoQJHHHEERxzzDF8/vnnAFx33XVMmjRp1/YTJkygf//+HHbYYbzxxhsAfPXVV5x99tl0796d4cOHU1hYWGFJYtq0afTq1YuePXtyzTXXAFBSUsKPf/zjXcsnT54MwB133EH37t3p3bs35513XnY/sEqo0yWM0l822frFIyKp5bPN8IMPPuCRRx6hsLAQgJtvvpk2bdpQUlLCkCFDGD58ON27d99jn02bNjFo0CBuvvlmrr76ah544AEmTJiw17HdnXfeeYfnnnuOG2+8kb/97W/ceeeddOjQgaeeeor33nuPfv36lRtfcXEx1113HUVFRbRs2ZITTzyRF154gfbt27N+/XoWLlwIwMaNGwG49dZbWblyJY0aNdq1LA51uoQB4Yu6YgXs3BkelSxEciOfbYYHH3zwrmQB8Pjjj9OvXz/69evHkiVLeP/99/faZ5999uHkk08G4Mgjj2TFihVJj33WWWfttc3rr7/OqFGjADjiiCPo0aNHufG9/fbbHH/88bRr146GDRty7rnnMnv2bA455BCWLl3KFVdcwcyZM2nZsiUAPXr04LzzzmP69OkZXXiXqTqfMEQkP1K1DeaizbBZs2a7ni9btow//vGPvPrqqyxYsIChQ4cmvR6hUaNGu57Xr1+fkpKSpMdu3LhxhdtUVdu2bVmwYAHHHXccU6ZM4dJLLwVg5syZjBs3jjlz5tC/f3927NiR1ddNlxKGiORFXG2GmzdvpkWLFuy7776sWbOGmTOzfw3wwIEDeeKJJwBYuHBh0hJMogEDBjBr1iw2bNhASUkJM2bMYNCgQaxbtw5355xzzuHGG29k3rx57Nixg+LiYo4//nhuvfVW1q9fz7aydXt5UqfbMEQkf+JqM+zXrx/du3fn8MMPp0uXLgwcODDrr/Hzn/+c888/n+7du++aSquTkunUqRO///3vGTx4MO7O6aefzqmnnsq8efO4+OKLcXfMjFtuuYWSkhLOPfdctmzZws6dO/nlL39JixYtsv4e0lGr7uldWFjouoGSSP4sWbKEbt26xR1G7EpKSigpKaFJkyYsW7aMk046iWXLltGgQfX6TZ7s72Vmc929MMUue6he70ZEpAbaunUrJ5xwAiUlJbg79957b7VLFtlQ+96RiEietWrVirlz58YdRs6p0VtERNKihCEiImlRwhARkbQoYYiISFqUMESkxhoyZMheF+JNmjSJ8ePHl7tf8+bNAfj0008ZPnx40m0GDx5MRd30J02atMdFdKecckpWxnq64YYbuO222zI+TrYpYYhIjTV69GhmzJixx7IZM2YwevTotPY/4IADePLJJ6v8+mUTxosvvkirVq2qfLzqLpf39H7AzD43s0Up1v/KzOZH0yIz22FmbaJ1K8xsYbQup1fiff11uNo0B6MFiEiODR8+nL/+9a+7bpi0YsUKPv30U4477rhd10b069ePXr168eyzz+61/4oVK+jZsycAX3/9NaNGjaJbt24MGzaMr7/+etd248eP3zU8+m9/+1sAJk+ezKeffsqQIUMYMmQIAAUFBaxfvx6A22+/nZ49e9KzZ89dw6OvWLGCbt268dOf/pQePXpw0kkn7fE6ycyfP5+jjz6a3r17M2zYML788stdr1865HnpwIf//Oc/d91Eqm/fvmzZsqXKn20yubwO4yHgLuCRZCvd/Q/AHwDM7HTgqjL37R7i7utzGB8AjRrBlClw9NHwox/l+tVEaq8rr4Rs30yuTx+IzrVJtWnThv79+/PSSy9x5plnMmPGDEaMGIGZ0aRJE55++mn23Xdf1q9fz9FHH80ZZ5yR8t7Wd999N02bNmXJkiUsWLBgjyHKJ06cSJs2bdixYwcnnHACCxYs4IorruD2229n1qxZtGvXbo9jzZ07lwcffJC3334bd2fAgAEMGjSI1q1bs2zZMh5//HHuu+8+RowYwVNPPVXuPS7OP/987rzzTgYNGsT111/P7373OyZNmsTNN9/Mxx9/TOPGjXdVg912221MmTKFgQMHsnXrVpo0aVKJT7tiOSthuPts4IsKNwxGA4/nKpby1K8P55wDL74ImzfHEYGIZCKxWiqxOsrdueaaa+jduzcnnngiq1evZu3atSmPM3v27F0n7t69e9O7d+9d65544gn69etH3759Wbx4cYWDC77++usMGzaMZs2a0bx5c8466yxee+01ALp27UqfPn2A8odRh3CPjo0bNzJo0CAALrjgAmbPnr0rxjFjxjBt2rRdV5UPHDiQq6++msmTJ7Nx48asX20e+5XeZtYUGApcnrDYgZfNzIF73X1qLmMYMQImT4bnn9f9MESqqrySQC6deeaZXHXVVcybN49t27Zx5JFHAjB9+nTWrVvH3LlzadiwIQUFBUmHNa/Ixx9/zG233cacOXNo3bo1F154YZWOU6p0eHQIQ6RXVCWVyl//+ldmz57N888/z8SJE1m4cCETJkzg1FNP5cUXX2TgwIHMnDmTww8/vMqxllUdGr1PB/5VpjrqWHfvB5wMXGZmP0i1s5mNNbMiMytat25dlQI45hjo1Ami0YlFpAZp3rw5Q4YM4Sc/+ckejd2bNm3ie9/7Hg0bNmTWrFmsXLmy3OP84Ac/4LHHHgNg0aJFLFiwAAjDozdr1oyWLVuydu1aXnrppV37tGjRImk7wXHHHcczzzzDtm3b+Oqrr3j66ac57rjjKv3eWrZsSevWrXeVTh599FEGDRrEzp07+eSTTxgyZAi33HILmzZtYuvWrXz00Uf06tWLX//61xx11FF88MEHlX7N8sRewgBGUaY6yt1XR4+fm9nTQH9gdrKdo9LHVAij1VYlgHr1QrXUlCmwcSPU4k4OIrXS6NGjGTZs2B49psaMGcPpp59Or169KCwsrPCX9vjx47nooovo1q0b3bp121VSOeKII+jbty+HH344nTt33mN49LFjxzJ06FAOOOAAZs2atWt5v379uPDCC+nfvz8Al1xyCX379i23+imVhx9+mHHjxrFt2zYOOuggHnzwQXbs2MF5553Hpk2bcHeuuOIKWrVqxW9+8xtmzZpFvXr16NGjx647CGZLToc3N7MC4AV375lifUvgY6Czu38VLWsG1HP3LdHzV4Ab3f1vFb1eJsObv/VWKGk8/DCcf36VDiFS52h485ol0+HNc9mt9nHgTeAwMys2s4vNbJyZjUvYbBjwcmmyiOwHvG5m7wHvAH9NJ1lkasCAcEOXP/0p168kIlIz5axKyt0rvHLG3R8idL9NXLYcOCI3UaVmFhq/J02CL7+E1q3zHYGISPVWHRq9q42RI6GkBJ55Ju5IRGqO2nTXztosG38nJYwERx4JXbuqWkokXU2aNGHDhg1KGtWcu7Nhw4aML+SrDr2kqo3SaqnbboMNG6Bt27gjEqneOnXqRHFxMVXt0i7506RJEzp16pTRMZQwyhg5Em65Bf7yF/jpT+OORqR6a9iwIV27do07DMkTVUmV0acPHHKILuITESlLCaOM0mqpV18FlbJFRHZTwkhi5EjYuROeeiruSEREqg8ljCR69YLDDlO1lIhIIiWMJMxCKeOf/4TPPos7GhGR6kEJI4URI1QtJSKSSAkjhR49oHt3VUuJiJRSwijHyJHw2mvw6adxRyIiEj8ljHKMGAHu8OSTcUciIhI/JYxyHH449O6taikREVDCqNCIEfCvf8Enn8QdiYhIvJQwKjBiRHhUtZSI1HVKGBU49FDo21fVUiIiShhpGDEi3PN75cq4IxERiU8u7+n9gJl9bmaLUqwfbGabzGx+NF2fsG6omS01sw/NbEKuYkxXabXUn/8cbxwiInHKZQnjIWBoBdu85u59oulGADOrD0wBTga6A6PNrHsO46zQQQdBYaHuxCcidVvOEoa7zwa+qMKu/YEP3X25u28HZgBnZjW4KhgxAoqKYPnyuCMREYlH3G0Yx5jZe2b2kpn1iJZ1BBI7sRZHy5Iys7FmVmRmRbm8TaSqpUSkroszYcwDurj7EcCdwDNVOYi7T3X3QncvbN++fVYDTNSlCwwYoGopEam7YksY7r7Z3bdGz18EGppZO2A10Dlh007RstiNGAHvvgvLlsUdiYhI/sWWMMysg5lZ9Lx/FMsGYA5wqJl1NbNGwCjgubjiTHTOOeFR12SISF2Uy261jwNvAoeZWbGZXWxm48xsXLTJcGCRmb0HTAZGeVACXA7MBJYAT7j74lzFWRmdO8P3v6+EISJ1U4NcHdjdR1ew/i7grhTrXgRezEVcmRo5En7xC/jggzA4oYhIXRF3L6ka5+yzwy1cVcoQkbpGCaOSOnaEY49VwhCRukcJowpGjoTFi8MkIlJXKGFUgaqlRKQuUsKogg4dYNCgkDDc445GRCQ/lDCqaOTI0FNq4cK4IxERyQ8ljCo66yyoV0/VUiJSdyhhVNH3vgfHH69qKRGpO5QwMjBiRBhXav78uCMREck9JYwMDBsG9eurWkpE6gYljAy0awcnnqhqKRGpG5QwMjRiRLgL39y5cUciIpJbShgZKikJj0cdFW6yNH16vPGIiOSKEkYGpk+Hq67aPb9qFYwdq6QhIrWTEkYGrr0Wtm3bc9m2bWG5iEhto4SRgVWrki9fuTK/cYiI5IMSRgYOPDD58kaN4Jtv8huLiEiuKWFkYOJEaNp0z2WNGsH27fCzn6mrrYjULrm8p/cDZva5mS1KsX6MmS0ws4Vm9oaZHZGwbkW0fL6ZFeUqxkyNGQNTp4beUWbh8YEH4De/gQcfhLvvjjtCEZHsydk9vYGHCPfsfiTF+o+BQe7+pZmdDEwFBiSsH+Lu63MYX1aMGROmRDt3wrvvhnt/9+oFxx0XT2wiItmUsxKGu88Gvihn/Rvu/mU0+xbQKVex5Fu9evDoo9C1KwwfDsXFcUckIpK56tKGcTHwUsK8Ay+b2VwzG1vejmY21syKzKxo3bp1OQ2yMlq1gmeeCd1szz5bjeAiUvPFnjDMbAghYfw6YfGx7t4POBm4zMx+kGp/d5/q7oXuXti+ffscR1s53bvDI4/AO+/AZZepEVxEarZYE4aZ9QbuB8509w2ly919dfT4OfA00D+eCDM3bBhcd11oDL/nnrijERGputgShpkdCPwF+LG7/ztheTMza1H6HDgJSNrTqqa44QY45RS44gp4/fW4oxERqZqc9ZIys8eBwUA7MysGfgs0BHD3e4DrgbbAf5sZQIm7FwL7AU9HyxoAj7n733IVZz7Urx/GlzrqqNAIPncudOwYd1QiIpVjXosq1gsLC72oqNpetsHixXD00dCjB/zzn9C4cdwRiUhdZ2Zzox/rFYq90bsu6dEDHn4Y3n4bLr9cjeAiUrMoYeTZWWeF0Wzvvz9cJS4iUlMoYcTgd7+Dk0+Gn/8c/vWvuKMREUmPEkYM6teHxx4LY08NHw6ffhp3RCIiFVPCiEmrVvD007BlS7gS/Ntv445IRKR8Shgx6tkzNIK/9RaceiqsWBF3RCIiqSlhxOzss+G++0LPqR49YNIk2LEj7qhERPamhFENXHJJuEZj8GC46ir4/vdh4cK4oxIR2ZMSRjVx4IHwwguhMXz5cujXL9yISW0bIlJdKGFUI2YwejQsWRIeb7oJ+vRR11sRqR6UMKqhdu3CsOgvvQRffw3HHhuGR9+8Oe7IRKQuS2vwQTM7GCh292/NbDDQG3jE3TfmMri6buhQWLQoDI8+eTI891y4T/hpp8UdmYhkw44doQq6uBi2bw9V0Nu3757SnW/WDO66K/fxpjta7VNAoZkdQrj39rPAY8ApuQpMgubNQ8+p0aND4/jpp8OoUfDHP8L3vhd3dCKSDnf47LPQmWXRovC4cCG8/36oRaiMhg2hUaPdU+PG0KFDbuIuK92EsdPdS8xsGHCnu99pZu/mMjDZ04ABYVj0W24JbRsvvwx33AE//nFo+xCR6mHz5tDrsTQplCaJDRt2b9OhA/TqBePHh+uxunYNJ/7GjfdOBonzDRtCvRgbEtJNGN+Z2WjgAuD0aFnD3IQkqTRqFHpOnX02/PSncMEF4T4b99wTvnAiknvffgtr14bps8/C4/Llu5PDypW7t23ePCSEs84KCaJXrzDfrl188Wci3YRxETAOmOjuH5tZV+DR3IUl5eneHV57LbRnTJgQvoBXXgmXXhq654pI5ZSUwOef704AiY9ll3355d77N2gAhx8OxxwDY8fuTg4HHhhviSDbKn0DJTNrDXR29wW5CanqqvsNlHLhk09Csnj66VA1deqpMG4c/OhHYZBDEQlKSuDjj+Hf/957Wr06+f1pWrSA/fYLVUgdOux+nmxZo0b5f0/ZUJkbKKXbS+ofwBnR9nOBz83sX+5+dZWjlKzo3BmeeioUg6dOhf/5H3j+eSgoCL90fvKT8IUWqQvcYc2a5Enho49C0ijVqhUcdhgMGRKqdEsTQGkS2G+/0PtIdkurhGFm77p7XzO7hFC6+K2ZLXD33hXs9wBwGvC5u/dMst6APxJ6W20DLnT3edG6C4Drok1vcveHK4qzLpYwytq+HZ59NlRXzZoVGsmGDQuNa4MGqYFcai73MLrzmjW7q4pKn69cuTsxbN26e58mTeDQQ+E//mPvqW1b/T9ADkoYQAMz2x8YAVxbiVgeAu4CHkmx/mTg0GgaANwNDDCzNsBvgULAgblm9py7J6k9lESNGsE554Tpgw9CqeOhh+CJJ8KvqXHjQmN569ZxRyoS7NiRPAkkm0/WBbVhQ+jUKXy/jz12z6TQuXPtakOIW7oljHOA3wD/cvfxZnYQ8Ad3PzuNfQuAF1KUMO4F/uHuj0fzS4HBpZO7X5psu1RUwkju669DwrjnnjCUepMm4VqO8ePhqKP0K0vyZ/16WLAg9CZasCBMixbBN9/svW2bNqF6aP/9d1cVJT4vnW/dWt/hTGS9hOHufwb+nDC/HKgwWaShI/BJwnxxtCzV8r2Y2VhgLMCBNbCL0PTp4R7fq1aFHhUTJ8KYMdl9jX32CaWKCy6Ad9+Fe++FadNCyaNv31DqOPfc0AVQJBu2bw8l3NKkUDqtWbN7m/bt4Ygj4Gc/C6WBxGSw337hGgSpXtJt9O4E3AkMjBa9BvzC3YtzFVi63H0q4epzCgsLK9flK2bTp4eG6W3bwvzKlWEesp80SvXtG0oat94aXv/uu0N33F/9Ci68MPzzHnZYbl5bag93+OqrcJHa5s3h5l+JiWHJkt0NzI0aha7gP/wh9O69e1JnjJon3SqpVwhDgZRee3EeMMbdf5jGvgWoSiqpgoI9L/Ip1aVL/u6+5w5vvgn//d+h2uq778I/9mWXhTGr1DW3dvv223CCX7ly98l/82bYtGnP+bLrtmyBnTv3Pl6nTnsmhd69Q+mhoS7zrbYqUyWVbsKY7+59KlqWYt8CUieMU4HLCb2kBgCT3b1/1Og9F+gXbToPONLdvyjvtWpawqhXL3nfb7Pk/4y5tnYt3H9/KIEUF4cqsvHj4eKLQ/WB1GzffReGrCgqgjlzwuPChWF5WU2bQsuWsO++e0/Jlh9wQLhQrU2b/L8vyUwuEsbfgQeB0l/4o4GL3P2ECvZ7nFBaaAesJfR8agjg7vdE3WrvAoYSutVe5O5F0b4/Aa6JDjXR3R+sKM6aljCqQwkjmZKSMDLulCnw6quhSmHkSLj8cujfP764JH07doQ2hKKi3dP8+bsbl1u1gsLC3dOhh+5OBC1ahCuXpW7IRcLoQmi3HvIdAAAQ90lEQVTDOIbQzfUN4Ofu/km5O+ZZTUsYZdswIPyymzo1d20YlfX++6G66uGHQ//2wsKQOEaODL2tJHPFxWFYinr1QhVg6VSZ+eLi3aWGoiKYNy+0MUC4+OzII8Pf7qijwuPBB6tnkQRZTxgpXuRKd59UpZ1zpKYlDMhPL6ls2LwZHn00jLn/wQfhoqdLLgk9rAoK4o6u5ti+PfzSf+ON0Hb0xhvhZJ8tTZqEjg2JpYfDDlNblKSWr4Sxyt2rVT/Wmpgwahr3cAX5lCnwzDNh/rTTwqBr7qHtJdVU3vr99w8nur59w/Pa8ut37drdieHNN8Ov/9JqoS5dwuf2/e+HHws7d4aqpNLH0qmi+R07wuinRx0VeiOpgVkqIxdXeid9nQz2lRrKDI4/PkyffBKu6bjvvjB+VbJtzULVSdkpcbkZfJHQnaF9+93Jo2/fcF/zQw/N/hW7O3aEE3pxcWj4bdo0VN80bbr7eaNG6SevkpLQiJyYIJYvD+saNQrVQj/7WUgQxxwTGopFahKVMCRjO3eGE27ZJFCZUsLmzfDee+HCwvnzw+Pixbt78DRrFi7y6tNndyLp2TP1xV3ffBNGIC0u3vMx8fmaNSFplKdevd1JpGwySXxctQrefnt3u0GHDiExlE59+6rNR6qnrFVJmdkWQiP3XquAfdy9WvWlUMKoXbZvD0mjNIGUJpPSweUaNIBu3cLJuHHjPZNB4t3NSrVoEa4T6Nhx78cmTcLJftu2vR+TLUt8/OqrcBFaafXSMceE6qbaUq0mtVte2jCqIyWM2m/nzlDNk5hA5s8P1UGdOqVOCB07hi6jIrKnfLVhiORdvXpwyCFhOuecuKMRqVs08K+IiKRFCUNERNKihCEiImlRwhARkbQoYYiISFqUMEREJC1KGCIikhYlDBERSYsShoiIpEUJQ0RE0qKEISIiaclpwjCzoWa21Mw+NLMJSdbfYWbzo+nfZrYxYd2OhHXP5TJOERGpWM4ShpnVB6YAJwPdgdFm1j1xG3e/yt37uHsfwj3D/5Kw+uvSde5+Rq7irOmmTw+3SK1XLzxOnx53RCJSW+WyhNEf+NDdl7v7dmAGcGY5248GHs9hPLXO9OkwdiysXBluf7pyZZhX0hCRXMhlwugIfJIwXxwt24uZdQG6Aq8mLG5iZkVm9paZ/WeqFzGzsdF2RevWrctG3DXGtdeGm/gk2rYtLBcRybbq0ug9CnjS3RNvmNkluqnHucAkMzs42Y7uPtXdC929sH379vmItdpYtapyy0VEMpHLhLEa6Jww3ylalswoylRHufvq6HE58A+gb/ZDrNkOTHFH9VTLRUQykcuEMQc41My6mlkjQlLYq7eTmR0OtAbeTFjW2swaR8/bAQOB93MYa400cSI0bbrnsqZNw3IRkWzLWcJw9xLgcmAmsAR4wt0Xm9mNZpbY62kUMMP3vLl4N6DIzN4DZgE3u7sSRhljxsDUqdClC5iFx6lTw3IRkWyzPc/TNVthYaEXFRXFHYaISI1hZnOj9uIKVZdGbxERqeaUMEREJC1KGCIikhYlDBERSYsShoiIpEUJQ0RE0qKEISIiaVHCEBGRtChhiIhIWpQwREQkLUoYdZzu2Cci6WoQdwASn9I79pXehKn0jn2gAQxFZG8qYdRhumOfiFSGEkYdpjv2iUhlKGHUYbpjn4hUhhJGHaY79olIZShh1GG6Y5+IVIZ6SdVxY8YoQYhIenJawjCzoWa21Mw+NLMJSdZfaGbrzGx+NF2SsO4CM1sWTRfkMk4REalYzkoYZlYfmAL8ECgG5pjZc+7+fplN/+Tul5fZtw3wW6AQcGButO+XuYpXRETKl8sSRn/gQ3df7u7bgRnAmWnu+yPgFXf/IkoSrwBDcxSniIikIZcJoyPwScJ8cbSsrLPNbIGZPWlmnSu5L2Y21syKzKxo3bp12YhbRESSiLuX1PNAgbv3JpQiHq7sAdx9qrsXunth+/btsx6giIgEuUwYq4HOCfOdomW7uPsGd/82mr0fODLdfUVEJL9ymTDmAIeaWVczawSMAp5L3MDM9k+YPQNYEj2fCZxkZq3NrDVwUrRMRERikrOE4e4lwOWEE/0S4Al3X2xmN5rZGdFmV5jZYjN7D7gCuDDa9wvg94SkMwe4MVom1YyGRxepO8zd444hawoLC72oqCjuMOqMssOjQxhaRFeLi9QcZjbX3QvT2TbuRm+pwTQ8ukjdooQhVabh0UXqFiUMqTINjy5StyhhSJVpeHSRukUJQ6pMw6OL1C0a3lwyouHRReoOlTBERCQtShgSK134J1JzqEpKYlP2wr+VK8M8qJpLpDpSCUNiowv/RGoWJQyJjS78E6lZlDAkNrrwT6RmUcKQ2OjCP5GaRQlDYqML/0RqFvWSkljpwj+RmkMlDBERSYsShtRouvBPJH9ymjDMbKiZLTWzD81sQpL1V5vZ+2a2wMz+bmZdEtbtMLP50fRc2X1FSi/8W7kS3Hdf+KekIZIbObtFq5nVB/4N/BAoJtybe7S7v5+wzRDgbXffZmbjgcHuPjJat9Xdm1fmNXWL1rqloCAkibK6dIEVK/IdjUjNVF1u0dof+NDdl7v7dmAGcGbiBu4+y91Lr/V9C+iUw3ikltGFfyL5lcuE0RH4JGG+OFqWysXASwnzTcysyMzeMrP/zEWAUrPpwj+R/KoWjd5mdh5QCPwhYXGXqJh0LjDJzA5Ose/YKLEUrVu3Lg/RSnWRjQv/1Ggukr5cJozVQOeE+U7Rsj2Y2YnAtcAZ7v5t6XJ3Xx09Lgf+AfRN9iLuPtXdC929sH379tmLXqq9TC/8U6O5SOXkstG7AaHR+wRCopgDnOvuixO26Qs8CQx192UJy1sD29z9WzNrB7wJnJnYYJ6MGr2lMtRoLlK5Ru+cXent7iVmdjkwE6gPPODui83sRqDI3Z8jVEE1B/5sZgCr3P0MoBtwr5ntJJSCbq4oWYhUlhrNRSonp0ODuPuLwItlll2f8PzEFPu9AfTKZWwiBx6YvIRRmUbz6dPD/TtWrQr7TZyooU6k9qoWjd4icci00VxtIFLXKGFInZVpo3k27hioXlpSk+Ss0TsOavSWfKpXL5QsyjKDnTsr3r/sPc0hlHA0xLvkU3W50lukVsv0wkGVUKSmUcIQqaJM20Ay7aWVjTYUJRypDCUMkSrKtA0k7hJKdWi0V8KqWZQwRDIwZky4yG/nzvBYmbaHuEsocVeJVYeEFbcalzDdvdZMRx55pIvUJNOmuXfp4m4WHqdNS3/fLl3cw6l2z6lLl/T2N0u+v1n6sTdtuue+TZum/x4yjb80hqp+ftnYPxOZfn7ZQriQOq1zbOwn+WxOShhSl8R9wq7pCSsbJ+w4E36mr19KCUOkjsjkhJHpCTPTE37cCSvT/eP+/LJVQlHCEJG0xPkLOe4Tbl1PeKUqkzDU6C1Sh8XZaB93L7NM98+000HcnR6qQglDRKok0xN+6THiSliZ7p9pwok7YVZJukWRmjCpSkqkbomzl1TcvZziaMPQWFIiIlUU9/D22Xj9yowlpYQhIlKHafBBERHJOiUMERFJixKGiIikRQlDRETSooQhIiJpqVW9pMxsHbAy7jhSaAesjzuIcii+zCi+zCi+zGQSXxd3b5/OhrUqYVRnZlaUbte1OCi+zCi+zCi+zOQrPlVJiYhIWpQwREQkLUoY+TM17gAqoPgyo/gyo/gyk5f41IYhIiJpUQlDRETSooQhIiJpUcLIIjPrbGazzOx9M1tsZr9Iss1gM9tkZvOj6fo8x7jCzBZGr73X0L4WTDazD81sgZn1y2NshyV8LvPNbLOZXVlmm7x+fmb2gJl9bmaLEpa1MbNXzGxZ9Ng6xb4XRNssM7ML8hjfH8zsg+jv97SZtUqxb7nfhRzGd4OZrU74G56SYt+hZrY0+i5OyGN8f0qIbYWZzU+xbz4+v6TnlNi+g+neOENTxROwP9Avet4C+DfQvcw2g4EXYoxxBdCunPWnAC8BBhwNvB1TnPWBzwgXFcX2+QE/APoBixKW3QpMiJ5PAG5Jsl8bYHn02Dp63jpP8Z0ENIie35IsvnS+CzmM7wbgl2n8/T8CDgIaAe+V/V/KVXxl1v8/4PoYP7+k55S4voMqYWSRu69x93nR8y3AEqBjvFFV2pnAIx68BbQys/1jiOME4CN3j/XKfXefDXxRZvGZwMPR84eB/0yy64+AV9z9C3f/EngFGJqP+Nz9ZXcviWbfAjpl+3XTleLzS0d/4EN3X+7u24EZhM89q8qLz8wMGAE8nu3XTVc555RYvoNKGDliZgVAX+DtJKuPMbP3zOwlM+uR18DAgZfNbK6ZjU2yviPwScJ8MfEkvVGk/keN8/MD2M/d10TPPwP2S7JNdfkcf0IoMSZT0Xchly6PqsweSFGdUh0+v+OAte6+LMX6vH5+Zc4psXwHlTBywMyaA08BV7r75jKr5xGqWY4A7gSeyXN4x7p7P+Bk4DIz+0GeX79CZtYIOAP4c5LVcX9+e/BQ9q+WfdPN7FqgBJieYpO4vgt3AwcDfYA1hGqf6mg05Zcu8vb5lXdOyed3UAkjy8ysIeEPO93d/1J2vbtvdvet0fMXgYZm1i5f8bn76ujxc+BpQtE/0Wqgc8J8p2hZPp0MzHP3tWVXxP35RdaWVtNFj58n2SbWz9HMLgROA8ZEJ5S9pPFdyAl3X+vuO9x9J3BfiteN+/NrAJwF/CnVNvn6/FKcU2L5DiphZFFU5/k/wBJ3vz3FNh2i7TCz/oS/wYY8xdfMzFqUPic0ji4qs9lzwPlRb6mjgU0JRd98SfnLLs7PL8FzQGmPkwuAZ5NsMxM4ycxaR1UuJ0XLcs7MhgL/BzjD3bel2Cad70Ku4ktsExuW4nXnAIeaWdeoxDmK8Lnny4nAB+5enGxlvj6/cs4p8XwHc9nCX9cm4FhC0XABMD+aTgHGAeOibS4HFhN6fbwFfD+P8R0Uve57UQzXRssT4zNgCqGHykKgMM+fYTNCAmiZsCy2z4+QuNYA3xHqgC8G2gJ/B5YB/wu0ibYtBO5P2PcnwIfRdFEe4/uQUHdd+h28J9r2AODF8r4LeYrv0ei7tYBw4tu/bHzR/CmEXkEf5TO+aPlDpd+5hG3j+PxSnVNi+Q5qaBAREUmLqqRERCQtShgiIpIWJQwREUmLEoaIiKRFCUNERNKihCFSATPbYXuOopu1kVPNrCBxpFSR6qxB3AGI1ABfu3ufuIMQiZtKGCJVFN0P4dbongjvmNkh0fICM3s1Glzv72Z2YLR8Pwv3p3gvmr4fHaq+md0X3e/gZTPbJ9r+iug+CAvMbEZMb1NkFyUMkYrtU6ZKamTCuk3u3gu4C5gULbsTeNjdexMG/pscLZ8M/NPDwIn9CFcIAxwKTHH3HsBG4Oxo+QSgb3Sccbl6cyLp0pXeIhUws63u3jzJ8hXA8e6+PBog7jN3b2tm6wnDXXwXLV/j7u3MbB3Qyd2/TThGAeGeBYdG878GGrr7TWb2N2ArYUTeZzwadFEkLiphiGTGUzyvjG8Tnu9gd9viqYRxvfoBc6IRVEVio4QhkpmRCY9vRs/fIIyuCjAGeC16/ndgPICZ1TezlqkOamb1gM7uPgv4NdAS2KuUI5JP+sUiUrF9zGx+wvzf3L20a21rM1tAKCWMjpb9HHjQzH4FrAMuipb/AphqZhcTShLjCSOlJlMfmBYlFQMmu/vGrL0jkSpQG4ZIFUVtGIXuvj7uWETyQVVSIiKSFpUwREQkLSphiIhIWpQwREQkLUoYIiKSFiUMERFJixKGiIik5f8DLsAh1Z2x7JcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "epochs = range(1, EPOCHS + 1)\n",
    "\n",
    "# \"bo\" specifies \"blue dot\"\n",
    "plt.plot(epochs, tr_loss_list, 'bo', label='Training loss')\n",
    "# b specifies a \"solid blue line\"\n",
    "plt.plot(epochs, val_loss_list, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "XWybdzYEUE7p"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XecVNX5x/HPQxNQOthAikpEBBbWpdhbQDQqsQKusUWJJth+UUOiUYNimho1IUY02LIKRKNiYokgisbGgoACKoggTVyKFBeRhef3x7m7DMvuzmyZmS3f9+t1XzO3P3N39j5zzrn3XHN3REREylIv3QGIiEj1p2QhIiJxKVmIiEhcShYiIhKXkoWIiMSlZCEiInEpWUjCzKy+mW02s45VuWw6mdnBZlbl14+b2ffNbEnM+Cdmdkwiy1ZgXw+b2a8qur5IIhqkOwBJHjPbHDPaFNgKbI/Gf+LuOeXZnrtvB/aq6mXrAnc/pCq2Y2aXARe4+/Ex276sKrYtUhYli1rM3YtO1tEv18vcfUppy5tZA3cvSEVsIvHo+1i9qBqqDjOzO8xsopk9ZWabgAvM7Agze9fMvjazVWZ2v5k1jJZvYGZuZp2j8X9E818ys01m9o6ZdSnvstH8U8zsUzPbYGZ/NrP/mdnFpcSdSIw/MbNFZrbezO6PWbe+mf3JzNaa2WJgcBnH5yYzm1Bs2lgzuyd6f5mZLYg+z2fRr/7StrXczI6P3jc1syei2OYBhxdb9mYzWxxtd56ZnRFN7wn8BTgmquJbE3Nsb4tZ/4ros681s+fMbL9Ejk15jnNhPGY2xczWmdmXZnZjzH5+HR2TjWaWa2b7l1TlZ2ZvFf6do+M5PdrPOuBmM+tqZtOifayJjluLmPU7RZ8xL5p/n5k1jmI+NGa5/cws38zalPZ5JQ5311AHBmAJ8P1i0+4AvgNOJ/xwaAL0BfoTSp0HAp8CI6PlGwAOdI7G/wGsAbKAhsBE4B8VWHZvYBMwJJr3f8A24OJSPksiMT4PtAA6A+sKPzswEpgHdADaANPDv0GJ+zkQ2AzsGbPtr4CsaPz0aBkDTgS2AL2ied8HlsRsazlwfPT+LuB1oBXQCZhfbNnzgP2iv8n5UQz7RPMuA14vFuc/gNui94OiGHsDjYG/Aq8lcmzKeZxbAKuBa4A9gOZAv2jeL4E5QNfoM/QGWgMHFz/WwFuFf+fosxUAVwL1Cd/H7wEnAY2i78n/gLtiPs9H0fHcM1r+qGjeOGBMzH5+Djyb7v/DmjykPQANKfpDl54sXouz3vXAP6P3JSWAv8UsewbwUQWWvRR4M2aeAasoJVkkGOOAmPn/Aq6P3k8nVMcVzju1+Ams2LbfBc6P3p8CfFLGsv8Gfha9LytZfBH7twB+GrtsCdv9CPhB9D5esngMuDNmXnNCO1WHeMemnMf5R8CMUpb7rDDeYtMTSRaL48RwTuF+gWOAL4H6JSx3FPA5YNH4bOCsqv6/qkuDqqFkWeyImXUzs/9E1QobgdFA2zLW/zLmfT5lN2qXtuz+sXF4+O9eXtpGEowxoX0BS8uIF+BJYHj0/vxovDCO08zsvaiK5GvCr/qyjlWh/cqKwcwuNrM5UVXK10C3BLcL4fMVbc/dNwLrgfYxyyT0N4tznA8gJIWSlDUvnuLfx33NbJKZrYhieLRYDEs8XEyxC3f/H6GUcrSZ9QA6Av+pYEyC2iwk/NKM9SDhl+zB7t4cuIXwSz+ZVhF++QJgZsauJ7fiKhPjKsJJplC8S3snAd83s/aEarInoxibAE8DvyVUEbUE/ptgHF+WFoOZHQg8QKiKaRNt9+OY7ca7zHcloWqrcHvNCNVdKxKIq7iyjvMy4KBS1itt3jdRTE1jpu1bbJnin+/3hKv4ekYxXFwshk5mVr+UOB4HLiCUgia5+9ZSlpMEKFlIcc2ADcA3UQPhT1Kwz38DmWZ2upk1INSDt0tSjJOAa82sfdTY+YuyFnb3LwlVJY8SqqAWRrP2INSj5wHbzew0Qt16ojH8ysxaWrgPZWTMvL0IJ8w8Qt68nFCyKLQa6BDb0FzMU8CPzayXme1BSGZvunupJbUylHWcJwMdzWykme1hZs3NrF8072HgDjM7yILeZtaakCS/JFxIUd/MRhCT2MqI4Rtgg5kdQKgKK/QOsBa408JFA03M7KiY+U8Qqq3OJyQOqQQlCynu58BFhAbnBwkN0Unl7quBocA9hH/+g4APCL8oqzrGB4CpwIfADELpIJ4nCW0QRVVQ7v41cB3wLKGR+BxC0kvErYQSzhLgJWJOZO4+F/gz8H60zCHAezHrvgosBFabWWx1UuH6LxOqi56N1u8IZCcYV3GlHmd33wAMBM4mJLBPgeOi2X8EniMc542ExubGUfXi5cCvCBc7HFzss5XkVqAfIWlNBp6JiaEAOA04lFDK+ILwdyicv4Twd97q7m+X87NLMYWNPyLVRlStsBI4x93fTHc8UnOZ2eOERvPb0h1LTaeb8qRaMLPBhCuPthAuvdxG+HUtUiFR+88QoGe6Y6kNVA0l1cXRwGJCXf3JwJlqkJSKMrPfEu71uNPdv0h3PLWBqqFERCQulSxERCSuWtNm0bZtW+/cuXO6wxARqVFmzpy5xt3LulQdqEXJonPnzuTm5qY7DBGRGsXM4vViAKgaSkREEqBkISIicSlZiIhIXEoWIiISl5KFiIjEpWQhUgk5OdC5M9SrF15zctIdkUhy1JpLZ0VSLScHRoyA/PwwvnRpGAfIrmg/ryLVlEoWIhV00007E0Wh/PwwXaS2UbIQqaAvSumerrTpIjWZkoXUaOlsM+hYygNZS5suUpMpWUiNVdhmsHQpuO9sM0hVwhgzBpo23XVa06Zhukhto2QhNVZVtBlUpmSSnQ3jxkGnTmAWXseNK1/jtq6mkpqi1jzPIisry9WRYN1Sr14oURRnBjt2xF+/+NVMEEoG5T3hV1S69y8CYGYz3T0r3nIqWUiNVdk2g3RfzZTuklFVqOz+FX8N4u61Yjj88MNd6pZ//MO9aVP3UL4IQ9OmYXoizHZdt3AwS27cVbX/yn7+yqrs/hV/9QDkegLn2LSf5KtqULKom/7xD/dOncIJtlOn8v2jdupU8sm6U6fkxFrV+6+K+NN5/BR/9aBkIRJHun8ZprtklO791/X4Cz9DRZNdVVGyEElAuv9Za/IvY61fufXT/WOlULVIFsBg4BNgETCqhPmdgKnAXOB1oEPMvO3A7GiYHG9fShY1U7pP1jVZun9Zp7vOv6bHn+5quEJpTxZAfeAz4ECgETAH6F5smX8CF0XvTwSeiJm3uTz7U7KoearLL6uaLN1tNpU9WdXl+NOd7ApVh2RxBPBKzPgvgV8WW2YecED03oCNMfOULGq52tJAWFPV9GRd0+NPdzVYoUSTRTLvs2gPLIsZXx5NizUHOCt6fybQzMzaROONzSzXzN41sx+WtAMzGxEtk5uXl1eVsUsKqCO+9KqKO9DTqabHX9nuYlL9/5Pum/KuB44zsw+A44AVhLYKgE4e7io8H7jXzA4qvrK7j3P3LHfPateuXcqClqqhjvjSLzsbliwJd7wvWVJzTrSFanL8lU12qf7/SWayWAEcEDPeIZpWxN1XuvtZ7t4HuCma9nX0uiJ6XUxo/O6TxFglDdQRn9R1lUl2qf7/SWaymAF0NbMuZtYIGAZMjl3AzNqaWWEMvwTGR9NbmdkehcsARwHzkxirpEFNr0YQSadU//8ktSNBMzsVuJdwZdR4dx9jZqMJDSqTzewc4LeAA9OBn7n7VjM7EngQ2EFIaPe6+9/L2pc6EhQRKb9EOxJUr7MiInWYep0VEZEqo2QhIiJxKVlIpdSp/vxF6rAG6Q5Aaq7iT3orfAY26IomkdpGJQupsHQ/aU5EUkfJQipM3XWI1B1KFlJh6q5DpO5QspAKU3cdInWHkoVUmLrrEKk7dDWUVEp2tpKDSF2gkoWIiMSlZCEiInEpWYiISFxKFiIiEpeShYiIxKVkISIicSlZiIhIXEoWIiISl5KFiIjEldRkYWaDzewTM1tkZqNKmN/JzKaa2Vwze93MOsTMu8jMFkbDRcmMU0REypa0ZGFm9YGxwClAd2C4mXUvtthdwOPu3gsYDfw2Wrc1cCvQH+gH3GpmrZIVq4iIlC2ZJYt+wCJ3X+zu3wETgCHFlukOvBa9nxYz/2TgVXdf5+7rgVeBwUmMVUREypDMZNEeWBYzvjyaFmsOcFb0/kygmZm1SXBdERFJkXQ3cF8PHGdmHwDHASuA7YmubGYjzCzXzHLz8vKSFaOISJ2XzGSxAjggZrxDNK2Iu69097PcvQ9wUzTt60TWjZYd5+5Z7p7Vrl27qo5fREQiyUwWM4CuZtbFzBoBw4DJsQuYWVszK4zhl8D46P0rwCAzaxU1bA+KpkkVy8mBzp2hXr3wmpOT7ohEpDpKWrJw9wJgJOEkvwCY5O7zzGy0mZ0RLXY88ImZfQrsA4yJ1l0H3E5IODOA0dE0qUI5OTBiBCxdCu7hdcQIJQwR2Z25e7pjqBJZWVmem5ub7jBqlM6dQ4IorlMnWLIk1dGISDqY2Ux3z4q3XLobuCWNvviifNNFpO5SsqjDOnYs33QRqbuULOqwMWOgadNdpzVtGqaLiMRSsqjDsrNh3LjQRmEWXseNC9NFRGI1SHcAkl7Z2UoOIhKfShYiIhKXkoWIiMSlZCEiInEpWYiISFxKFiIiEpeShYiIxKVkISIicSlZ1HDqYlxEUkE35dVghV2M5+eH8cIuxkE32olI1VLJoga76aadiaJQfn6YLiJSlZQsajB1MS4iqaJkUYOpi3ERSRUlixpMXYyLSKooWdRg6mJcRFJFV0PVcOpiXERSIaklCzMbbGafmNkiMxtVwvyOZjbNzD4ws7lmdmo0vbOZbTGz2dHwt2TGKSIiZUtaycLM6gNjgYHAcmCGmU129/kxi90MTHL3B8ysO/Ai0Dma95m7905WfCIikrhkliz6AYvcfbG7fwdMAIYUW8aB5tH7FsDKJMYjIiIVlMxk0R5YFjO+PJoW6zbgAjNbTihVXBUzr0tUPfWGmR1T0g7MbISZ5ZpZbl5eXhWGLiIisdJ9NdRw4FF37wCcCjxhZvWAVUBHd+8D/B/wpJk1L76yu49z9yx3z2rXrl1KAxcRqUuSmSxWAAfEjHeIpsX6MTAJwN3fARoDbd19q7uvjabPBD4DvpfEWEVEpAzJTBYzgK5m1sXMGgHDgMnFlvkCOAnAzA4lJIs8M2sXNZBjZgcCXYHFSYxVRETKkLSrody9wMxGAq8A9YHx7j7PzEYDue4+Gfg58JCZXUdo7L7Y3d3MjgVGm9k2YAdwhbuvS1asIiJSNnP3dMdQJbKysjw3NzfdYYiI1ChmNtPds+Itl+4GbhERqQGULEREJC4lCxERiStusjCzq8ysVSqCERGR6imRksU+hH6dJkUdA1qygxIRkeolbrJw95sJ9zn8HbgYWGhmd5rZQUmOTUREqomE2iw8XF/7ZTQUAK2Ap83sD0mMTUREqom4N+WZ2TXAhcAa4GHgBnffFvXhtBC4MbkhiohIuiVyB3dr4Cx3Xxo70d13mNlpyQlLRESqk0SqoV4CirraMLPmZtYfwN0XJCswERGpPhJJFg8Am2PGN0fTRESkjkgkWZjHdCDl7jtIYgeEdU1ODnTuDPXqhdecnHRHJCKyu0RO+ovN7Gp2liZ+iroLrxI5OTBiBOTnh/GlS8M4QHZ2+uKqCfLz4d134Y034K234JtvoEmT0oemTcueX9rQuHFI5CJ1XdxeZ81sb+B+4ERCN+JTgWvd/avkh5e4mtjrbOfOIUEU16kTLFmS6miqt82b4e23Q3J44w14/33Yti2cyHv3hjZtYMuW3Ydvvw2vW7dWfN977BE/qey7L3TtCgcfHIYDDwzTk80d1q0Lx6IwloYNoapund22DdavD/soa6hfH1q3Lnto2TIsJ9VLor3OqovyNKpXL/yzF2cGO3akPp7qZMOGUGKYPj0kh9xc2L49nGyysuC448Jw1FHQokX87e3YsTNxVNWQn7/zdcWKcNKM1aHDzuQROxx0EOy1V/yYt26FlSvDtksbVq7cPRHWq1f+Uta335acBDZtKj0+M2jVKgw7doTlN2wo+zO1bFlyIqlsYt1vP+jXDzIzYc89K7etuibRZJHIfRaNCY8/PYzwJDsA3P3SSkUodOxYcsmiY8fUx5Ju69bBm2/uLDnMnh1OQA0bhpPAL34RksORRyZ2oi2uXr1wkmzatOpjL7RuHXz2GSxatOsweTJ8Vawcvu++uyaP7dt3TwRr1uy+jyZNoH37MBxxxM73jRollti2bAkn9OLzGzfeeeJu3x569oxfUmjRYvcquoIC+Prr+CWRwuHzz2Ht2sqV/Nx3VuXWqwc9eoTvTL9+0LdvGG+gVtZKS6Qa6p/Ax8D5wGggG1jg7tckP7zE1cSSRfE2Cwgns3Hj6kabxbJlMHEiTJgAs2aFf/o99oABA3aWHAYMSO4JPlU2biw5kSxaFEoHAHvvvfPkX9rQsmXVVTHVJl99BTNmhOrJwqGwpNekSShxFCaQfv2gSxcdx0JVVg1lZh+4ex8zm+vuvcysIfCmuw+oqmCrQk1MFhASxk03wRdfhBLFmDG1O1GsXQvPPBM+9/TpYVrfvnD66SE59OsXfuXWJfn54Zdvo0bpjqT2cIfFi3dNHrNmheo2CG1cscnjkENKLy3VdlWZLN53935mNp1wJdSXwPvufmDVhFo1amqyqAu++QZeeAGefBJefjk0mh5ySEiKw4eHqhiRZNu2DT76aNcEMm/eru2Ghe0w8argig8NG1au/cu9YlfrxbZD7bFHxY5LlbVZAOOi51ncDEwG9gJ+nWAQg4H7gPrAw+7+u2LzOwKPAS2jZUa5+4vRvF8S2kq2A1e7+yuJ7FOqh23b4NVXQ4J47rmQMNq3h2uugfPPD1cwqRpAUqlhQ+jTJww/+UmYtmkTzJwZ2g5LalNZswY+/TS8//rrqo2nfv2dJ3uznYmjoKD82+rbNyS/ZCozWUSdBW509/XAdCDh0oSZ1QfGAgOB5YRnYkx29/kxi90MTHL3B8ysO/Ai0Dl6P4zQqL4/MMXMvufu28vx2STFduwIl7c++SRMmhSqnFq1CiWI88+HY46pe0V8qd6aNYPjj09s2e3bS2+8Lygof2mgYcOS91NQsPtFCfGGdu2q7JCUqsxkEXUWeCMwqQLb7gcscvfFAGY2ARgCxCYLB5pH71sAUVMfQ4AJ7r4V+NzMFkXbe6cCcUgSuYcrlyZOhKeeCm0vTZrAkCEhQZx8suripXaoXz+0dbRpk9z9NGgQklizZsndT3klUg01xcyuByYC3xROdPd1pa8CQHtgWcz4cqB/sWVuA/5rZlcBewLfj1n33WLrti++AzMbAYwA6FgXrzdNk3XrQhXTyy/DK6/AqlXhC37yyXDnnSFRVOTyVhGpvhJJFkOj15/FTHPKUSVVhuHAo+5+t5kdATxhZj0SXdndxwHjIDRwV0E8UoLt28NNcS+/HIb33w9VTq1awaBBIUmcfjq0bZvuSEUkWeImC3fvUsFtrwAOiBnvEE2L9WNgcLSfd6IbANsmuK4k0ZdfhlLDyy/Df/8bShNm4TLDX/8aBg8OjWrqvkGkbkjkDu4LS5ru7o/HWXUG0NXMuhBO9MMIN/bF+gI4CXjUzA4l3CGeR7jq6kkzu4fQwN0VSHJbf922bVtonC4sPcyeHabvs08oNQweDAMHJr++VkSqp0SqofrGvG9MOLnPAspMFu5eYGYjgVcIl8WOd/d5ZjYayHX3ycDPgYfM7DpC1dbFUXfo88xsEqExvAD4ma6ESo4PPww3Ar74YriMsEGD0N/Sb38bEkSvXrqCSUQq0JGgmbUkXKk0ODkhVYxuyiufxYvhllvCZa7Nm8PQoXDKKXDiiWFcROqGqrwpr7hvgIq2Y0iarVoFd9wR+p9q2BBuvDEMrVunOzIRqc4SabN4gVBFBOHJet2p2H0Xkkbr18Mf/gD33RfaJy6/HG6+GfbfP92RiUhNkEjJ4q6Y9wXAUndfnqR4pIp98w3cf39IFBs2hL6YRo8O3WKLiCQqkWTxBbDK3b8FMLMmZtbZ3ZckNTKplO++g4cegttvh9WrwxVNd9wRGqxFRMorketc/gnEPrdtezRNqqHt2+GJJ6BbNxg5MvTu+r//hQfwKFGISEUlkiwauPt3hSPRe/X2U824w/PPQ0YGXHhheEjOSy/B66+Hp8uJiFRGIskiz8zOKBwxsyFACQ98lHQpTAg//GFovJ44MXTPMXiwugEXkaqRSJvFFUCOmf0lGl8OlHhXt6TWsmXwf/8HTz8NHTqENoqLL9bzhkWk6iXSN9RnwAAz2ysa35z0qKRMW7fCPfeEBusdO8LVTddfH7oGFxFJhrjVUGZ2p5m1dPfN7r7ZzFqZ2R2pCE5298or0LMn/OpXocfXBQtCx35KFCKSTIm0WZzi7kUPFIyemndq8kKSkixdCmedFdohIDReP/ssdO6c1rBEpI5IJFnUN7OiR4GbWROggo8Gl/L69ttQ3XTooaFUceedofO/wdWqZy4Rqe0SaQrNAaaa2SOAARcDjyUzKAlefBGuvho++wzOOQfuvhv0QEARSYdEGrh/b2ZzCI88dUKX452SHVhd9vnncO214Ua6Qw4JDx8aODDdUYlIXZbokwpWExLFucCJwIKkRVSHbdkCv/kNdO8OU6fC738Pc+cqUYhI+pVasjCz7xGekT2ccBPeRMLzL05IUWx1ygsvwDXXhFLF0KFw113h3gkRkeqgrGqoj4E3gdPcfRFA9EQ7qUJbt8KwYfDcc6ERe+rU8AAiEZHqpKxqqLOAVcA0M3vIzE4iNHBLFbruupAofvtbmDNHiUJEqqdSk4W7P+fuw4BuwDTgWmBvM3vAzAalKsDaLCcHHngAbrgBRo0KT64TEamO4jZwu/s37v6ku58OdAA+AH6RyMbNbLCZfWJmi8xsVAnz/2Rms6PhUzP7Ombe9ph5k8vxmWqEefNgxAg49thw74SISHVWri7noru3x0VDmcysPjAWGEjofHCGmU129/kx27suZvmrgD4xm9ji7r3LE19NsWkTnH02NGsGEyao4z8Rqf4SvXS2IvoBi9x9cfQMjAnAkDKWHw48lcR4qgV3uOwyWLgwdCW+337pjkhEJL5kJov2wLKY8eXRtN2YWSegC/BazOTGZpZrZu+a2Q9LWW9EtExuXl5eVcWdVH/+M0yaFKqejjsu3dGIiCQmmcmiPIYBT7v79phpndw9CzgfuNfMDiq+kruPc/csd89q165dqmKtsHfegZ//HM44A268Md3RiIgkLpnJYgVwQMx4h2haSYZRrArK3VdEr4uB19m1PaPGycuD884LfTs99pieYCciNUsyk8UMoKuZdTGzRoSEsNtVTWbWDWgFvBMzrVVhT7dm1hY4CphffN2aYvt2OP/8kDCefjo8H1tEpCZJ2nU47l5gZiMJHQ/WB8a7+zwzGw3kunth4hgGTHB3j1n9UOBBM9tBSGi/i72KqqYZPRqmTIGHH4Y+Nbp8JCJ1le16jq65srKyPDc3N91h7Obll+HUU+Gii2D8eFU/iUj1YmYzo/bhMlWXBu5aaelSyM4Oj0EdO1aJQkRqLiWLSsrJCY82rVcvvObkhOlbt8K550JBATzzDDRtms4oRUQqR/cOV0JOTuiyIz8/jC9dGsYhXCY7Ywb8619w8MHpi1FEpCooWVTCTTftTBSF8vPDU+7WrIHrr4czz0xPbCIiVUnVUJXwxRclT1+zBo45Rh0EikjtoWRRCR07ljy9Xr3QQaC6HBeR2kLJohLGjCm54XrUKNh//9THIyKSLEoWlZCdDePGQadOO6edd15IIiIitYkauCspOxsOOig8xGjwYHiq1neyLiJ1kUoWlbRlSyhNdOgQOgispyMqIrWQShaV9NRTsGwZTJ0KrVqlOxoRkeTQ7+BKcA/dePToASeckO5oRESSRyWLSnjvPZg1Cx54QP0+iUjtppJFJYwdC82bwwUXpDsSEZHkUrKooK++Cs/Svugi2GuvdEcjIpJcShYV9PDD8N138NOfpjsSEZHkU7KogIIC+Nvf4KSToFu3dEcjIpJ8ShYV8O9/h8tlf/azdEciIpIaShYVMHYsHHAAnH56uiMREUmNpCYLMxtsZp+Y2SIzG1XC/D+Z2exo+NTMvo6Zd5GZLYyGi5IZZ3l88glMmQI/+Qk00IXHIlJHJO10Z2b1gbHAQGA5MMPMJrv7/MJl3P26mOWvAvpE71sDtwJZgAMzo3XXJyveRP31r6Hr8csuS3ckIiKpk8ySRT9gkbsvdvfvgAnAkDKWHw4UdsN3MvCqu6+LEsSrwOAkxpqQzZvh0UdDX1D77JPuaEREUieZyaI9sCxmfHk0bTdm1gnoArxWnnXNbISZ5ZpZbl5eXpUEXZacHNi4UQ3bIlL3VJcG7mHA0+6+vTwrufs4d89y96x27dolKbTCfYWG7T59YMCApO5KRKTaSWayWAEcEDPeIZpWkmHsrIIq77op8dZb8OGHoVShfqBEpK5JZrKYAXQ1sy5m1oiQECYXX8jMugGtgHdiJr8CDDKzVmbWChgUTUubsWNDF+TDh6czChGR9EhasnD3AmAk4SS/AJjk7vPMbLSZnRGz6DBggrt7zLrrgNsJCWcGMDqalharVsEzz8All5T8zG0RkdouqXcKuPuLwIvFpt1SbPy2UtYdD4xPWnDlMG5c6OLjyivTHYmISHpUlwbuamvbNnjwwfB87YMPTnc0IiLpoXuQ43juuVANNW5cuiMREUkflSziGDsWOneGU05JdyQiIumjZFGGjz6CN94IbRX166c7GhGR9FGyKMNf/wp77AGXXpruSERE0kvJohQbN8ITT8CwYdC2bbqjERFJLyWLUjz+eOg4UP1AiYgoWZTIPVRB9esHffumOxoRkfTTpbMlmDYNFiyAxx5LdyQiItWDShYlGDs2tFOcd166IxERqR5Usihm2bJwI94NN0DjxumORqRitm3bxvLly/n222/THYpUE40bN6ZDhw40bNiwQusrWRTz4IOhzeIC7GR8AAAU+klEQVSKK9IdiUjFLV++nGbNmtG5c2dMferXee7O2rVrWb58OV26dKnQNlQNFWPrVnjoITjttHDXtkhN9e2339KmTRslCgHAzGjTpk2lSppKFjGeeQa++kqXy0rtoEQhsSr7fVCyiDF2bOhZduDAdEciIlK9KFlEZs+Gt9+Gn/4U6umoSB2TkxOqXuvVC685OZXb3tq1a+nduze9e/dm3333pX379kXj3333XULbuOSSS/jkk0/KXGbs2LHkVDZYSYgauCNjx0KTJnDxxemORCS1cnJgxAjIzw/jS5eGcYDs7Ipts02bNsyePRuA2267jb322ovrr79+l2XcHXenXim/zh555JG4+/lZDawzLigooEGDmnfq1W9oYP368A+TnR2esy1Sl9x0085EUSg/P0yvaosWLaJ79+5kZ2dz2GGHsWrVKkaMGEFWVhaHHXYYo0ePLlr26KOPZvbs2RQUFNCyZUtGjRpFRkYGRxxxBF999RUAN998M/fee2/R8qNGjaJfv34ccsghvP322wB88803nH322XTv3p1zzjmHrKysokQW69Zbb6Vv37706NGDK664gsInPX/66aeceOKJZGRkkJmZyZIlSwC488476dmzJxkZGdwUHazCmAG+/PJLDo6emPbwww/zwx/+kBNOOIGTTz6ZjRs3cuKJJ5KZmUmvXr3497//XRTHI488Qq9evcjIyOCSSy5hw4YNHHjggRQUFACwfv36XcZTRckCePRR2LJFDdtSN33xRfmmV9bHH3/Mddddx/z582nfvj2/+93vyM3NZc6cObz66qvMnz9/t3U2bNjAcccdx5w5czjiiCMYP77kJy67O++//z5//OMfixLPn//8Z/bdd1/mz5/Pr3/9az744IMS173mmmuYMWMGH374IRs2bODll18GYPjw4Vx33XXMmTOHt99+m7333psXXniBl156iffff585c+bw85//PO7n/uCDD/jXv/7F1KlTadKkCc899xyzZs1iypQpXHfddQDMmTOH3//+97z++uvMmTOHu+++mxYtWnDUUUcVxfPUU09x7rnnprx0ktRkYWaDzewTM1tkZqNKWeY8M5tvZvPM7MmY6dvNbHY0TE5WjDt2hH6gjjwSevdO1l5Eqq+OHcs3vbIOOuggsrKyisafeuopMjMzyczMZMGCBSUmiyZNmnBK9ASyww8/vOjXfXFnnXXWbsu89dZbDBs2DICMjAwOO+ywEtedOnUq/fr1IyMjgzfeeIN58+axfv161qxZw+mnnw6EG9uaNm3KlClTuPTSS2nSpAkArVu3jvu5Bw0aRKuo6sLdGTVqFL169WLQoEEsW7aMNWvW8NprrzF06NCi7RW+XnbZZUXVco888giXXHJJ3P1VtaSlJjOrD4wFBgLLgRlmNtnd58cs0xX4JXCUu683s71jNrHF3ZN++l6yBDZtgpjSr0idMmbMrm0WAE2bhunJsOeeexa9X7hwIffddx/vv/8+LVu25IILLijxXoBGjRoVva9fv36pVTB77LFH3GVKkp+fz8iRI5k1axbt27fn5ptvrtA9CQ0aNGDHjh0Au60f+7kff/xxNmzYwKxZs2jQoAEdOnQoc3/HHXccI0eOZNq0aTRs2JBu3bqVO7bKSmbJoh+wyN0Xu/t3wARgSLFlLgfGuvt6AHf/KonxlOjAA0Nx+5xzUr1nkeohOzs8Y75TJzALr+PGVbxxuzw2btxIs2bNaN68OatWreKVV16p8n0cddRRTJo0CYAPP/ywxJLLli1bqFevHm3btmXTpk0888wzALRq1Yp27drxwgsvACEB5OfnM3DgQMaPH8+WLVsAWLduHQCdO3dm5syZADz99NOlxrRhwwb23ntvGjRowKuvvsqKFSsAOPHEE5k4cWLR9gpfAS644AKys7PTUqqA5CaL9sCymPHl0bRY3wO+Z2b/M7N3zWxwzLzGZpYbTf9hSTswsxHRMrl5eXkVDrRRI6hgdykitUJ2dihl79gRXlORKAAyMzPp3r073bp148ILL+Soo46q8n1cddVVrFixgu7du/Ob3/yG7t2706JFi12WadOmDRdddBHdu3fnlFNOoX///kXzcnJyuPvuu+nVqxdHH300eXl5nHbaaQwePJisrCx69+7Nn/70JwBuuOEG7rvvPjIzM1m/fn2pMf3oRz/i7bffpmfPnkyYMIGuXbsCoZrsxhtv5Nhjj6V3797ccMMNRetkZ2ezYcMGhg4dWpWHJ2FW2OJf5Rs2OwcY7O6XReM/Avq7+8iYZf4NbAPOAzoA04Ge7v61mbV39xVmdiDwGnCSu39W2v6ysrI8Nzc3KZ9FpKZZsGABhx56aLrDqBYKCgooKCigcePGLFy4kEGDBrFw4cIad/nqhAkTeOWVVxK6pLg0JX0vzGymu2eVskqRZB6tFcABMeMdommxlgPvufs24HMz+xToCsxw9xUA7r7YzF4H+gClJgsRkZJs3ryZk046iYKCAtydBx98sMYliiuvvJIpU6YUXRGVDsk8YjOArmbWhZAkhgHnF1vmOWA48IiZtSVUSy02s1ZAvrtvjaYfBfwhibGKSC3VsmXLonaEmuqBBx5IdwjJSxbuXmBmI4FXgPrAeHefZ2ajgVx3nxzNG2Rm84HtwA3uvtbMjgQeNLMdhHaV38VeRSUiIqmV1LKYu78IvFhs2i0x7x34v2iIXeZtoGcyYxMRkcTpDm4REYlLyUJEROJSshCRKnfCCSfsdoPdvffey5VXXlnmenvttRcAK1eu5JxS7pQ9/vjjiXeZ/L333kt+zC3pp556Kl9//XUioUsplCxEpMoNHz6cCRMm7DJtwoQJDB8+PKH1999//zLvgI6neLJ48cUXadmyZYW3l2ruXtRtSHWhZCFSy117LRx/fNUO115b9j7POecc/vOf/xQ96GjJkiWsXLmSY445pui+h8zMTHr27Mnzzz+/2/pLliyhR48eQOiKY9iwYRx66KGceeaZRV1sQLj/oLB781tvvRWA+++/n5UrV3LCCSdwwgknAKEbjjVr1gBwzz330KNHD3r06FHUvfmSJUs49NBDufzyyznssMMYNGjQLvsp9MILL9C/f3/69OnD97//fVavXg2EezkuueQSevbsSa9evYq6C3n55ZfJzMwkIyODk046CQjP97jrrruKttmjRw+WLFnCkiVLOOSQQ7jwwgvp0aMHy5YtK/HzAcyYMYMjjzySjIwM+vXrx6ZNmzj22GN36Xr96KOPZs6cOWX/ocqhZt2ZIiI1QuvWrenXrx8vvfQSQ4YMYcKECZx33nmYGY0bN+bZZ5+lefPmrFmzhgEDBnDGGWeU+ozoBx54gKZNm7JgwQLmzp1LZmZm0bwxY8bQunVrtm/fzkknncTcuXO5+uqrueeee5g2bRpt27bdZVszZ87kkUce4b333sPd6d+/P8cddxytWrVi4cKFPPXUUzz00EOcd955PPPMM1xwwQW7rH/00Ufz7rvvYmY8/PDD/OEPf+Duu+/m9ttvp0WLFnz44YdAeOZEXl4el19+OdOnT6dLly679PNUmoULF/LYY48xYMCAUj9ft27dGDp0KBMnTqRv375s3LiRJk2a8OMf/5hHH32Ue++9l08//ZRvv/2WjIyMcv3dyqJkIVLLRT+eU66wKqowWfz9738HQhXLr371K6ZPn069evVYsWIFq1evZt999y1xO9OnT+fqq68GoFevXvTq1ato3qRJkxg3bhwFBQWsWrWK+fPn7zK/uLfeeoszzzyzqAfYs846izfffJMzzjiDLl260Dt6TkFp3aAvX76coUOHsmrVKr777ju6dOkCwJQpU3apdmvVqhUvvPACxx57bNEyiXRj3qlTp6JEUdrnMzP2228/+vbtC0Dz5s0BOPfcc7n99tv54x//yPjx47m4ih/7Weeroar62cMiEgwZMoSpU6cya9Ys8vPzOfzww4HQMV9eXh4zZ85k9uzZ7LPPPhXqDvzzzz/nrrvuYurUqcydO5cf/OAHFdpOocLuzaH0Ls6vuuoqRo4cyYcffsiDDz5Y6W7MYdeuzGO7MS/v52vatCkDBw7k+eefZ9KkSWRXcW+QdTpZFD57eOlScN/57GElDJHK22uvvTjhhBO49NJLd2nYLuyeu2HDhkybNo2lS5eWuZ1jjz2WJ58Mz0X76KOPmDt3LhC6N99zzz1p0aIFq1ev5qWXXipap1mzZmzatGm3bR1zzDE899xz5Ofn88033/Dss89yzDHHJPyZNmzYQPv2ofPsxx57rGj6wIEDGTt2bNH4+vXrGTBgANOnT+fzzz8Hdu3GfNasWQDMmjWraH5xpX2+Qw45hFWrVjFjxgwANm3aVJTYLrvsMq6++mr69u1b9KClqlKnk0Uqnz0sUhcNHz6cOXPm7JIssrOzyc3NpWfPnjz++ONxH+Rz5ZVXsnnzZg499FBuueWWohJKRkYGffr0oVu3bpx//vm7dG8+YsQIBg8eXNTAXSgzM5OLL76Yfv360b9/fy677DL69OmT8Oe57bbbOPfcczn88MN3aQ+5+eabWb9+PT169CAjI4Np06bRrl07xo0bx1lnnUVGRkZR1+Jnn30269at47DDDuMvf/kL3/ve90rcV2mfr1GjRkycOJGrrrqKjIwMBg4cWFTiOPzww2nevHlSnnmRtC7KU60iXZTXqxdKFMWZhX79RWoqdVFeN61cuZLjjz+ejz/+mHr1di8LVKaL8jpdskj1s4dFRJLl8ccfp3///owZM6bERFFZdTpZjBkTnjUcK5nPHhYRSZYLL7yQZcuWce655yZl+3U6WaTz2cMiyVZbqpilalT2+1Dn77PIzlZykNqncePGrF27ljZt2pR6s5vUHe7O2rVrady4cYW3UeeThUht1KFDB5YvX05eXl66Q5FqonHjxnTo0KHC6ytZiNRCDRs2LLpzWKQq1Ok2CxERSYyShYiIxKVkISIicdWaO7jNLA8ou5OZ9GoLrEl3EGVQfJWj+CpH8VVOZeLr5O7t4i1Ua5JFdWdmuYncUp8uiq9yFF/lKL7KSUV8qoYSEZG4lCxERCQuJYvUGZfuAOJQfJWj+CpH8VVO0uNTm4WIiMSlkoWIiMSlZCEiInEpWVQRMzvAzKaZ2Xwzm2dm15SwzPFmtsHMZkfDLWmIc4mZfRjtf7dHC1pwv5ktMrO5ZpaZwtgOiTk2s81so5ldW2yZlB5DMxtvZl+Z2Ucx01qb2atmtjB6LfFhx2Z2UbTMQjO7KIXx/dHMPo7+fs+aWctS1i3zu5DE+G4zsxUxf8NTS1l3sJl9En0XR6UwvokxsS0xs9mlrJuK41fieSUt30F311AFA7AfkBm9bwZ8CnQvtszxwL/THOcSoG0Z808FXgIMGAC8l6Y46wNfEm4YStsxBI4FMoGPYqb9ARgVvR8F/L6E9VoDi6PXVtH7VimKbxDQIHr/+5LiS+S7kMT4bgOuT+Dv/xlwINAImFP8/ylZ8RWbfzdwSxqPX4nnlXR8B1WyqCLuvsrdZ0XvNwELgPbpjapChgCPe/Au0NLM9ktDHCcBn7l7Wu/Kd/fpwLpik4cAj0XvHwN+WMKqJwOvuvs6d18PvAoMTkV87v5fdy+IRt8FKt4vdSWVcvwS0Q9Y5O6L3f07YALhuFepsuKz8CCQ84Cnqnq/iSrjvJLy76CSRRKYWWegD/BeCbOPMLM5ZvaSmR2W0sACB/5rZjPNbEQJ89sDy2LGl5OepDeM0v9J030M93H3VdH7L4F9SlimuhzHSwklxZLE+y4k08iommx8KVUo1eH4HQOsdveFpcxP6fErdl5J+XdQyaKKmdlewDPAte6+sdjsWYRqlQzgz8BzqY4PONrdM4FTgJ+Z2bFpiKFMZtYIOAP4Zwmzq8MxLOKhvF8trz83s5uAAiCnlEXS9V14ADgI6A2sIlT1VEfDKbtUkbLjV9Z5JVXfQSWLKmRmDQl/0Bx3/1fx+e6+0d03R+9fBBqaWdtUxujuK6LXr4BnCcX9WCuAA2LGO0TTUukUYJa7ry4+ozocQ2B1YdVc9PpVCcuk9Tia2cXAaUB2dDLZTQLfhaRw99Xuvt3ddwAPlbLfdB+/BsBZwMTSlknV8SvlvJLy76CSRRWJ6jf/Dixw93tKWWbfaDnMrB/h+K9NYYx7mlmzwveEhtCPii02GbgwuipqALAhpribKqX+okv3MYxMBgqvLLkIeL6EZV4BBplZq6iaZVA0LenMbDBwI3CGu+eXskwi34VkxRfbBnZmKfudAXQ1sy5RSXMY4binyveBj919eUkzU3X8yjivpP47mMyW/Lo0AEcTioJzgdnRcCpwBXBFtMxIYB7hyo53gSNTHOOB0b7nRHHcFE2PjdGAsYQrUT4EslIc456Ek3+LmGlpO4aEpLUK2Eao8/0x0AaYCiwEpgCto2WzgIdj1r0UWBQNl6QwvkWEuurC7+HfomX3B14s67uQovieiL5bcwknvf2KxxeNn0q4+uezVMYXTX+08DsXs2w6jl9p55WUfwfV3YeIiMSlaigREYlLyUJEROJSshARkbiULEREJC4lCxERiUvJQiQOM9tuu/aGW2U9oJpZ59geT0WqqwbpDkCkBtji7r3THYRIOqlkIVJB0fMM/hA90+B9Mzs4mt7ZzF6LOsqbamYdo+n7WHi+xJxoODLaVH0zeyh6XsF/zaxJtPzV0XMM5prZhDR9TBFAyUIkEU2KVUMNjZm3wd17An8B7o2m/Rl4zN17ETrxuz+afj/whodOEDMJd/4CdAXGuvthwNfA2dH0UUCfaDtXJOvDiSRCd3CLxGFmm919rxKmLwFOdPfFUWdvX7p7GzNbQ+jCYls0fZW7tzWzPKCDu2+N2UZnwjMHukbjvwAauvsdZvYysJnQs+5zHnWgKJIOKlmIVI6X8r48tsa8387OtsQfEPrpygRmRD2hiqSFkoVI5QyNeX0nev82oZdUgGzgzej9VOBKADOrb2YtStuomdUDDnD3acAvgBbAbqUbkVTRLxWR+JqY2eyY8ZfdvfDy2VZmNpdQOhgeTbsKeMTMbgDygEui6dcA48zsx4QSxJWEHk9LUh/4R5RQDLjf3b+usk8kUk5qsxCpoKjNIsvd16Q7FpFkUzWUiIjEpZKFiIjEpZKFiIjEpWQhIiJxKVmIiEhcShYiIhKXkoWIiMT1/5kWopkeFg5bAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.clf()   # Clear plot\n",
    "\n",
    "plt.plot(epochs, tr_accuracy_list, 'bo', label='Training accuracy')\n",
    "plt.plot(epochs, val_accuracy_list, 'b', label='Validation accuracy')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0UdrazksUI0_"
   },
   "source": [
    "### Step 10) Testing your model\n",
    "Now that we have successfully trained our model and our training accuracy has jumped over 90%, we need to test it. The test accuracy is a better evaluation metric for how our model will perform in the real world."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "ta7jDHRwUGIa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2246/2246==============================] - 0s 63us/sample - loss: 1.2530 - acc: 0.7765\n",
      "Test accuracy: 0.78\n"
     ]
    }
   ],
   "source": [
    "loss, accuracy = model.evaluate(test_data, test_labels)\n",
    "print('Test accuracy: %.2f' % (accuracy))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "default_view": {},
   "name": "4-reuters.ipynb",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}