From 1f0e432733b5c89f5f1f5b2ca4325811f56ac962 Mon Sep 17 00:00:00 2001 From: KristineYW <63246056+KristineYW@users.noreply.github.com> Date: Mon, 13 Jul 2020 19:11:49 -0400 Subject: [PATCH 1/3] Created using Colaboratory --- ...ang_LS_DS_431_Intro_to_NN_Assignment.ipynb | 1097 +++++++++++++++++ 1 file changed, 1097 insertions(+) create mode 100644 Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb diff --git a/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb b/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb new file mode 100644 index 000000000..6cf455a64 --- /dev/null +++ b/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb @@ -0,0 +1,1097 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb", + "provenance": [], + "collapsed_sections": [], + "include_colab_link": true + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "view-in-github", + "colab_type": "text" + }, + "source": [ + "\"Open" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "dVfaLrjLvxvQ" + }, + "source": [ + "\n", + "

\n", + "

\n", + "\n", + "# Neural Networks\n", + "\n", + "## *Data Science Unit 4 Sprint 2 Assignment 1*" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "wxtoY12mwmih" + }, + "source": [ + "## Define the Following:\n", + "You can add image, diagrams, whatever you need to ensure that you understand the concepts below.\n", + "\n", + "### Input Layer: \n", + "- The initial data for the neural network. This is usually how many columns/features there are. \n", + "### Hidden Layer: \n", + "-The intermediate layer/layers between the inputs and the outputs. This is where all the computation happens, and where we are trying to determine the weights. The number of hidden layers is a hyperparameter and can be changed according to the complexity of the model. \n", + "### Output Layer:\n", + "-Our expected results (targets) from the inputs. The shape is a vector of values. The model is trained based on reducing errors to reach the output layer. \n", + "### Neuron:\n", + "- The biological basis for neural networks. \n", + "### Weight:\n", + "- The number(s) attempted to be calculated with the activation functions to reach the output layers with the given inputs. There are as many weights as there are inputs. \n", + "### Activation Function:\n", + "- The activation function determines if a given node should be “activated” or not based on the weighted sum. The specific function to use is based on the requirements of the model (i.e. linear, binary, or multiclass). \n", + "### Node Map: \n", + "- Node maps are the diagrams of how to get from inputs to outputs. With a node map, we can visualize the relationship between the input layer, the hidden layer(s), and the output layer. \n", + "### Perceptron:\n", + "- Perceptrons are the simplest kind of neural networks, with just a single node created from any number of inputs to get to a single output. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "NXuy9WcWzxa4" + }, + "source": [ + "## Inputs -> Outputs\n", + "\n", + "### Explain the flow of information through a neural network from inputs to outputs. Be sure to include: inputs, weights, bias, and activation functions. How does it all flow from beginning to end?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "PlSwIJMC0A8F" + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "6sWR43PTwhSk" + }, + "source": [ + "## Write your own perceptron code that can correctly classify (99.0% accuracy) a NAND gate. \n", + "\n", + "| x1 | x2 | y |\n", + "|----|----|---|\n", + "| 0 | 0 | 1 |\n", + "| 1 | 0 | 1 |\n", + "| 0 | 1 | 1 |\n", + "| 1 | 1 | 0 |" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "DOoDKYtX65d0", + "colab_type": "code", + "colab": {} + }, + "source": [ + "import pandas as pd\n", + "data = { 'x1': [0,1,0,1],\n", + " 'x2': [0,0,1,1],\n", + " 'y': [1,1,1,0]\n", + " }\n", + "\n", + "df = pd.DataFrame.from_dict(data).astype('int')" + ], + "execution_count": 66, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "8S73w4yInNt7", + "colab_type": "code", + "colab": {} + }, + "source": [ + "import numpy as np\n", + "\n", + "np.random.seed(812)\n", + "\n", + "inputs = np.array([[0,0, 1],\n", + " [1,0, 1],\n", + " [0,1, 1],\n", + " [1,1, 1]\n", + " ])\n", + "\n", + "correct_outputs = [[1], [1], [1], [0]]" + ], + "execution_count": 153, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "Qh5K6njKnzQA", + "colab_type": "code", + "colab": {} + }, + "source": [ + "def sigmoid(x):\n", + " return 1/(1 + np.exp(-x))\n", + "\n", + "def sigmoid_derivative(x):\n", + " sx = sigmoid(x)\n", + " return sx * (1 - sx)" + ], + "execution_count": 154, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "uXFt8Sp3oLMd", + "colab_type": "code", + "colab": {} + }, + "source": [ + "weights = np.random.random((3,1))" + ], + "execution_count": 155, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "3d0oQOe0qO3s", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 68 + }, + "outputId": "aee54805-0b5b-42b0-a547-874f8b1b1245" + }, + "source": [ + "weights" + ], + "execution_count": 156, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0.5049808 ],\n", + " [0.60592761],\n", + " [0.45748719]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 156 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "iGAlq0ztt9ZO", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + }, + "outputId": "c180ae88-020c-4aa8-a229-6efbda112efc" + }, + "source": [ + "print(inputs.shape)\n", + "print(weights.shape)" + ], + "execution_count": 157, + "outputs": [ + { + "output_type": "stream", + "text": [ + "(4, 3)\n", + "(3, 1)\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "7QdOqo1HpqGC", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "fa1f8fb5-7d3b-420d-f720-5809f2b13f76" + }, + "source": [ + "weighted_sum = np.dot(inputs, weights)\n", + "\n", + "weighted_sum" + ], + "execution_count": 158, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0.45748719],\n", + " [0.96246799],\n", + " [1.06341479],\n", + " [1.5683956 ]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 158 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "oy85-EMTpBVe", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "9184e776-6d24-4761-c385-087a0f519fe8" + }, + "source": [ + "activated_output = sigmoid(weighted_sum)\n", + "\n", + "activated_output" + ], + "execution_count": 159, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0.6124179 ],\n", + " [0.72361567],\n", + " [0.74334258],\n", + " [0.82755477]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 159 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "d92nadpasMnf", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "0fe5f53a-be4e-4e86-d274-4eef27409d23" + }, + "source": [ + "error = correct_outputs - activated_output\n", + "\n", + "error" + ], + "execution_count": 160, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[ 0.3875821 ],\n", + " [ 0.27638433],\n", + " [ 0.25665742],\n", + " [-0.82755477]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 160 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "WwUT3CHXuR3l", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "988515af-8f6c-4814-edc8-e20f4dbf5dbb" + }, + "source": [ + "adjustments = error * sigmoid_derivative(weighted_sum)\n", + "\n", + "adjustments" + ], + "execution_count": 161, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[ 0.09199735],\n", + " [ 0.05527577],\n", + " [ 0.04896623],\n", + " [-0.11809858]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 161 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "R78JT5t4uefH", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 68 + }, + "outputId": "8a3323ad-6729-439b-985c-993a16d64767" + }, + "source": [ + "weights += np.dot(inputs.T, adjustments)\n", + "\n", + "weights" + ], + "execution_count": 162, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0.44215799],\n", + " [0.53679525],\n", + " [0.53562795]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 162 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "Sgh7VFGwnXGH", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 170 + }, + "outputId": "f5cb8b9b-0640-4ce5-fa0f-277643ab0dc2" + }, + "source": [ + "for iteration in range(100000):\n", + " \n", + " # Weighted sum of inputs / weights\n", + " weighted_sum = np.dot(inputs, weights)\n", + " \n", + " # Activate!\n", + " activated_output = sigmoid(weighted_sum)\n", + " \n", + " # Cac error\n", + " error = correct_outputs - activated_output\n", + " \n", + " adjustments = error * sigmoid_derivative(weighted_sum)\n", + " \n", + " # Update the Weights\n", + " weights += np.dot(inputs.T, adjustments)\n", + " \n", + "print(\"Weights after training\")\n", + "print(weights)\n", + "\n", + "print(\"Output after training\")\n", + "print(activated_output)\n" + ], + "execution_count": 165, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Weights after training\n", + "[[-10.50132913]\n", + " [-10.50132913]\n", + " [ 15.8367857 ]]\n", + "Output after training\n", + "[[0.99999987]\n", + " [0.99520537]\n", + " [0.99520537]\n", + " [0.00567571]]\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Xf7sdqVs0s4x" + }, + "source": [ + "## Implement your own Perceptron Class and use it to classify a binary dataset: \n", + "- [The Pima Indians Diabetes dataset](https://raw.githubusercontent.com/ryanleeallred/datasets/master/diabetes.csv) \n", + "\n", + "You may need to search for other's implementations in order to get inspiration for your own. There are *lots* of perceptron implementations on the internet with varying levels of sophistication and complexity. Whatever your approach, make sure you understand **every** line of your implementation and what its purpose is." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "diSqydmN65d9", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 + }, + "outputId": "d276606d-b252-451f-8376-1127d2fffbbc" + }, + "source": [ + "diabetes = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/diabetes.csv')\n", + "diabetes.head()" + ], + "execution_count": 214, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
061487235033.60.627501
11856629026.60.351310
28183640023.30.672321
318966239428.10.167210
40137403516843.12.288331
\n", + "
" + ], + "text/plain": [ + " Pregnancies Glucose BloodPressure ... DiabetesPedigreeFunction Age Outcome\n", + "0 6 148 72 ... 0.627 50 1\n", + "1 1 85 66 ... 0.351 31 0\n", + "2 8 183 64 ... 0.672 32 1\n", + "3 1 89 66 ... 0.167 21 0\n", + "4 0 137 40 ... 2.288 33 1\n", + "\n", + "[5 rows x 9 columns]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 214 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bFW8rAVL65eA", + "colab_type": "text" + }, + "source": [ + "Although neural networks can handle non-normalized data, scaling or normalizing your data will improve your neural network's learning speed. Try to apply the sklearn `MinMaxScaler` or `Normalizer` to your diabetes dataset. " + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "MhRwOZDW65eB", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 + }, + "outputId": "944a651e-5517-4b69-a8bf-46189d91c75e" + }, + "source": [ + "from sklearn.preprocessing import MinMaxScaler, Normalizer\n", + "\n", + "feats = list(diabetes)[:-1]\n", + "\n", + "X = diabetes.drop(['Outcome'], axis=1)\n", + "\n", + "target = diabetes['Outcome']\n", + "\n", + "X.head()" + ], + "execution_count": 215, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAge
061487235033.60.62750
11856629026.60.35131
28183640023.30.67232
318966239428.10.16721
40137403516843.12.28833
\n", + "
" + ], + "text/plain": [ + " Pregnancies Glucose BloodPressure ... BMI DiabetesPedigreeFunction Age\n", + "0 6 148 72 ... 33.6 0.627 50\n", + "1 1 85 66 ... 26.6 0.351 31\n", + "2 8 183 64 ... 23.3 0.672 32\n", + "3 1 89 66 ... 28.1 0.167 21\n", + "4 0 137 40 ... 43.1 2.288 33\n", + "\n", + "[5 rows x 8 columns]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 215 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "KQ1GeuduOLLp", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "outputId": "9fc7fc4f-2ae2-4418-d3f9-9dc58e20d146" + }, + "source": [ + "X.shape" + ], + "execution_count": 210, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(768, 8)" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 210 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "h4r-821h0W1D", + "colab_type": "code", + "colab": {} + }, + "source": [ + "min_max_scaler = MinMaxScaler()\n", + "x_scaled = min_max_scaler.fit_transform(X)\n", + "X = pd.DataFrame(x_scaled, columns=[feats])" + ], + "execution_count": 224, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "hcqLw4FN0sl1", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 + }, + "outputId": "f7b245da-f801-42df-dccb-e261c85a6953" + }, + "source": [ + "X.head()" + ], + "execution_count": 217, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAge
00.3529410.7437190.5901640.3535350.0000000.5007450.2344150.483333
10.0588240.4271360.5409840.2929290.0000000.3964230.1165670.166667
20.4705880.9195980.5245900.0000000.0000000.3472430.2536290.183333
30.0588240.4472360.5409840.2323230.1111110.4187780.0380020.000000
40.0000000.6884420.3278690.3535350.1985820.6423250.9436380.200000
\n", + "
" + ], + "text/plain": [ + " Pregnancies Glucose ... DiabetesPedigreeFunction Age\n", + "0 0.352941 0.743719 ... 0.234415 0.483333\n", + "1 0.058824 0.427136 ... 0.116567 0.166667\n", + "2 0.470588 0.919598 ... 0.253629 0.183333\n", + "3 0.058824 0.447236 ... 0.038002 0.000000\n", + "4 0.000000 0.688442 ... 0.943638 0.200000\n", + "\n", + "[5 rows x 8 columns]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 217 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "colab_type": "code", + "id": "-W0tiX1F1hh2", + "colab": {} + }, + "source": [ + "# Introduce learning rate somewhere of 0.1 or 0.01\n", + "\n", + "class Perceptron(object):\n", + " \n", + " def __init__(self, rate = 0.01, niter = 10):\n", + " self.rate = rate\n", + " self.niter = niter\n", + " \n", + " def __sigmoid(self, x):\n", + " return 1/(1 + np.exp(-x))\n", + " \n", + " def __sigmoid_derivative(self, x):\n", + " sx = sigmoid(x)\n", + " return sx * (1 - sx)\n", + "\n", + " weights = 2 * np.random.random((8,1)) - 1\n", + "\n", + " def fit(self, X, y):\n", + "\n", + " inputs = X.values\n", + " correct_outputs = [[i] for i in target]\n", + "\n", + " \"\"\"Fit training data\n", + " X : Training vectors, X.shape : [#samples, #features]\n", + " y : Target values, y.shape : [#samples]\n", + " \"\"\"\n", + "\n", + " weights = 2 * np.random.random((8,1)) - 1\n", + "\n", + " for i in range(self.niter):\n", + " # Weighted sum of inputs / weights\n", + " weighted_sum = np.dot(inputs, weights)\n", + " \n", + " # Activate!\n", + " activated_output = sigmoid(weighted_sum)\n", + " \n", + " # Cac error\n", + " error = correct_outputs - activated_output\n", + " \n", + " adjustments = error * sigmoid_derivative(weighted_sum)\n", + " \n", + " # Update the Weights\n", + " weights += np.dot(inputs.T, adjustments)\n", + "\n", + " def predict(self, X):\n", + " return activated_output" + ], + "execution_count": 255, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "omxqQ8NqbA5y", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "e10a5694-af06-42d0-a345-937cf0eef4aa" + }, + "source": [ + "error" + ], + "execution_count": 248, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[ 1.32488220e-07],\n", + " [ 4.79463280e-03],\n", + " [ 4.79463280e-03],\n", + " [-5.67570855e-03]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 248 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "XLLrwkWZTIhT", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "e0921306-d039-4f1f-e61c-500b903ca90c" + }, + "source": [ + "model = Perceptron()\n", + "\n", + "model.fit(X, y)\n", + "\n", + "final_output = model.predict(X)\n", + "\n", + "print(final_output)" + ], + "execution_count": 258, + "outputs": [ + { + "output_type": "stream", + "text": [ + "[[0.99999987]\n", + " [0.99520537]\n", + " [0.99520537]\n", + " [0.00567571]]\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "k8o161cEcVOj", + "colab_type": "code", + "colab": {} + }, + "source": [ + "from sklearn.metrics import accuracy_score" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "6QR4oAW1xdyu" + }, + "source": [ + "## Stretch Goals:\n", + "\n", + "- Research \"backpropagation\" to learn how weights get updated in neural networks (tomorrow's lecture). \n", + "- Implement a multi-layer perceptron. (for non-linearly separable classes)\n", + "- Try and implement your own backpropagation algorithm.\n", + "- What are the pros and cons of the different activation functions? How should you decide between them for the different layers of a neural network?" + ] + } + ] +} \ No newline at end of file From ab9c6c5e55753bd1aa1ec7323128c3f6c7681bd2 Mon Sep 17 00:00:00 2001 From: KristineYW <63246056+KristineYW@users.noreply.github.com> Date: Mon, 13 Jul 2020 21:56:58 -0400 Subject: [PATCH 2/3] Created using Colaboratory --- ...ang_LS_DS_431_Intro_to_NN_Assignment.ipynb | 1080 +++++++++++++++-- 1 file changed, 986 insertions(+), 94 deletions(-) diff --git a/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb b/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb index 6cf455a64..e1da4005f 100644 --- a/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb +++ b/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb @@ -136,7 +136,7 @@ "\n", "df = pd.DataFrame.from_dict(data).astype('int')" ], - "execution_count": 66, + "execution_count": 1, "outputs": [] }, { @@ -159,7 +159,7 @@ "\n", "correct_outputs = [[1], [1], [1], [0]]" ], - "execution_count": 153, + "execution_count": 2, "outputs": [] }, { @@ -177,7 +177,7 @@ " sx = sigmoid(x)\n", " return sx * (1 - sx)" ], - "execution_count": 154, + "execution_count": 3, "outputs": [] }, { @@ -190,7 +190,7 @@ "source": [ "weights = np.random.random((3,1))" ], - "execution_count": 155, + "execution_count": 4, "outputs": [] }, { @@ -202,12 +202,12 @@ "base_uri": "https://localhost:8080/", "height": 68 }, - "outputId": "aee54805-0b5b-42b0-a547-874f8b1b1245" + "outputId": "c2ab2f0f-a608-49b2-ef66-b62f76f9ddb2" }, "source": [ "weights" ], - "execution_count": 156, + "execution_count": 5, "outputs": [ { "output_type": "execute_result", @@ -221,7 +221,7 @@ "metadata": { "tags": [] }, - "execution_count": 156 + "execution_count": 5 } ] }, @@ -234,13 +234,13 @@ "base_uri": "https://localhost:8080/", "height": 51 }, - "outputId": "c180ae88-020c-4aa8-a229-6efbda112efc" + "outputId": "a77bc3bb-70a4-4f80-f675-5a20fac7911b" }, "source": [ "print(inputs.shape)\n", "print(weights.shape)" ], - "execution_count": 157, + "execution_count": 6, "outputs": [ { "output_type": "stream", @@ -261,14 +261,14 @@ "base_uri": "https://localhost:8080/", "height": 85 }, - "outputId": "fa1f8fb5-7d3b-420d-f720-5809f2b13f76" + "outputId": "b448ad1e-5956-4d9e-a4f2-ff5c9c62f07b" }, "source": [ "weighted_sum = np.dot(inputs, weights)\n", "\n", "weighted_sum" ], - "execution_count": 158, + "execution_count": 7, "outputs": [ { "output_type": "execute_result", @@ -283,7 +283,7 @@ "metadata": { "tags": [] }, - "execution_count": 158 + "execution_count": 7 } ] }, @@ -296,14 +296,14 @@ "base_uri": "https://localhost:8080/", "height": 85 }, - "outputId": "9184e776-6d24-4761-c385-087a0f519fe8" + "outputId": "3b3fd634-81de-46e1-e024-7effe553d0fe" }, "source": [ "activated_output = sigmoid(weighted_sum)\n", "\n", "activated_output" ], - "execution_count": 159, + "execution_count": 8, "outputs": [ { "output_type": "execute_result", @@ -318,7 +318,7 @@ "metadata": { "tags": [] }, - "execution_count": 159 + "execution_count": 8 } ] }, @@ -331,14 +331,14 @@ "base_uri": "https://localhost:8080/", "height": 85 }, - "outputId": "0fe5f53a-be4e-4e86-d274-4eef27409d23" + "outputId": "dc9c18e7-2f21-4fb0-dafc-75fd0740191a" }, "source": [ "error = correct_outputs - activated_output\n", "\n", "error" ], - "execution_count": 160, + "execution_count": 9, "outputs": [ { "output_type": "execute_result", @@ -353,7 +353,7 @@ "metadata": { "tags": [] }, - "execution_count": 160 + "execution_count": 9 } ] }, @@ -366,14 +366,14 @@ "base_uri": "https://localhost:8080/", "height": 85 }, - "outputId": "988515af-8f6c-4814-edc8-e20f4dbf5dbb" + "outputId": "a04c200f-1a0b-4d39-e663-f29e3d4520e0" }, "source": [ "adjustments = error * sigmoid_derivative(weighted_sum)\n", "\n", "adjustments" ], - "execution_count": 161, + "execution_count": 10, "outputs": [ { "output_type": "execute_result", @@ -388,7 +388,39 @@ "metadata": { "tags": [] }, - "execution_count": 161 + "execution_count": 10 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "HcIlEtY8yxY4", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 68 + }, + "outputId": "b27ca578-7124-4f7e-985a-782f19cb5440" + }, + "source": [ + "inputs.T" + ], + "execution_count": 11, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0, 1, 0, 1],\n", + " [0, 0, 1, 1],\n", + " [1, 1, 1, 1]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 11 } ] }, @@ -401,14 +433,14 @@ "base_uri": "https://localhost:8080/", "height": 68 }, - "outputId": "8a3323ad-6729-439b-985c-993a16d64767" + "outputId": "28c36cd1-2751-4934-917f-66f724e0db47" }, "source": [ "weights += np.dot(inputs.T, adjustments)\n", "\n", "weights" ], - "execution_count": 162, + "execution_count": 13, "outputs": [ { "output_type": "execute_result", @@ -422,7 +454,66 @@ "metadata": { "tags": [] }, - "execution_count": 162 + "execution_count": 13 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "lavbYNs1y1B0", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 51 + }, + "outputId": "86e907b6-b3a0-477e-de4c-ed43a67e6671" + }, + "source": [ + "print(adjustments.shape)\n", + "print(inputs.T.shape)" + ], + "execution_count": 12, + "outputs": [ + { + "output_type": "stream", + "text": [ + "(4, 1)\n", + "(3, 4)\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "uyORAp7FuIlf", + "colab_type": "code", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 68 + }, + "outputId": "e64dbac0-a9d9-4f34-b8b3-9cd3296b74ea" + }, + "source": [ + "inputs.T" + ], + "execution_count": 14, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0, 1, 0, 1],\n", + " [0, 0, 1, 1],\n", + " [1, 1, 1, 1]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 14 } ] }, @@ -435,7 +526,7 @@ "base_uri": "https://localhost:8080/", "height": 170 }, - "outputId": "f5cb8b9b-0640-4ce5-fa0f-277643ab0dc2" + "outputId": "212abea2-ee98-486f-a633-ed1a042d36d6" }, "source": [ "for iteration in range(100000):\n", @@ -460,20 +551,20 @@ "print(\"Output after training\")\n", "print(activated_output)\n" ], - "execution_count": 165, + "execution_count": 15, "outputs": [ { "output_type": "stream", "text": [ "Weights after training\n", - "[[-10.50132913]\n", - " [-10.50132913]\n", - " [ 15.8367857 ]]\n", + "[[-10.39526168]\n", + " [-10.39526168]\n", + " [ 15.67772134]]\n", "Output after training\n", - "[[0.99999987]\n", - " [0.99520537]\n", - " [0.99520537]\n", - " [0.00567571]]\n" + "[[0.99999984]\n", + " [0.99494573]\n", + " [0.99494573]\n", + " [0.00598321]]\n" ], "name": "stdout" } @@ -501,13 +592,13 @@ "base_uri": "https://localhost:8080/", "height": 204 }, - "outputId": "d276606d-b252-451f-8376-1127d2fffbbc" + "outputId": "a9bea7e9-08fd-406b-888e-28827492267d" }, "source": [ "diabetes = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/diabetes.csv')\n", "diabetes.head()" ], - "execution_count": 214, + "execution_count": 25, "outputs": [ { "output_type": "execute_result", @@ -621,7 +712,7 @@ "metadata": { "tags": [] }, - "execution_count": 214 + "execution_count": 25 } ] }, @@ -644,7 +735,7 @@ "base_uri": "https://localhost:8080/", "height": 204 }, - "outputId": "944a651e-5517-4b69-a8bf-46189d91c75e" + "outputId": "08d35d6a-471c-4f68-8cf1-91efd102d364" }, "source": [ "from sklearn.preprocessing import MinMaxScaler, Normalizer\n", @@ -653,11 +744,11 @@ "\n", "X = diabetes.drop(['Outcome'], axis=1)\n", "\n", - "target = diabetes['Outcome']\n", + "y = diabetes['Outcome']\n", "\n", "X.head()" ], - "execution_count": 215, + "execution_count": 26, "outputs": [ { "output_type": "execute_result", @@ -765,7 +856,7 @@ "metadata": { "tags": [] }, - "execution_count": 215 + "execution_count": 26 } ] }, @@ -778,12 +869,12 @@ "base_uri": "https://localhost:8080/", "height": 34 }, - "outputId": "9fc7fc4f-2ae2-4418-d3f9-9dc58e20d146" + "outputId": "ef60a96d-d1ee-4b2b-d1e8-49ddd7692c41" }, "source": [ "X.shape" ], - "execution_count": 210, + "execution_count": 27, "outputs": [ { "output_type": "execute_result", @@ -795,7 +886,7 @@ "metadata": { "tags": [] }, - "execution_count": 210 + "execution_count": 27 } ] }, @@ -811,7 +902,7 @@ "x_scaled = min_max_scaler.fit_transform(X)\n", "X = pd.DataFrame(x_scaled, columns=[feats])" ], - "execution_count": 224, + "execution_count": 28, "outputs": [] }, { @@ -823,12 +914,12 @@ "base_uri": "https://localhost:8080/", "height": 204 }, - "outputId": "f7b245da-f801-42df-dccb-e261c85a6953" + "outputId": "e9b6d03b-154e-4508-e4e7-7f11b5af7912" }, "source": [ "X.head()" ], - "execution_count": 217, + "execution_count": 29, "outputs": [ { "output_type": "execute_result", @@ -936,7 +1027,7 @@ "metadata": { "tags": [] }, - "execution_count": 217 + "execution_count": 29 } ] }, @@ -956,10 +1047,10 @@ " self.rate = rate\n", " self.niter = niter\n", " \n", - " def __sigmoid(self, x):\n", + " def sigmoid(self, x):\n", " return 1/(1 + np.exp(-x))\n", " \n", - " def __sigmoid_derivative(self, x):\n", + " def sigmoid_derivative(self, x):\n", " sx = sigmoid(x)\n", " return sx * (1 - sx)\n", "\n", @@ -967,9 +1058,6 @@ "\n", " def fit(self, X, y):\n", "\n", - " inputs = X.values\n", - " correct_outputs = [[i] for i in target]\n", - "\n", " \"\"\"Fit training data\n", " X : Training vectors, X.shape : [#samples, #features]\n", " y : Target values, y.shape : [#samples]\n", @@ -979,104 +1067,908 @@ "\n", " for i in range(self.niter):\n", " # Weighted sum of inputs / weights\n", - " weighted_sum = np.dot(inputs, weights)\n", - " \n", + " weighted_sum = np.dot(X, weights)\n", + "\n", " # Activate!\n", - " activated_output = sigmoid(weighted_sum)\n", - " \n", + " activated_output = self.sigmoid(weighted_sum)\n", + "\n", " # Cac error\n", - " error = correct_outputs - activated_output\n", - " \n", - " adjustments = error * sigmoid_derivative(weighted_sum)\n", - " \n", + " error = y - activated_output\n", + "\n", + " # Can add in learning rate for better performance.\n", + " adjustments = error * self.sigmoid_derivative(weighted_sum)\n", + "\n", " # Update the Weights\n", - " weights += np.dot(inputs.T, adjustments)\n", + " weights += np.dot(X.T, adjustments)\n", + "\n", + " return activated_output\n", "\n", " def predict(self, X):\n", " return activated_output" ], - "execution_count": 255, + "execution_count": 30, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "DGj0SO65tF4n", + "colab_type": "code", + "colab": {} + }, + "source": [ + "X = np.array(X)\n", + "y = [[i] for i in y]" + ], + "execution_count": 31, "outputs": [] }, { "cell_type": "code", "metadata": { - "id": "omxqQ8NqbA5y", + "id": "p2JK6RXGnPyZ", "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", - "height": 85 + "height": 1000 }, - "outputId": "e10a5694-af06-42d0-a345-937cf0eef4aa" + "outputId": "077b9cda-f85e-49cb-9de8-452269ccbe6b" }, "source": [ - "error" + "model = Perceptron(rate = 0.01, niter=10)\n", + "\n", + "model.fit(X, y)" ], - "execution_count": 248, + "execution_count": 43, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ - "array([[ 1.32488220e-07],\n", - " [ 4.79463280e-03],\n", - " [ 4.79463280e-03],\n", - " [-5.67570855e-03]])" + "array([[1.61829738e-14],\n", + " [1.43597322e-10],\n", + " [3.02399304e-13],\n", + " [2.10288669e-10],\n", + " [3.90313700e-13],\n", + " [1.78191955e-11],\n", + " [7.07190417e-10],\n", + " [5.66680883e-08],\n", + " [1.53550458e-15],\n", + " [5.69845914e-12],\n", + " [2.99354008e-13],\n", + " [9.75971642e-15],\n", + " [1.73979965e-14],\n", + " [1.70126903e-14],\n", + " [4.65504642e-14],\n", + " [4.16338948e-07],\n", + " [9.22379817e-15],\n", + " [8.68770233e-12],\n", + " [2.64653812e-10],\n", + " [1.58942369e-12],\n", + " [6.18552937e-15],\n", + " [3.34723502e-13],\n", + " [1.85391420e-16],\n", + " [1.67159002e-13],\n", + " [2.34854692e-16],\n", + " [2.26605089e-13],\n", + " [4.72756598e-14],\n", + " [2.55080645e-10],\n", + " [1.67897803e-14],\n", + " [1.53730398e-13],\n", + " [1.36439578e-13],\n", + " [1.67503024e-14],\n", + " [1.38518822e-09],\n", + " [1.51854321e-11],\n", + " [7.72511383e-14],\n", + " [1.70088030e-11],\n", + " [1.10219812e-13],\n", + " [1.04766459e-13],\n", + " [3.58114353e-12],\n", + " [1.42677903e-14],\n", + " [9.54523583e-14],\n", + " [2.21293605e-14],\n", + " [3.90055085e-13],\n", + " [7.65600416e-19],\n", + " [1.10821806e-12],\n", + " [1.72006809e-15],\n", + " [2.62496175e-11],\n", + " [1.47772975e-10],\n", + " [9.64973196e-13],\n", + " [1.12079268e-04],\n", + " [6.01487791e-11],\n", + " [1.57988477e-09],\n", + " [1.06318818e-10],\n", + " [6.32430629e-17],\n", + " [1.04474385e-14],\n", + " [3.32015207e-08],\n", + " [1.05120202e-15],\n", + " [4.51094369e-15],\n", + " [9.36846629e-15],\n", + " [3.67682184e-12],\n", + " [1.71763221e-03],\n", + " [6.21387172e-13],\n", + " [5.66677078e-09],\n", + " [5.51440645e-12],\n", + " [6.00442940e-12],\n", + " [5.91028176e-12],\n", + " [1.74854984e-13],\n", + " [2.67657810e-14],\n", + " [8.00041478e-10],\n", + " [7.50548359e-14],\n", + " [1.05226588e-11],\n", + " [9.38493694e-13],\n", + " [1.84666880e-15],\n", + " [8.40765306e-14],\n", + " [2.95514454e-11],\n", + " [2.59631590e-06],\n", + " [2.88864239e-11],\n", + " [1.54602023e-12],\n", + " [5.10672972e-08],\n", + " [5.49430587e-11],\n", + " [3.67755369e-09],\n", + " [4.27522225e-03],\n", + " [1.88573011e-12],\n", + " [1.97517069e-10],\n", + " [3.67448550e-16],\n", + " [1.40972174e-12],\n", + " [2.34509055e-14],\n", + " [5.57569393e-12],\n", + " [8.04783655e-15],\n", + " [8.16634758e-11],\n", + " [3.49362783e-08],\n", + " [3.56143038e-13],\n", + " [7.54894042e-14],\n", + " [2.89505656e-12],\n", + " [5.12433857e-13],\n", + " [5.86950062e-14],\n", + " [1.06305982e-10],\n", + " [3.83286814e-08],\n", + " [2.17744950e-10],\n", + " [1.32247401e-15],\n", + " [5.05928718e-14],\n", + " [4.84426951e-11],\n", + " [3.17741884e-12],\n", + " [2.07297260e-10],\n", + " [3.46520003e-11],\n", + " [1.77507865e-11],\n", + " [3.23419980e-13],\n", + " [1.90091452e-12],\n", + " [9.68864929e-11],\n", + " [1.67317788e-12],\n", + " [4.23726089e-14],\n", + " [2.47428342e-14],\n", + " [1.47190507e-11],\n", + " [3.92681826e-10],\n", + " [2.11061509e-13],\n", + " [1.25081623e-14],\n", + " [1.61061664e-12],\n", + " [1.48120179e-09],\n", + " [1.05171896e-10],\n", + " [4.54648838e-11],\n", + " [3.15700113e-16],\n", + " [2.40974185e-12],\n", + " [2.54251276e-12],\n", + " [3.62346364e-13],\n", + " [1.70922151e-11],\n", + " [6.05948348e-11],\n", + " [1.87322165e-13],\n", + " [1.50557818e-11],\n", + " [1.11831507e-13],\n", + " [2.00025451e-12],\n", + " [1.06951028e-13],\n", + " [3.63080761e-12],\n", + " [4.84626908e-14],\n", + " [5.94195093e-13],\n", + " [1.98980101e-10],\n", + " [1.05022424e-11],\n", + " [2.15516331e-11],\n", + " [2.20543875e-10],\n", + " [1.94822053e-12],\n", + " [8.72700777e-13],\n", + " [4.68659244e-12],\n", + " [1.03281301e-13],\n", + " [1.12865350e-10],\n", + " [4.54896747e-12],\n", + " [4.13156498e-13],\n", + " [2.80527579e-09],\n", + " [3.39637745e-12],\n", + " [2.03097798e-12],\n", + " [6.09286349e-14],\n", + " [1.76777952e-10],\n", + " [6.33142028e-14],\n", + " [9.51647391e-11],\n", + " [3.81916792e-16],\n", + " [2.44809721e-15],\n", + " [4.93292938e-16],\n", + " [9.17295854e-17],\n", + " [9.53004505e-10],\n", + " [1.42740864e-10],\n", + " [5.50549157e-11],\n", + " [5.92239868e-17],\n", + " [8.14636862e-15],\n", + " [1.55788383e-13],\n", + " [1.15282313e-13],\n", + " [7.00778620e-11],\n", + " [4.08088531e-13],\n", + " [1.12186654e-12],\n", + " [1.06188132e-12],\n", + " [8.07778875e-12],\n", + " [2.49099226e-11],\n", + " [7.16647196e-13],\n", + " [3.45221450e-12],\n", + " [1.86091794e-13],\n", + " [9.68202698e-07],\n", + " [1.01158895e-11],\n", + " [1.81630041e-10],\n", + " [1.34730530e-15],\n", + " [9.98038570e-12],\n", + " [5.23206362e-18],\n", + " [2.55367640e-14],\n", + " [4.82904192e-14],\n", + " [6.58214000e-11],\n", + " [7.90626070e-12],\n", + " [2.76160496e-08],\n", + " [1.96537708e-09],\n", + " [1.81905694e-12],\n", + " [1.36761962e-15],\n", + " [9.94890536e-16],\n", + " [3.05039501e-15],\n", + " [3.07368365e-13],\n", + " [5.86474741e-14],\n", + " [6.43356539e-10],\n", + " [7.02697085e-14],\n", + " [5.12805270e-13],\n", + " [1.85394938e-10],\n", + " [2.69545405e-10],\n", + " [1.22181841e-15],\n", + " [1.92147720e-09],\n", + " [1.44530612e-10],\n", + " [8.95991990e-13],\n", + " [1.17874789e-12],\n", + " [2.61605078e-12],\n", + " [2.74479188e-13],\n", + " [2.20457161e-11],\n", + " [2.06682721e-10],\n", + " [1.64703808e-13],\n", + " [7.72181666e-12],\n", + " [7.76006736e-17],\n", + " [5.06292786e-16],\n", + " [4.84672939e-11],\n", + " [3.71438490e-16],\n", + " [6.20197541e-10],\n", + " [3.52646500e-15],\n", + " [8.20587085e-17],\n", + " [3.86155603e-13],\n", + " [5.10858784e-14],\n", + " [6.82488643e-16],\n", + " [1.66310482e-12],\n", + " [7.83494448e-13],\n", + " [4.35267785e-12],\n", + " [4.78482673e-12],\n", + " [5.93150527e-14],\n", + " [6.13942441e-15],\n", + " [3.31711177e-07],\n", + " [1.14097952e-13],\n", + " [1.43542828e-10],\n", + " [1.23784847e-11],\n", + " [2.42758026e-11],\n", + " [3.42240660e-13],\n", + " [3.45976264e-17],\n", + " [1.78700898e-13],\n", + " [2.05501291e-14],\n", + " [1.40832771e-15],\n", + " [4.32722412e-11],\n", + " [5.46609176e-12],\n", + " [1.21981798e-10],\n", + " [2.65850462e-14],\n", + " [2.71708551e-16],\n", + " [4.99554843e-16],\n", + " [2.89349496e-15],\n", + " [2.28239681e-10],\n", + " [2.16534694e-10],\n", + " [7.90788231e-12],\n", + " [1.24137604e-10],\n", + " [1.02998181e-11],\n", + " [3.51002620e-14],\n", + " [2.82856438e-16],\n", + " [1.78125387e-12],\n", + " [9.84770935e-17],\n", + " [7.33801616e-14],\n", + " [2.61374386e-12],\n", + " [5.34043232e-11],\n", + " [2.40978686e-12],\n", + " [5.32234855e-11],\n", + " [3.30322838e-11],\n", + " [5.08518805e-12],\n", + " [7.41407272e-12],\n", + " [1.60856522e-11],\n", + " [2.46114956e-11],\n", + " [1.29905104e-12],\n", + " [6.76167870e-16],\n", + " [4.73609144e-14],\n", + " [6.06789165e-08],\n", + " [5.99603248e-12],\n", + " [6.20136681e-14],\n", + " [2.03182312e-11],\n", + " [8.88938753e-13],\n", + " [4.25731297e-08],\n", + " [1.79133579e-13],\n", + " [7.28854074e-09],\n", + " [1.78590353e-07],\n", + " [1.55981836e-15],\n", + " [9.53760017e-11],\n", + " [1.00671470e-11],\n", + " [7.86401245e-12],\n", + " [6.69691068e-13],\n", + " [4.01620180e-13],\n", + " [3.24656317e-11],\n", + " [8.05979218e-11],\n", + " [3.00800758e-12],\n", + " [7.02861158e-11],\n", + " [1.50670074e-12],\n", + " [2.69185123e-14],\n", + " [3.06679692e-14],\n", + " [2.04673805e-14],\n", + " [6.84831376e-12],\n", + " [9.88898983e-14],\n", + " [4.62133337e-16],\n", + " [6.29254767e-15],\n", + " [9.33451267e-10],\n", + " [3.87300016e-13],\n", + " [2.49563513e-12],\n", + " [7.86210342e-12],\n", + " [7.71989980e-15],\n", + " [2.68977586e-12],\n", + " [3.62074654e-11],\n", + " [1.03175500e-13],\n", + " [3.20796479e-13],\n", + " [3.20679036e-13],\n", + " [2.93633690e-14],\n", + " [2.32193224e-12],\n", + " [1.16338306e-08],\n", + " [2.28177344e-12],\n", + " [9.34833877e-13],\n", + " [5.05378775e-15],\n", + " [3.70853775e-12],\n", + " [1.84169907e-13],\n", + " [5.53922907e-14],\n", + " [2.37252824e-11],\n", + " [1.46832622e-12],\n", + " [7.61073313e-13],\n", + " [4.06894126e-11],\n", + " [1.36273724e-12],\n", + " [6.17649577e-13],\n", + " [1.91134170e-10],\n", + " [2.85568729e-14],\n", + " [6.34418790e-12],\n", + " [4.79043743e-11],\n", + " [1.31463487e-13],\n", + " [1.11047373e-12],\n", + " [1.94993154e-14],\n", + " [1.06637048e-11],\n", + " [3.00124244e-12],\n", + " [1.03397704e-11],\n", + " [6.98012155e-16],\n", + " [1.84449373e-12],\n", + " [1.40942203e-12],\n", + " [1.52363821e-12],\n", + " [2.02719835e-14],\n", + " [7.82706248e-14],\n", + " [2.45479030e-12],\n", + " [3.90428838e-13],\n", + " [3.57481532e-10],\n", + " [7.69100514e-10],\n", + " [2.20157485e-12],\n", + " [8.35441584e-10],\n", + " [2.13250956e-15],\n", + " [1.11698372e-07],\n", + " [2.26718044e-12],\n", + " [2.62538034e-15],\n", + " [1.71120355e-15],\n", + " [1.21572885e-11],\n", + " [1.80197570e-11],\n", + " [1.14879956e-08],\n", + " [3.20494886e-13],\n", + " [1.22979340e-12],\n", + " [2.18044378e-15],\n", + " [7.59172170e-11],\n", + " [2.80034324e-06],\n", + " [3.19556774e-10],\n", + " [9.94316163e-11],\n", + " [2.44665623e-12],\n", + " [4.55980104e-13],\n", + " [6.66817674e-12],\n", + " [3.97230665e-10],\n", + " [3.88696032e-12],\n", + " [5.40227247e-15],\n", + " [6.03114272e-12],\n", + " [2.33754296e-10],\n", + " [1.20726456e-13],\n", + " [8.50811513e-16],\n", + " [1.21053714e-14],\n", + " [2.09283337e-13],\n", + " [5.76340135e-16],\n", + " [2.07109718e-14],\n", + " [6.37914686e-14],\n", + " [2.60779663e-11],\n", + " [6.33855870e-12],\n", + " [8.24165578e-10],\n", + " [1.61429055e-11],\n", + " [5.62299155e-15],\n", + " [4.65203578e-17],\n", + " [7.46383115e-10],\n", + " [8.10462231e-11],\n", + " [1.61507548e-11],\n", + " [2.75159097e-12],\n", + " [1.65661637e-16],\n", + " [3.01933336e-11],\n", + " [3.09502724e-11],\n", + " [2.19497865e-14],\n", + " [6.09864854e-15],\n", + " [3.76655996e-12],\n", + " [3.10735599e-10],\n", + " [1.37547794e-10],\n", + " [1.37275913e-10],\n", + " [1.50844189e-11],\n", + " [7.25265340e-10],\n", + " [4.24686633e-13],\n", + " [1.10429285e-15],\n", + " [7.70874120e-15],\n", + " [5.89417370e-12],\n", + " [7.80242468e-12],\n", + " [1.31565951e-14],\n", + " [2.66770574e-11],\n", + " [1.21327345e-11],\n", + " [8.53035029e-14],\n", + " [3.20320450e-12],\n", + " [4.59823899e-11],\n", + " [2.28536897e-12],\n", + " [1.02626627e-09],\n", + " [1.26185987e-14],\n", + " [1.27491970e-10],\n", + " [1.02534593e-11],\n", + " [1.24759747e-14],\n", + " [3.59554909e-12],\n", + " [4.53924057e-13],\n", + " [5.10025744e-12],\n", + " [6.74015180e-12],\n", + " [1.81627445e-09],\n", + " [5.95386042e-15],\n", + " [1.74273322e-15],\n", + " [3.43248779e-14],\n", + " [1.80617952e-12],\n", + " [5.97802611e-15],\n", + " [2.54313272e-12],\n", + " [2.07839662e-12],\n", + " [2.40332620e-15],\n", + " [3.43909180e-11],\n", + " [8.00648224e-15],\n", + " [2.23863495e-09],\n", + " [7.72131574e-12],\n", + " [8.62658431e-15],\n", + " [9.81317629e-11],\n", + " [2.85643409e-12],\n", + " [2.09668216e-11],\n", + " [1.22997635e-15],\n", + " [1.13376620e-15],\n", + " [1.28722441e-03],\n", + " [4.73527128e-14],\n", + " [3.99400864e-15],\n", + " [1.04830640e-12],\n", + " [1.54919005e-05],\n", + " [1.06357684e-11],\n", + " [9.96399705e-11],\n", + " [7.65538187e-12],\n", + " [9.12089695e-11],\n", + " [2.91881562e-08],\n", + " [1.48128350e-15],\n", + " [7.21690641e-13],\n", + " [7.55497505e-10],\n", + " [2.00408519e-13],\n", + " [1.35843188e-16],\n", + " [7.51002048e-11],\n", + " [5.28931772e-12],\n", + " [3.14987414e-12],\n", + " [2.29235379e-11],\n", + " [1.59139278e-18],\n", + " [4.74917314e-11],\n", + " [9.59074505e-13],\n", + " [1.37520204e-11],\n", + " [6.60222712e-12],\n", + " [1.68056233e-09],\n", + " [9.28088292e-12],\n", + " [6.54008085e-12],\n", + " [2.40198908e-07],\n", + " [3.26405902e-11],\n", + " [7.18694767e-15],\n", + " [2.28713515e-11],\n", + " [2.56888406e-11],\n", + " [9.34143849e-17],\n", + " [1.82049302e-14],\n", + " [8.50920702e-13],\n", + " [8.93761206e-09],\n", + " [1.33751071e-12],\n", + " [5.41166059e-12],\n", + " [5.66528401e-14],\n", + " [3.92445989e-10],\n", + " [1.10452955e-08],\n", + " [9.43701707e-12],\n", + " [1.03819063e-07],\n", + " [8.70911177e-16],\n", + " [8.90754614e-15],\n", + " [1.21437025e-12],\n", + " [4.34413257e-12],\n", + " [5.31176667e-14],\n", + " [9.10274245e-11],\n", + " [1.41818709e-13],\n", + " [2.10424178e-13],\n", + " [2.63324660e-12],\n", + " [3.03865549e-13],\n", + " [2.49581868e-14],\n", + " [3.79887297e-14],\n", + " [1.15719871e-13],\n", + " [3.03111575e-10],\n", + " [3.04258517e-12],\n", + " [7.92860521e-09],\n", + " [1.33049920e-13],\n", + " [1.63012818e-13],\n", + " [1.35408968e-16],\n", + " [3.39157314e-11],\n", + " [2.11209477e-15],\n", + " [2.12183602e-11],\n", + " [4.86610498e-13],\n", + " [5.13892855e-12],\n", + " [4.57796571e-13],\n", + " [2.03566936e-03],\n", + " [7.19738548e-14],\n", + " [5.16444240e-11],\n", + " [6.32835509e-11],\n", + " [4.76066646e-15],\n", + " [1.79483166e-14],\n", + " [1.38470093e-12],\n", + " [6.66474573e-12],\n", + " [1.81330979e-10],\n", + " [2.99921365e-12],\n", + " [5.02924910e-13],\n", + " [4.06211997e-12],\n", + " [1.63584186e-15],\n", + " [1.24216248e-11],\n", + " [4.94204321e-10],\n", + " [6.43708899e-13],\n", + " [1.27434893e-12],\n", + " [5.54254482e-11],\n", + " [3.03727612e-11],\n", + " [7.24513482e-10],\n", + " [6.76492278e-10],\n", + " [2.40627597e-13],\n", + " [9.42401580e-16],\n", + " [4.19714328e-14],\n", + " [4.88771443e-11],\n", + " [9.38251707e-14],\n", + " [3.24975000e-10],\n", + " [2.35960860e-13],\n", + " [8.53758908e-05],\n", + " [2.73357071e-13],\n", + " [8.02902854e-11],\n", + " [1.11301092e-09],\n", + " [1.00264661e-09],\n", + " [1.37894516e-11],\n", + " [8.35416179e-12],\n", + " [1.77163438e-10],\n", + " [1.82381723e-11],\n", + " [1.59334003e-12],\n", + " [1.15383904e-12],\n", + " [1.01508608e-06],\n", + " [9.96095875e-11],\n", + " [1.22685915e-07],\n", + " [2.87203661e-12],\n", + " [3.76208356e-09],\n", + " [8.79629885e-14],\n", + " [1.85018784e-15],\n", + " [4.37783006e-14],\n", + " [6.33243674e-13],\n", + " [2.28941911e-14],\n", + " [5.18126662e-13],\n", + " [7.47168869e-12],\n", + " [8.44588108e-17],\n", + " [7.94423749e-17],\n", + " [1.43503928e-12],\n", + " [4.37723111e-15],\n", + " [3.57081337e-17],\n", + " [1.86582946e-11],\n", + " [2.02047123e-11],\n", + " [2.55988087e-13],\n", + " [1.88146439e-10],\n", + " [3.70180163e-11],\n", + " [9.75955588e-13],\n", + " [3.25662627e-12],\n", + " [1.72464760e-12],\n", + " [4.17268830e-14],\n", + " [1.00872388e-11],\n", + " [5.12639785e-13],\n", + " [3.81724529e-15],\n", + " [1.13948173e-11],\n", + " [4.80009231e-11],\n", + " [2.47041842e-11],\n", + " [6.16790178e-10],\n", + " [3.77499157e-12],\n", + " [9.55219086e-12],\n", + " [8.35210210e-14],\n", + " [3.54453651e-12],\n", + " [1.12660617e-10],\n", + " [1.37279365e-12],\n", + " [3.39600533e-11],\n", + " [8.36264347e-11],\n", + " [3.19707997e-14],\n", + " [3.67493750e-11],\n", + " [2.06035928e-10],\n", + " [4.60486556e-13],\n", + " [2.31463560e-12],\n", + " [3.04726796e-17],\n", + " [2.68669315e-15],\n", + " [4.66375218e-11],\n", + " [9.18707246e-14],\n", + " [1.08265211e-12],\n", + " [3.67246907e-14],\n", + " [2.83970903e-09],\n", + " [6.07861851e-13],\n", + " [1.29799990e-10],\n", + " [3.65019963e-16],\n", + " [1.09491826e-04],\n", + " [5.74761475e-16],\n", + " [1.47884478e-13],\n", + " [3.47705860e-13],\n", + " [1.64707517e-10],\n", + " [5.74982648e-14],\n", + " [1.20609179e-14],\n", + " [1.70377959e-11],\n", + " [7.90000624e-08],\n", + " [1.23315478e-13],\n", + " [5.66861124e-09],\n", + " [2.63977926e-12],\n", + " [3.89406130e-06],\n", + " [2.81672320e-12],\n", + " [3.01804400e-15],\n", + " [5.91502676e-09],\n", + " [5.38744015e-12],\n", + " [4.75324987e-16],\n", + " [7.18585002e-10],\n", + " [7.45391440e-15],\n", + " [2.63197426e-10],\n", + " [1.07734032e-10],\n", + " [1.56201946e-13],\n", + " [7.14501691e-17],\n", + " [3.20878003e-13],\n", + " [6.53941936e-15],\n", + " [8.19972343e-11],\n", + " [3.29038863e-13],\n", + " [7.82072676e-09],\n", + " [3.61648271e-14],\n", + " [1.00686331e-06],\n", + " [6.28808723e-14],\n", + " [6.56054336e-12],\n", + " [4.85897102e-17],\n", + " [4.17583830e-12],\n", + " [1.71330233e-10],\n", + " [1.01621334e-13],\n", + " [1.37930414e-10],\n", + " [3.94359672e-12],\n", + " [3.77981115e-13],\n", + " [1.97059689e-10],\n", + " [1.48533274e-11],\n", + " [1.92460219e-12],\n", + " [3.93228730e-10],\n", + " [2.16384849e-12],\n", + " [1.42294101e-10],\n", + " [1.33600219e-12],\n", + " [1.05853241e-11],\n", + " [9.28499404e-12],\n", + " [9.13414406e-14],\n", + " [2.18892769e-10],\n", + " [2.39551895e-12],\n", + " [4.46989514e-12],\n", + " [1.57760348e-13],\n", + " [2.05358901e-06],\n", + " [4.72712500e-12],\n", + " [1.54972939e-14],\n", + " [4.30448227e-13],\n", + " [3.47519244e-13],\n", + " [9.47631279e-15],\n", + " [2.64701678e-10],\n", + " [1.28845734e-09],\n", + " [1.38694417e-11],\n", + " [2.25006745e-13],\n", + " [3.30068837e-10],\n", + " [9.07437889e-12],\n", + " [7.02733627e-13],\n", + " [3.61258974e-10],\n", + " [9.78788991e-15],\n", + " [7.05014599e-16],\n", + " [7.40239801e-13],\n", + " [1.41958406e-14],\n", + " [1.46815056e-16],\n", + " [9.87309273e-18],\n", + " [9.34476512e-16],\n", + " [1.78540280e-12],\n", + " [4.00802707e-13],\n", + " [1.70874895e-14],\n", + " [1.42307942e-12],\n", + " [4.71251698e-12],\n", + " [9.94367759e-15],\n", + " [1.23179307e-14],\n", + " [7.77170757e-10],\n", + " [2.13210967e-14],\n", + " [7.38676916e-17],\n", + " [2.40511944e-13],\n", + " [4.21957251e-14],\n", + " [3.26414403e-14],\n", + " [4.31257165e-10],\n", + " [1.14039996e-10],\n", + " [2.36622522e-10],\n", + " [7.62617937e-09],\n", + " [3.76590546e-15],\n", + " [3.68670669e-12],\n", + " [1.11705639e-12],\n", + " [1.20297308e-11],\n", + " [4.64267698e-13],\n", + " [9.29340757e-11],\n", + " [6.35076410e-10],\n", + " [1.14524351e-12],\n", + " [1.32303082e-15],\n", + " [2.46091808e-12],\n", + " [1.21910039e-17],\n", + " [3.01201335e-13],\n", + " [2.23657217e-14],\n", + " [2.89940274e-09],\n", + " [5.80438073e-15],\n", + " [9.73026436e-14],\n", + " [1.29225692e-05],\n", + " [7.45067378e-14],\n", + " [5.35123885e-13],\n", + " [3.65501868e-13],\n", + " [1.28141685e-13],\n", + " [9.32487852e-16],\n", + " [3.96097873e-08],\n", + " [5.41324214e-12],\n", + " [7.55966463e-13],\n", + " [2.28111787e-05],\n", + " [5.71633924e-11],\n", + " [2.44201743e-14],\n", + " [6.83322195e-12],\n", + " [8.19463292e-13],\n", + " [2.35531898e-13],\n", + " [5.21096652e-14],\n", + " [3.83528100e-11],\n", + " [3.75776068e-11],\n", + " [2.01623331e-14],\n", + " [2.15106371e-15],\n", + " [2.91537725e-12],\n", + " [5.52856597e-12],\n", + " [4.87886721e-13],\n", + " [4.34301048e-12],\n", + " [2.30506597e-12],\n", + " [4.77565265e-13],\n", + " [2.20294398e-14],\n", + " [4.56132359e-13],\n", + " [9.84072304e-14],\n", + " [4.72827736e-13],\n", + " [1.87148709e-13],\n", + " [1.87319402e-13],\n", + " [3.13494304e-09],\n", + " [6.44349466e-13],\n", + " [9.31923834e-13],\n", + " [2.51233291e-16],\n", + " [9.04930763e-11],\n", + " [2.03125052e-11],\n", + " [1.94638557e-11],\n", + " [6.51486926e-13],\n", + " [9.41980799e-12],\n", + " [3.75221478e-11],\n", + " [5.11980354e-12],\n", + " [1.24272906e-15],\n", + " [5.61398511e-10],\n", + " [1.89738667e-10],\n", + " [4.95224017e-15],\n", + " [3.14529910e-17],\n", + " [4.12741284e-14],\n", + " [4.43836366e-16],\n", + " [2.62269228e-13],\n", + " [8.10227674e-15],\n", + " [2.01572381e-12],\n", + " [1.35494787e-12],\n", + " [1.70628080e-13],\n", + " [8.14949845e-11],\n", + " [1.89147582e-16],\n", + " [6.78305010e-15],\n", + " [7.47777708e-15],\n", + " [4.22806639e-15],\n", + " [2.56030666e-12],\n", + " [9.99177135e-12],\n", + " [2.31869537e-16],\n", + " [2.02773017e-10],\n", + " [5.26313236e-16],\n", + " [3.81260805e-10],\n", + " [3.33813322e-14],\n", + " [1.23520435e-12],\n", + " [3.24288043e-12],\n", + " [3.10268655e-11],\n", + " [3.27769004e-11]])" ] }, "metadata": { "tags": [] }, - "execution_count": 248 + "execution_count": 43 } ] }, { "cell_type": "code", "metadata": { - "id": "XLLrwkWZTIhT", + "id": "Q831yzCT2Tdk", "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 85 }, - "outputId": "e0921306-d039-4f1f-e61c-500b903ca90c" + "outputId": "c1dda7e6-134d-4944-daae-af415dc14a68" }, "source": [ - "model = Perceptron()\n", + "predictions = model.predict(X)\n", "\n", - "model.fit(X, y)\n", - "\n", - "final_output = model.predict(X)\n", - "\n", - "print(final_output)" + "predictions" ], - "execution_count": 258, + "execution_count": 35, "outputs": [ { - "output_type": "stream", - "text": [ - "[[0.99999987]\n", - " [0.99520537]\n", - " [0.99520537]\n", - " [0.00567571]]\n" - ], - "name": "stdout" + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0.99999984],\n", + " [0.99494573],\n", + " [0.99494573],\n", + " [0.00598321]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 35 } ] }, { "cell_type": "code", "metadata": { - "id": "k8o161cEcVOj", + "id": "Ex_kXAjAnTcI", "colab_type": "code", - "colab": {} + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "outputId": "68daac4c-3a9b-4490-86aa-c4f2fc58cf2f" }, "source": [ - "from sklearn.metrics import accuracy_score" + "model.predict(X)" ], - "execution_count": null, - "outputs": [] + "execution_count": 34, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0.99999984],\n", + " [0.99494573],\n", + " [0.99494573],\n", + " [0.00598321]])" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 34 + } + ] }, { "cell_type": "markdown", From ad062966f283ac38ba5b36271812dbd3a5ac07fd Mon Sep 17 00:00:00 2001 From: KristineYW <63246056+KristineYW@users.noreply.github.com> Date: Mon, 13 Jul 2020 22:00:12 -0400 Subject: [PATCH 3/3] Rename Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb to module1-Intro-to-Neural-Networks/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb --- .../Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb => module1-Intro-to-Neural-Networks/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb (99%) diff --git a/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb b/module1-Intro-to-Neural-Networks/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb similarity index 99% rename from Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb rename to module1-Intro-to-Neural-Networks/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb index e1da4005f..eb6df063a 100644 --- a/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb +++ b/module1-Intro-to-Neural-Networks/Kristine_Wang_LS_DS_431_Intro_to_NN_Assignment.ipynb @@ -1986,4 +1986,4 @@ ] } ] -} \ No newline at end of file +}