Machine Learning Titanic Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fbb8457a",
   "metadata": {
    "papermill": {
     "duration": 0.008748,
     "end_time": "2023-04-07T23:16:06.331476",
     "exception": false,
     "start_time": "2023-04-07T23:16:06.322728",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Titanic Machine Learning Project\n",
    "\n",
    "The sinking of the Titanic in 1912 was a tragedy that resulted in the loss of many lives. The collision with an iceberg sank the \"unsinkable\" ship and resulted in the death of 1502 out of 2224 passengers and crew. In this project I am tasked with analyzing available data on the passengers, such as their name, age, gender, socio-economic class, and fare to potentially provide insights into the factors that influenced survival. For this project I will be utilizing the power of machine learning, Random Forest Classifier and XGBoost algorithms, to predict what sort of people were more likely to survive. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90b1c900",
   "metadata": {
    "papermill": {
     "duration": 0.006767,
     "end_time": "2023-04-07T23:16:06.345721",
     "exception": false,
     "start_time": "2023-04-07T23:16:06.338954",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Loading The Data\n",
    "In this project, I am given 2 datasets with passenger information. The dataset **train.csv** comprises of data on 891 passengers and includes the target variable \"Survived\" which indicates whether or not a passenger survived. The dataset **test.csv** on the otherhand contains data on a seperate 418 passengers, but does not contain a variable that indicates the fate of the passenger. I will analyze and use Machine Learning techniques on the **train.csv** data in order to create a model. This model will then be used to predict the outcome of those passengers in the **test.csv** dataset. This project is a \"competition project\" on the Kaggle site, so I will never know what passengers in the **test.csv** dataset survived, instead I will submit my guess on which 418 passengers survived using my model and will recieve a score based on how accurate I was."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a398be3",
   "metadata": {
    "papermill": {
     "duration": 0.006915,
     "end_time": "2023-04-07T23:16:06.359829",
     "exception": false,
     "start_time": "2023-04-07T23:16:06.352914",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "I will begin by importing packages that I will use throughout this project. I will also import the datasets from the Kaggle site."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4c7edb31",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:06.376802Z",
     "iopub.status.busy": "2023-04-07T23:16:06.376066Z",
     "iopub.status.idle": "2023-04-07T23:16:07.738347Z",
     "shell.execute_reply": "2023-04-07T23:16:07.737291Z"
    },
    "papermill": {
     "duration": 1.373669,
     "end_time": "2023-04-07T23:16:07.740856",
     "exception": false,
     "start_time": "2023-04-07T23:16:06.367187",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/titanic/train.csv\n",
      "/kaggle/input/titanic/test.csv\n",
      "/kaggle/input/titanic/gender_submission.csv\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from xgboost import XGBClassifier\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "034a0cd1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:07.757085Z",
     "iopub.status.busy": "2023-04-07T23:16:07.756646Z",
     "iopub.status.idle": "2023-04-07T23:16:07.796030Z",
     "shell.execute_reply": "2023-04-07T23:16:07.794850Z"
    },
    "papermill": {
     "duration": 0.050248,
     "end_time": "2023-04-07T23:16:07.798581",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.748333",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = pd.read_csv(\"/kaggle/input/titanic/train.csv\")\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "380651e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:07.815528Z",
     "iopub.status.busy": "2023-04-07T23:16:07.814749Z",
     "iopub.status.idle": "2023-04-07T23:16:07.838184Z",
     "shell.execute_reply": "2023-04-07T23:16:07.836915Z"
    },
    "papermill": {
     "duration": 0.034402,
     "end_time": "2023-04-07T23:16:07.840611",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.806209",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>female</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>male</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>Wirz, Mr. Albert</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315154</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>Hirvonen, Mrs. Alexander (Helga E Lindqvist)</td>\n",
       "      <td>female</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3101298</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                          Name     Sex  \\\n",
       "0          892       3                              Kelly, Mr. James    male   \n",
       "1          893       3              Wilkes, Mrs. James (Ellen Needs)  female   \n",
       "2          894       2                     Myles, Mr. Thomas Francis    male   \n",
       "3          895       3                              Wirz, Mr. Albert    male   \n",
       "4          896       3  Hirvonen, Mrs. Alexander (Helga E Lindqvist)  female   \n",
       "\n",
       "    Age  SibSp  Parch   Ticket     Fare Cabin Embarked  \n",
       "0  34.5      0      0   330911   7.8292   NaN        Q  \n",
       "1  47.0      1      0   363272   7.0000   NaN        S  \n",
       "2  62.0      0      0   240276   9.6875   NaN        Q  \n",
       "3  27.0      0      0   315154   8.6625   NaN        S  \n",
       "4  22.0      1      1  3101298  12.2875   NaN        S  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data = pd.read_csv(\"/kaggle/input/titanic/test.csv\")\n",
    "test_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "457c7e69",
   "metadata": {
    "papermill": {
     "duration": 0.007536,
     "end_time": "2023-04-07T23:16:07.856257",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.848721",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Here I can see that both datasets contain explanitory variables such as, PassengerId, Pclass, Name, Sex, Age, SibSp, Parch, Ticket, Fare, Cabin, and Embarked. The **train_data** also contains the \"Survived\" variable, which will obviously be used as the target variable. The variable appears to be categorical, that is a 1 if the passenger survived and a 0 if they did not survive.\n",
    "\n",
    "I will now take a closer look at the variables using the descibe method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "791c83af",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:07.874793Z",
     "iopub.status.busy": "2023-04-07T23:16:07.873688Z",
     "iopub.status.idle": "2023-04-07T23:16:07.915123Z",
     "shell.execute_reply": "2023-04-07T23:16:07.913954Z"
    },
    "papermill": {
     "duration": 0.052536,
     "end_time": "2023-04-07T23:16:07.917450",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.864914",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "14b0ccd0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:07.935858Z",
     "iopub.status.busy": "2023-04-07T23:16:07.934757Z",
     "iopub.status.idle": "2023-04-07T23:16:07.963925Z",
     "shell.execute_reply": "2023-04-07T23:16:07.962772Z"
    },
    "papermill": {
     "duration": 0.040752,
     "end_time": "2023-04-07T23:16:07.966263",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.925511",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>418.000000</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>332.000000</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>417.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1100.500000</td>\n",
       "      <td>2.265550</td>\n",
       "      <td>30.272590</td>\n",
       "      <td>0.447368</td>\n",
       "      <td>0.392344</td>\n",
       "      <td>35.627188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>120.810458</td>\n",
       "      <td>0.841838</td>\n",
       "      <td>14.181209</td>\n",
       "      <td>0.896760</td>\n",
       "      <td>0.981429</td>\n",
       "      <td>55.907576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>892.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.170000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>996.250000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.895800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1100.500000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1204.750000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1309.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>76.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId      Pclass         Age       SibSp       Parch        Fare\n",
       "count   418.000000  418.000000  332.000000  418.000000  418.000000  417.000000\n",
       "mean   1100.500000    2.265550   30.272590    0.447368    0.392344   35.627188\n",
       "std     120.810458    0.841838   14.181209    0.896760    0.981429   55.907576\n",
       "min     892.000000    1.000000    0.170000    0.000000    0.000000    0.000000\n",
       "25%     996.250000    1.000000   21.000000    0.000000    0.000000    7.895800\n",
       "50%    1100.500000    3.000000   27.000000    0.000000    0.000000   14.454200\n",
       "75%    1204.750000    3.000000   39.000000    1.000000    0.000000   31.500000\n",
       "max    1309.000000    3.000000   76.000000    8.000000    9.000000  512.329200"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b796bc7",
   "metadata": {
    "papermill": {
     "duration": 0.007893,
     "end_time": "2023-04-07T23:16:07.982282",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.974389",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "There are a couple of things this shows:\n",
    "- The count for the Age is different for both datasets indicating that some passengers are missing an Age in their data. (Also the case for 1 passenger in the **test_data** with the Fare variable.)\n",
    "- Name, Sex, Ticket, Cabin, and Embarked are not included in the result, since they are categorical variables.\n",
    "- In general all the variables tend to have similar numbers across both datasets hopefully indicating that the data was properly/randomly split when separating the **train** and **test** datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "549895f3",
   "metadata": {
    "papermill": {
     "duration": 0.007683,
     "end_time": "2023-04-07T23:16:07.998379",
     "exception": false,
     "start_time": "2023-04-07T23:16:07.990696",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Split the Data\n",
    "\n",
    "Next I will split the **train_data** into 4 groups using the train_test_split method. (Using 80% train, 20% Validation)\n",
    "- X_train: 80% of the data with all variables except the target variable. Will be used to build the model(s).\n",
    "- X_valid: 20% of the data with all variables except the target variable. Will be used to test the accuracy of model(s), and check results of adjustments against the model.\n",
    "- y_train: The same 80% of the data as the X_train set, but only comprised of the target \"Survived\" variable.\n",
    "- y_test: The same 20% of the data as the X_test set, but only comprised of the target \"Survived\" variable.\n",
    "\n",
    "I need to split the data since like I mentioned before I will never know the Survived variable of the **test_data**. Thus I cannot use that data to assess the accuracy of my models and determine what proper adjustments to make against it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "341c30ce",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.016446Z",
     "iopub.status.busy": "2023-04-07T23:16:08.016011Z",
     "iopub.status.idle": "2023-04-07T23:16:08.025815Z",
     "shell.execute_reply": "2023-04-07T23:16:08.024674Z"
    },
    "papermill": {
     "duration": 0.021764,
     "end_time": "2023-04-07T23:16:08.028291",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.006527",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create a copy of the train_data that I will make adjustments against.\n",
    "X = train_data\n",
    "\n",
    "# Separate the target or prediction variable from the explanatory variables.\n",
    "y = X.Survived\n",
    "X.drop(['Survived'], axis=1, inplace=True)\n",
    "\n",
    "# Split the taining data set and validation set from training data\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X, y, train_size=0.8, test_size=0.2,\n",
    "                                                                random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48811a08",
   "metadata": {
    "papermill": {
     "duration": 0.007913,
     "end_time": "2023-04-07T23:16:08.044518",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.036605",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Assess and Clean Variables in the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad0dec19",
   "metadata": {
    "papermill": {
     "duration": 0.008714,
     "end_time": "2023-04-07T23:16:08.061309",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.052595",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "#### Missing Data\n",
    "Next I have to handle the missing data points. There are a couple of approaches to handle missing values in the data. In this case I will be using simple imputation to fill in the missing values. For this project I will be calculating the missing values using the \"constant\" strategy of the SimpleImputer method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2835dee9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.079532Z",
     "iopub.status.busy": "2023-04-07T23:16:08.079120Z",
     "iopub.status.idle": "2023-04-07T23:16:08.085589Z",
     "shell.execute_reply": "2023-04-07T23:16:08.084502Z"
    },
    "papermill": {
     "duration": 0.018206,
     "end_time": "2023-04-07T23:16:08.087750",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.069544",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Select numerical columns to impute against\n",
    "numerical_cols = [cname for cname in X_train.columns if \n",
    "                X_train[cname].dtype in ['int64', 'float64']]\n",
    "\n",
    "# Preprocessing for numerical data\n",
    "numerical_transformer = SimpleImputer(strategy = \"constant\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4fffc65",
   "metadata": {
    "papermill": {
     "duration": 0.007851,
     "end_time": "2023-04-07T23:16:08.103689",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.095838",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "#### Categorical Data\n",
    "As I mentioned earlier, many of the explanatory variables are categorical. There are also a couple of ways to handle categorical variables, but here I will be using One-Hot Encoding which creates new columns in the data indicating the presence (or absence) of each possible value of the categorical data. Since this typically does not perform well with categorical variables with a large number of differing values, I will find the cardinality of the categorical variables. I will then use One-Hot Encoding on those with less than 10 cardinality, and drop the variables with 10 or more cardinality from the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4a1856b7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.121623Z",
     "iopub.status.busy": "2023-04-07T23:16:08.121189Z",
     "iopub.status.idle": "2023-04-07T23:16:08.132851Z",
     "shell.execute_reply": "2023-04-07T23:16:08.131750Z"
    },
    "papermill": {
     "duration": 0.023521,
     "end_time": "2023-04-07T23:16:08.135247",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.111726",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# \"Cardinality\" means the number of unique values in a column\n",
    "# Select categorical columns with relatively low cardinality.\n",
    "categorical_cols = [cname for cname in X_train.columns if\n",
    "                    X_train[cname].nunique() < 10 and \n",
    "                    X_train[cname].dtype == \"object\"]\n",
    "\n",
    "# Using Pipeline to help preprocess the categorical data both Imputing and One-Hot Encoding\n",
    "categorical_transformer = Pipeline(steps=[\n",
    "    (\"imputer\",SimpleImputer(strategy = \"constant\")),\n",
    "    (\"one hot\",OneHotEncoder(handle_unknown = \"ignore\"))])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b16b4a5",
   "metadata": {
    "papermill": {
     "duration": 0.007743,
     "end_time": "2023-04-07T23:16:08.151207",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.143464",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Selecting Varaibles For Model\n",
    "Now based on how I handled the data in the previous section, I will select which columns to use for the model to predict. Keeping only the selected variables for each of the 3 X datasets. (X_train, X_valid, X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f7405521",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.169015Z",
     "iopub.status.busy": "2023-04-07T23:16:08.168556Z",
     "iopub.status.idle": "2023-04-07T23:16:08.178399Z",
     "shell.execute_reply": "2023-04-07T23:16:08.177296Z"
    },
    "papermill": {
     "duration": 0.021564,
     "end_time": "2023-04-07T23:16:08.180749",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.159185",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Keep selected columns only\n",
    "my_cols = categorical_cols + numerical_cols\n",
    "\n",
    "X_train_mycol = X_train[my_cols].copy()\n",
    "X_valid_mycol = X_valid[my_cols].copy()\n",
    "X_test_mycol = test_data[my_cols].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d11e6c9a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.198772Z",
     "iopub.status.busy": "2023-04-07T23:16:08.198352Z",
     "iopub.status.idle": "2023-04-07T23:16:08.212133Z",
     "shell.execute_reply": "2023-04-07T23:16:08.211264Z"
    },
    "papermill": {
     "duration": 0.025479,
     "end_time": "2023-04-07T23:16:08.214464",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.188985",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>495</th>\n",
       "      <td>male</td>\n",
       "      <td>C</td>\n",
       "      <td>496</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14.4583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>648</th>\n",
       "      <td>male</td>\n",
       "      <td>S</td>\n",
       "      <td>649</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>278</th>\n",
       "      <td>male</td>\n",
       "      <td>Q</td>\n",
       "      <td>279</td>\n",
       "      <td>3</td>\n",
       "      <td>7.0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>29.1250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>female</td>\n",
       "      <td>C</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>146.5208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>255</th>\n",
       "      <td>female</td>\n",
       "      <td>C</td>\n",
       "      <td>256</td>\n",
       "      <td>3</td>\n",
       "      <td>29.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>15.2458</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Sex Embarked  PassengerId  Pclass   Age  SibSp  Parch      Fare\n",
       "495    male        C          496       3   NaN      0      0   14.4583\n",
       "648    male        S          649       3   NaN      0      0    7.5500\n",
       "278    male        Q          279       3   7.0      4      1   29.1250\n",
       "31   female        C           32       1   NaN      1      0  146.5208\n",
       "255  female        C          256       3  29.0      0      2   15.2458"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the first 5 rows of the valid data to show which columns I am keeping.\n",
    "X_valid_mycol.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e689095e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.233185Z",
     "iopub.status.busy": "2023-04-07T23:16:08.232497Z",
     "iopub.status.idle": "2023-04-07T23:16:08.237619Z",
     "shell.execute_reply": "2023-04-07T23:16:08.236626Z"
    },
    "papermill": {
     "duration": 0.016755,
     "end_time": "2023-04-07T23:16:08.239787",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.223032",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Bundle preprocessing for numerical and categorical data\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('num', numerical_transformer, numerical_cols),\n",
    "        ('cat', categorical_transformer, categorical_cols)\n",
    "    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1eafb39",
   "metadata": {
    "papermill": {
     "duration": 0.008137,
     "end_time": "2023-04-07T23:16:08.256455",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.248318",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Creating The Models\n",
    "For this project I will using both Random Forest Classification and XGBoost (gradient boosting) to create models. \n",
    "\n",
    "The Random Forest Classifier works by constructing a collection of decision trees. Data is inputted into each decision tree in the forest, and each tree independently makes a prediction. The final prediction is then determined by combining the predictions of all the trees, then deciding the outcome by majority voting.\n",
    "\n",
    "Gradient boosting works by going through cycles to iteratively add models into an ensemble. It begins by initializing the ensemble with a single model, whose predictions can be pretty naive. It then iteratively trains new models to correct the errors made by previous models in the ensemble. The errors or residuals from the previous models are used as targets for the next model to improve upon. This process continues iteratively, with each new model trying to minimize the residual errors of the ensemble.\n",
    "\n",
    "I will create 5 models for using each technique with differing parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e16a1467",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.274963Z",
     "iopub.status.busy": "2023-04-07T23:16:08.274563Z",
     "iopub.status.idle": "2023-04-07T23:16:08.282552Z",
     "shell.execute_reply": "2023-04-07T23:16:08.281420Z"
    },
    "papermill": {
     "duration": 0.019942,
     "end_time": "2023-04-07T23:16:08.284693",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.264751",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the Random Forsest models for different n_estimators.\n",
    "model_RF1 = RandomForestClassifier(n_estimators = 100, random_state = 0)\n",
    "model_RF2 = RandomForestClassifier(n_estimators = 200, random_state = 0)\n",
    "model_RF3 = RandomForestClassifier(n_estimators = 300, random_state = 0)\n",
    "model_RF4 = RandomForestClassifier(n_estimators = 400,  random_state = 0)\n",
    "model_RF5 = RandomForestClassifier(n_estimators = 500, random_state = 0)\n",
    "\n",
    "# Define the XGBoost models for different n_estimators.\n",
    "model_XGB1 = XGBClassifier(n_estimators = 100, learning_rate = 0.05, random_state = 0)\n",
    "model_XGB2 = XGBClassifier(n_estimators = 200, learning_rate = 0.05, random_state = 0)\n",
    "model_XGB3 = XGBClassifier(n_estimators = 300, learning_rate = 0.05, random_state = 0)\n",
    "model_XGB4 = XGBClassifier(n_estimators = 400, learning_rate = 0.05, random_state = 0)\n",
    "model_XGB5 = XGBClassifier(n_estimators = 500, learning_rate = 0.05, random_state = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4b1f321",
   "metadata": {
    "papermill": {
     "duration": 0.00821,
     "end_time": "2023-04-07T23:16:08.301323",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.293113",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Model Testing\n",
    "Next I will create a score_model function that uses mean absolute error between the predictions based on the X-valid data (calculated using the model trained on the X_train_mycol and y_train data) and the y_valid data.\n",
    "The model that gives the lowest mean absolute error should be considered the best model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8efc8eda",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:08.320027Z",
     "iopub.status.busy": "2023-04-07T23:16:08.319622Z",
     "iopub.status.idle": "2023-04-07T23:16:18.448706Z",
     "shell.execute_reply": "2023-04-07T23:16:18.447488Z"
    },
    "papermill": {
     "duration": 10.14213,
     "end_time": "2023-04-07T23:16:18.451814",
     "exception": false,
     "start_time": "2023-04-07T23:16:08.309684",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model 1 MAE: 0.13966480446927373\n",
      "Model 2 MAE: 0.13966480446927373\n",
      "Model 3 MAE: 0.1340782122905028\n",
      "Model 4 MAE: 0.12849162011173185\n",
      "Model 5 MAE: 0.12849162011173185\n",
      "Model 6 MAE: 0.16201117318435754\n",
      "Model 7 MAE: 0.1452513966480447\n",
      "Model 8 MAE: 0.13966480446927373\n",
      "Model 9 MAE: 0.13966480446927373\n",
      "Model 10 MAE: 0.1452513966480447\n"
     ]
    }
   ],
   "source": [
    "def score_model(model, X_t=X_train_mycol, X_v=X_valid_mycol, y_t=y_train, y_v=y_valid):\n",
    "    my_pipeline = Pipeline(steps=[('preprocessor', preprocessor),\n",
    "                                  ('model', model)\n",
    "                                 ])\n",
    "    my_pipeline.fit(X_t, y_t)\n",
    "    preds = my_pipeline.predict(X_v)\n",
    "    return mean_absolute_error(y_v, preds)\n",
    "\n",
    "#create set of all the combined models and then iterate through them calculating their mean absolute error.\n",
    "models = [model_RF1, model_RF2, model_RF3, model_RF4, model_RF5, model_XGB1, model_XGB2, model_XGB3, model_XGB4, model_XGB5]\n",
    "for i in range(0, len(models)):\n",
    "    mae = score_model(models[i])\n",
    "    print(\"Model \" + str(i+1) + \" MAE: \" + str(mae))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26fe84b2",
   "metadata": {
    "papermill": {
     "duration": 0.011158,
     "end_time": "2023-04-07T23:16:18.477403",
     "exception": false,
     "start_time": "2023-04-07T23:16:18.466245",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "It appears **model_RF4** should be considered the best and final model as it is tied for having the lowest mean absolute error with more complex models. When breaking ties between models it is usually best to favor the least complex model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "faf600b4",
   "metadata": {
    "papermill": {
     "duration": 0.008813,
     "end_time": "2023-04-07T23:16:18.495583",
     "exception": false,
     "start_time": "2023-04-07T23:16:18.486770",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Creating and Fitting the Final Model\n",
    "I will now create the final model, once again calculate its mean absolute value under its new final model name, and then use the **X_test** data to predict the survival fate of all the passengers in the **X_test** dataset. An **output** dataset consisting of the PassengerIds and predicted Survial values will be created and submitted to the Kaggle compition.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "106fc1ee",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:18.515837Z",
     "iopub.status.busy": "2023-04-07T23:16:18.515411Z",
     "iopub.status.idle": "2023-04-07T23:16:18.520770Z",
     "shell.execute_reply": "2023-04-07T23:16:18.519810Z"
    },
    "papermill": {
     "duration": 0.017891,
     "end_time": "2023-04-07T23:16:18.522994",
     "exception": false,
     "start_time": "2023-04-07T23:16:18.505103",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Creating the Final Model.\n",
    "model_final = RandomForestClassifier(n_estimators = 400,  random_state = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "7c832267",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:18.542944Z",
     "iopub.status.busy": "2023-04-07T23:16:18.542527Z",
     "iopub.status.idle": "2023-04-07T23:16:19.416144Z",
     "shell.execute_reply": "2023-04-07T23:16:19.415073Z"
    },
    "papermill": {
     "duration": 0.886395,
     "end_time": "2023-04-07T23:16:19.418813",
     "exception": false,
     "start_time": "2023-04-07T23:16:18.532418",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE: 0.12849162011173185\n"
     ]
    }
   ],
   "source": [
    "# Bundle preprocessing and modeling code in a pipeline\n",
    "my_pipeline = Pipeline(steps=[('preprocessor', preprocessor),\n",
    "                              ('model', model_final)])\n",
    "\n",
    "# Preprocessing of training data, fit model \n",
    "my_pipeline.fit(X_train_mycol, y_train)\n",
    "\n",
    "# Preprocessing of validation data, get predictions\n",
    "preds = my_pipeline.predict(X_valid_mycol)\n",
    "\n",
    "# Evaluating the model under the new \"final model\" name\n",
    "score = mean_absolute_error(y_valid, preds)\n",
    "print('MAE:', score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e7c0d2c7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:19.439107Z",
     "iopub.status.busy": "2023-04-07T23:16:19.438682Z",
     "iopub.status.idle": "2023-04-07T23:16:19.527158Z",
     "shell.execute_reply": "2023-04-07T23:16:19.525928Z"
    },
    "papermill": {
     "duration": 0.101953,
     "end_time": "2023-04-07T23:16:19.530181",
     "exception": false,
     "start_time": "2023-04-07T23:16:19.428228",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Preprocessing of test data, fit model using the X_test_mycol dataset\n",
    "preds_test = my_pipeline.predict(X_test_mycol)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "d1239da4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:19.551091Z",
     "iopub.status.busy": "2023-04-07T23:16:19.550000Z",
     "iopub.status.idle": "2023-04-07T23:16:19.561600Z",
     "shell.execute_reply": "2023-04-07T23:16:19.560428Z"
    },
    "papermill": {
     "duration": 0.02426,
     "end_time": "2023-04-07T23:16:19.563844",
     "exception": false,
     "start_time": "2023-04-07T23:16:19.539584",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived\n",
       "0          892         0\n",
       "1          893         0\n",
       "2          894         0\n",
       "3          895         0\n",
       "4          896         0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Create the output dataset and take a quick look at it.\n",
    "output = pd.DataFrame({'PassengerId': test_data.PassengerId, 'Survived': preds_test})\n",
    "output.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "ff766c51",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-07T23:16:19.584715Z",
     "iopub.status.busy": "2023-04-07T23:16:19.584321Z",
     "iopub.status.idle": "2023-04-07T23:16:19.594390Z",
     "shell.execute_reply": "2023-04-07T23:16:19.592813Z"
    },
    "papermill": {
     "duration": 0.023384,
     "end_time": "2023-04-07T23:16:19.596908",
     "exception": false,
     "start_time": "2023-04-07T23:16:19.573524",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Submission was successfully saved!\n"
     ]
    }
   ],
   "source": [
    "#Submit the output dataset.\n",
    "output.to_csv('submission.csv', index=False)\n",
    "print(\"Submission was successfully saved!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2d6028c",
   "metadata": {
    "papermill": {
     "duration": 0.008985,
     "end_time": "2023-04-07T23:16:19.615440",
     "exception": false,
     "start_time": "2023-04-07T23:16:19.606455",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Result\n",
    "In conclusion, after submitting the output, the result is that the final model was able to correctly identify the fate of 75.94% of the passengers within the **test** dataset."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 23.344397,
   "end_time": "2023-04-07T23:16:20.448617",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-04-07T23:15:57.104220",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}