Day_026_Feature_Combination.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 範例 : 計程車費率預測\n",
    "https://www.kaggle.com/c/new-york-city-taxi-fare-prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [教學目標]\n",
    "- 使用並觀察特徵組合, 在計程車費率預測競賽的影響"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [範例重點]\n",
    "- 增加精度差與緯度差兩個特徵, 觀察線性迴歸與梯度提升樹的預測結果有什麼影響 (In[4], Out[4], In[5], Out[5]) \n",
    "- 再增加座標距離特徵, 觀察線性迴歸與梯度提升樹的預測結果有什麼影響 (In[6], Out[6], In[7], Out[7])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pickup_datetime</th>\n",
       "      <th>pickup_longitude</th>\n",
       "      <th>pickup_latitude</th>\n",
       "      <th>dropoff_longitude</th>\n",
       "      <th>dropoff_latitude</th>\n",
       "      <th>passenger_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2011-10-21 23:54:10 UTC</td>\n",
       "      <td>-73.990580</td>\n",
       "      <td>40.761071</td>\n",
       "      <td>-73.981128</td>\n",
       "      <td>40.758634</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-02-03 10:42:03 UTC</td>\n",
       "      <td>-73.988403</td>\n",
       "      <td>40.723431</td>\n",
       "      <td>-73.989647</td>\n",
       "      <td>40.741695</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2014-03-16 18:58:58 UTC</td>\n",
       "      <td>-74.015785</td>\n",
       "      <td>40.715110</td>\n",
       "      <td>-74.012029</td>\n",
       "      <td>40.707888</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2009-06-13 16:10:54 UTC</td>\n",
       "      <td>-73.977322</td>\n",
       "      <td>40.787275</td>\n",
       "      <td>-73.958030</td>\n",
       "      <td>40.778838</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2014-06-12 03:25:56 UTC</td>\n",
       "      <td>-73.989683</td>\n",
       "      <td>40.729717</td>\n",
       "      <td>-73.982490</td>\n",
       "      <td>40.761887</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           pickup_datetime  pickup_longitude  pickup_latitude  \\\n",
       "0  2011-10-21 23:54:10 UTC        -73.990580        40.761071   \n",
       "1  2015-02-03 10:42:03 UTC        -73.988403        40.723431   \n",
       "2  2014-03-16 18:58:58 UTC        -74.015785        40.715110   \n",
       "3  2009-06-13 16:10:54 UTC        -73.977322        40.787275   \n",
       "4  2014-06-12 03:25:56 UTC        -73.989683        40.729717   \n",
       "\n",
       "   dropoff_longitude  dropoff_latitude  passenger_count  \n",
       "0         -73.981128         40.758634                2  \n",
       "1         -73.989647         40.741695                1  \n",
       "2         -74.012029         40.707888                2  \n",
       "3         -73.958030         40.778838                3  \n",
       "4         -73.982490         40.761887                3  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 做完特徵工程前的所有準備\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import datetime\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "\n",
    "data_path = 'data/'\n",
    "df = pd.read_csv(data_path + 'taxi_data1.csv')\n",
    "\n",
    "train_Y = df['fare_amount']\n",
    "df = df.drop(['fare_amount'] , axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pickup_datetime</th>\n",
       "      <th>pickup_longitude</th>\n",
       "      <th>pickup_latitude</th>\n",
       "      <th>dropoff_longitude</th>\n",
       "      <th>dropoff_latitude</th>\n",
       "      <th>passenger_count</th>\n",
       "      <th>pickup_year</th>\n",
       "      <th>pickup_month</th>\n",
       "      <th>pickup_day</th>\n",
       "      <th>pickup_hour</th>\n",
       "      <th>pickup_minute</th>\n",
       "      <th>pickup_second</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2011-10-21 23:54:10</td>\n",
       "      <td>-73.990580</td>\n",
       "      <td>40.761071</td>\n",
       "      <td>-73.981128</td>\n",
       "      <td>40.758634</td>\n",
       "      <td>2</td>\n",
       "      <td>2011</td>\n",
       "      <td>10</td>\n",
       "      <td>21</td>\n",
       "      <td>23</td>\n",
       "      <td>54</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-02-03 10:42:03</td>\n",
       "      <td>-73.988403</td>\n",
       "      <td>40.723431</td>\n",
       "      <td>-73.989647</td>\n",
       "      <td>40.741695</td>\n",
       "      <td>1</td>\n",
       "      <td>2015</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>42</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2014-03-16 18:58:58</td>\n",
       "      <td>-74.015785</td>\n",
       "      <td>40.715110</td>\n",
       "      <td>-74.012029</td>\n",
       "      <td>40.707888</td>\n",
       "      <td>2</td>\n",
       "      <td>2014</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>18</td>\n",
       "      <td>58</td>\n",
       "      <td>58</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2009-06-13 16:10:54</td>\n",
       "      <td>-73.977322</td>\n",
       "      <td>40.787275</td>\n",
       "      <td>-73.958030</td>\n",
       "      <td>40.778838</td>\n",
       "      <td>3</td>\n",
       "      <td>2009</td>\n",
       "      <td>6</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "      <td>10</td>\n",
       "      <td>54</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2014-06-12 03:25:56</td>\n",
       "      <td>-73.989683</td>\n",
       "      <td>40.729717</td>\n",
       "      <td>-73.982490</td>\n",
       "      <td>40.761887</td>\n",
       "      <td>3</td>\n",
       "      <td>2014</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>3</td>\n",
       "      <td>25</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      pickup_datetime  pickup_longitude  pickup_latitude  dropoff_longitude  \\\n",
       "0 2011-10-21 23:54:10        -73.990580        40.761071         -73.981128   \n",
       "1 2015-02-03 10:42:03        -73.988403        40.723431         -73.989647   \n",
       "2 2014-03-16 18:58:58        -74.015785        40.715110         -74.012029   \n",
       "3 2009-06-13 16:10:54        -73.977322        40.787275         -73.958030   \n",
       "4 2014-06-12 03:25:56        -73.989683        40.729717         -73.982490   \n",
       "\n",
       "   dropoff_latitude  passenger_count  pickup_year  pickup_month  pickup_day  \\\n",
       "0         40.758634                2         2011            10          21   \n",
       "1         40.741695                1         2015             2           3   \n",
       "2         40.707888                2         2014             3          16   \n",
       "3         40.778838                3         2009             6          13   \n",
       "4         40.761887                3         2014             6          12   \n",
       "\n",
       "   pickup_hour  pickup_minute  pickup_second  \n",
       "0           23             54             10  \n",
       "1           10             42              3  \n",
       "2           18             58             58  \n",
       "3           16             10             54  \n",
       "4            3             25             56  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 時間特徵分解方式:使用datetime\n",
    "df['pickup_datetime'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strptime(x, '%Y-%m-%d %H:%M:%S UTC'))\n",
    "df['pickup_year'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strftime(x, '%Y')).astype('int64')\n",
    "df['pickup_month'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strftime(x, '%m')).astype('int64')\n",
    "df['pickup_day'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strftime(x, '%d')).astype('int64')\n",
    "df['pickup_hour'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strftime(x, '%H')).astype('int64')\n",
    "df['pickup_minute'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strftime(x, '%M')).astype('int64')\n",
    "df['pickup_second'] = df['pickup_datetime'].apply(lambda x: datetime.datetime.strftime(x, '%S')).astype('int64')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear Reg Score : 0.026876871475640173\n",
      "Gradient Boosting Reg Score : 0.7123111475816869\n"
     ]
    }
   ],
   "source": [
    "# 將結果使用線性迴歸 / 梯度提升樹分別看結果\n",
    "df = df.drop(['pickup_datetime'] , axis=1)\n",
    "scaler = MinMaxScaler()\n",
    "train_X = scaler.fit_transform(df)\n",
    "Linear = LinearRegression()\n",
    "print(f'Linear Reg Score : {cross_val_score(Linear, train_X, train_Y, cv=5).mean()}')\n",
    "GDBT = GradientBoostingRegressor()\n",
    "print(f'Gradient Boosting Reg Score : {cross_val_score(GDBT, train_X, train_Y, cv=5).mean()}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude_diff</th>\n",
       "      <th>latitude_diff</th>\n",
       "      <th>pickup_longitude</th>\n",
       "      <th>pickup_latitude</th>\n",
       "      <th>dropoff_longitude</th>\n",
       "      <th>dropoff_latitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.009452</td>\n",
       "      <td>-0.002437</td>\n",
       "      <td>-73.990580</td>\n",
       "      <td>40.761071</td>\n",
       "      <td>-73.981128</td>\n",
       "      <td>40.758634</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.001244</td>\n",
       "      <td>0.018265</td>\n",
       "      <td>-73.988403</td>\n",
       "      <td>40.723431</td>\n",
       "      <td>-73.989647</td>\n",
       "      <td>40.741695</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.003756</td>\n",
       "      <td>-0.007222</td>\n",
       "      <td>-74.015785</td>\n",
       "      <td>40.715110</td>\n",
       "      <td>-74.012029</td>\n",
       "      <td>40.707888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.019292</td>\n",
       "      <td>-0.008437</td>\n",
       "      <td>-73.977322</td>\n",
       "      <td>40.787275</td>\n",
       "      <td>-73.958030</td>\n",
       "      <td>40.778838</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.007193</td>\n",
       "      <td>0.032170</td>\n",
       "      <td>-73.989683</td>\n",
       "      <td>40.729717</td>\n",
       "      <td>-73.982490</td>\n",
       "      <td>40.761887</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   longitude_diff  latitude_diff  pickup_longitude  pickup_latitude  \\\n",
       "0        0.009452      -0.002437        -73.990580        40.761071   \n",
       "1       -0.001244       0.018265        -73.988403        40.723431   \n",
       "2        0.003756      -0.007222        -74.015785        40.715110   \n",
       "3        0.019292      -0.008437        -73.977322        40.787275   \n",
       "4        0.007193       0.032170        -73.989683        40.729717   \n",
       "\n",
       "   dropoff_longitude  dropoff_latitude  \n",
       "0         -73.981128         40.758634  \n",
       "1         -73.989647         40.741695  \n",
       "2         -74.012029         40.707888  \n",
       "3         -73.958030         40.778838  \n",
       "4         -73.982490         40.761887  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 增加緯度差, 經度差兩個特徵\n",
    "df['longitude_diff'] = df['dropoff_longitude'] - df['pickup_longitude']\n",
    "df['latitude_diff'] = df['dropoff_latitude'] - df['pickup_latitude']\n",
    "df[['longitude_diff', 'latitude_diff', 'pickup_longitude', 'pickup_latitude', 'dropoff_longitude', 'dropoff_latitude']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear Reg Score : 0.026777745780496964\n",
      "Gradient Boosting Reg Score : 0.7973827697520066\n"
     ]
    }
   ],
   "source": [
    "# 結果 : 光是用經緯度差, 準確度就有巨幅上升\n",
    "train_X = scaler.fit_transform(df)\n",
    "print(f'Linear Reg Score : {cross_val_score(Linear, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'Gradient Boosting Reg Score : {cross_val_score(GDBT, train_X, train_Y, cv=5).mean()}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>distance_2D</th>\n",
       "      <th>longitude_diff</th>\n",
       "      <th>latitude_diff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.009761</td>\n",
       "      <td>0.009452</td>\n",
       "      <td>-0.002437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.018307</td>\n",
       "      <td>-0.001244</td>\n",
       "      <td>0.018265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.008140</td>\n",
       "      <td>0.003756</td>\n",
       "      <td>-0.007222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.021056</td>\n",
       "      <td>0.019292</td>\n",
       "      <td>-0.008437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.032964</td>\n",
       "      <td>0.007193</td>\n",
       "      <td>0.032170</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   distance_2D  longitude_diff  latitude_diff\n",
       "0     0.009761        0.009452      -0.002437\n",
       "1     0.018307       -0.001244       0.018265\n",
       "2     0.008140        0.003756      -0.007222\n",
       "3     0.021056        0.019292      -0.008437\n",
       "4     0.032964        0.007193       0.032170"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 增加座標距離特徵\n",
    "df['distance_2D'] = (df['longitude_diff']**2 + df['latitude_diff']**2)**0.5\n",
    "df[['distance_2D', 'longitude_diff', 'latitude_diff']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear Reg Score : 0.027473522287347694\n",
      "Gradient Boosting Reg Score : 0.8060327194920864\n"
     ]
    }
   ],
   "source": [
    "# 結果 : 加上座標距離後, 準確度再度上升(包含線性迴歸)\n",
    "train_X = scaler.fit_transform(df)\n",
    "print(f'Linear Reg Score : {cross_val_score(Linear, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'Gradient Boosting Reg Score : {cross_val_score(GDBT, train_X, train_Y, cv=5).mean()}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業1\n",
    "* 參考今日教材，試著使用經緯度一圈的長度比這一概念，組合出一個新特徵，再觀察原特徵加上新特徵是否提升了正確率?\n",
    "\n",
    "# 作業2\n",
    "* 試著只使用新特徵估計目標值(忽略原特徵)，效果跟作業1的結果比較起來效果如何?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}