Common Problems in Machine Learning .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9f1f4079",
   "metadata": {},
   "source": [
    "# Common Problems in Machine Learning\n",
    "\n",
    "There are a lot of challenges that machine learning professionals face to inculcate ML skills and create an application from scratch.\n",
    "\n",
    "\n",
    "some are:\n",
    "\n",
    "1. Poor Data Quality or Imbalanced Dataset\n",
    "2. Underfitting of Training Data\n",
    "3. Overfitting of Training Data\n",
    "4. Feature Selection and Dimensionality Reduction\n",
    "5. Data Leakage"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f516625",
   "metadata": {},
   "source": [
    "## Overfitting\n",
    "\n",
    "Overfitting occurs when a machine learning model learns the training data too well and fails to generalize to new, unseen data. \n",
    "\n",
    "The model becomes overly complex and captures noise or random fluctuations in the training data, resulting in poor performance on the test data.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b87fe30e",
   "metadata": {},
   "source": [
    "<img src=\"https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fdiegolosey.com%2Fwp-content%2Fuploads%2F2020%2F07%2Funderfitting-1536x477.png&f=1&nofb=1&ipt=d2421880db5ffa5dffe78180f78f7c043b560f614fbd375ff4d52e73df02c169&ipo=images\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c85aa7b",
   "metadata": {},
   "source": [
    "## Underfitting \n",
    "\n",
    "happens when a machine learning model is too simple to capture the underlying patterns and relationships in the training data. \n",
    "\n",
    "It fails to learn the essential characteristics of the data, leading to poor performance on both the training and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a3c5c700",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error: 3.2869204384208827e-31\n"
     ]
    }
   ],
   "source": [
    "# Example of underfitting\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "# Generate a linear dataset\n",
    "X = [[1], [2], [3]]\n",
    "y = [2, 4, 6]\n",
    "\n",
    "# Fit a linear regression model\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "\n",
    "# Make predictions\n",
    "y_pred = model.predict(X)\n",
    "\n",
    "# Calculate mean squared error\n",
    "mse = mean_squared_error(y, y_pred)\n",
    "print(\"Mean Squared Error:\", mse)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d568901a",
   "metadata": {},
   "source": [
    "# How to Detect Overfitting in Machine Learning\n",
    "\n",
    "A key challenge with overfitting, and with machine learning in general, is that we can’t know how well our model will perform on new data until we actually test it.\n",
    "\n",
    "To address this, we can split our initial dataset into separate training and test subsets.\n",
    "\n",
    "<img src=\"https://elitedatascience.com/wp-content/uploads/2017/06/Train-Test-Split-Diagram-768x266.jpg\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4c1946b",
   "metadata": {},
   "source": [
    "> If our model does much better on the training set than on the test set, then we’re likely overfitting.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69cc4d04",
   "metadata": {},
   "source": [
    "Further reading:\n",
    "    https://elitedatascience.com/overfitting-in-machine-learning\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33e4e21c",
   "metadata": {},
   "source": [
    "### Overcome Overfitting with Cross-validation\n",
    "Cross-validation is a powerful preventative measure against overfitting.\n",
    "\n",
    "The idea is clever: Use your initial training data to generate multiple mini train-test splits. Use these splits to tune your model.\n",
    "\n",
    "In standard k-fold cross-validation, we partition the data into k subsets, called folds. Then, we iteratively train the algorithm on k-1 folds while using the remaining fold as the test set (called the “holdout fold”).\n",
    "\n",
    "\n",
    "<img src=\"https://elitedatascience.com/wp-content/uploads/2017/06/Cross-Validation-Diagram-768x295.jpg\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "87c935f9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-Validation Scores: [0.96666667 1.         0.93333333 0.96666667 1.        ]\n",
      "Mean Score: 0.9733333333333334\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/maladeep/miniforge3/envs/jupyter/lib/python3.10/site-packages/sklearn/linear_model/_logistic.py:458: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.datasets import load_iris\n",
    "\n",
    "# Load the iris dataset\n",
    "data = load_iris()\n",
    "X, y = data.data, data.target\n",
    "\n",
    "# Create a logistic regression model\n",
    "model = LogisticRegression()\n",
    "\n",
    "# Perform cross-validation\n",
    "cv_scores = cross_val_score(model, X, y, cv=5)\n",
    "\n",
    "# Print the cross-validation scores\n",
    "print(\"Cross-Validation Scores:\", cv_scores)\n",
    "print(\"Mean Score:\", cv_scores.mean())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a8a1aa",
   "metadata": {},
   "source": [
    "In the above code:\n",
    "\n",
    "* The iris dataset is loaded using load_iris() function, and the input features are stored in X, while the target variable is stored in y.\n",
    "* We create a LogisticRegression model object.\n",
    "* cross_val_score function is used to perform cross-validation. We pass the model (model), input features (X), and target variable (y) as arguments. The cv parameter is set to 5, which indicates 5-fold cross-validation.\n",
    "* The cross-validation scores are stored in cv_scores.\n",
    "* Finally, we print the cross-validation scores and the mean score across all folds.\n",
    "\n",
    "\n",
    "The cross-validation scores are displayed as an array of values, with each value representing the model's accuracy on a specific fold of the data. In this case, there are five folds, and the model achieved accuracy values of approximately 0.97, 1.0, 0.93, 0.97, and 1.0 on each respective fold.\n",
    "\n",
    "The mean score is calculated by taking the average of all the cross-validation scores. In this case, the mean score is approximately 0.973, indicating an overall high accuracy of the logistic regression model across all folds."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8307d211",
   "metadata": {},
   "source": [
    "> A cross-validation score of 0.973 implies that the model is able to generalize well to unseen data and perform consistently across multiple subsets of the dataset. This suggests that the model is not overfitting the training data and is capturing meaningful patterns that are applicable to new data points."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17690817",
   "metadata": {},
   "source": [
    "## Feature Selection and Dimensionality Reducing\n",
    "\n",
    "\n",
    "Feature selection and dimensionality reduction techniques are used to identify and retain the most informative features in the dataset. \n",
    "\n",
    "Having too many irrelevant or redundant features can negatively impact model performance, increase training time, and introduce noise into the model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb729b82",
   "metadata": {},
   "source": [
    "## Data Leakage\n",
    " \n",
    " Data leakage occurs when information from the test set leaks into the training process, leading to overly optimistic performance estimates.\n",
    " \n",
    " It can happen when features that are not available in a real-world scenario are used during training.\n",
    " \n",
    " **To overcome you can**\n",
    "\n",
    "* split your dataset into training and test sets before any preprocessing steps or feature engineering. This means performing the train-test split on the raw, unprocessed data. \n",
    " \n",
    "* This helps prevent any information or knowledge from the test set influencing the preprocessing steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2b658e65",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Split the data into training and test sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "# Apply preprocessing steps only on the training data\n",
    "# ...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1765e65c",
   "metadata": {},
   "source": [
    "Image Credit: elitedatascience"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7739205",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}