DataMining_week08.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DataMining-week08.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "pL9x2mNnU_FZ",
        "qZ88PrdBexT6",
        "YNl67BhPehC1"
      ],
      "toc_visible": true,
      "authorship_tag": "ABX9TyPrCysbDKtBW7ieo5L6DTYw",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MLRG-CEFET-RJ/ml-class/blob/master/DataMining_week08.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aqShYNI4OXMH",
        "colab_type": "text"
      },
      "source": [
        "# Data preprocessing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "um4X8U8pXrA1",
        "colab_type": "text"
      },
      "source": [
        "One important part of every Data Mining project is to pre-process the data to be used. In this section, we describe some ways of doing data preprocessing through some Python packages."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9vakDJYlmlVW",
        "colab_type": "text"
      },
      "source": [
        "## Features: discrete vs. quantitative\n",
        "\n",
        "Features are divided into two major types: discrete and quantitative. The feature type defines the applicable analysis and pre-processing techniques.\n",
        "\n",
        "- Quantitative features are those associated with a virtually infinite number of possible values.\n",
        "\n",
        "- Discrete features can only take on one value within a specified set of values.\n",
        "\n",
        "For discrete (or categorical) features, the arithmetic rules do not apply. This family is subdivided into *nominal* and *ordinal* features. \n",
        "\n",
        " - Nominal data are categorical values ​​that have 2 or more possible values, but in which the order of these values ​​is meaningless. For example, we can use a numerical representation to interpret clients' marital status. It is simply a numerical representation of the category. Dichotomous data is also categorical, but can only assume two values. For example, we can represent the presence or absence of a given symptom by the values ​​0 (meaning absence) and 1 (meaning presence).\n",
        "\n",
        "- Ordinal data is similar to nominal data, as it has 2 or more possible values. The main difference is that these values ​​have an associated order or rank. For example, the size of a tumor could take on values ​​1, 2, 3, 4 and 5, with higher values ​​meaning larger tumors.\n",
        "\n",
        "The quantitative family subdivides into  *intervals* and *ratios*. The description below (taken from [here](https://www.graphpad.com/guides/prism/7/statistics/the_different_kinds_of_variabl.htm)) explains how they differ.\n",
        "\n",
        "- An interval features is a one where the difference between two values is meaningful. The difference between a temperature of 100 degrees and 90 degrees is the same difference as between 90 degrees and 80 degrees.\n",
        "\n",
        "- A ratio features, has all the properties of an interval variable, but also has a clear definition of 0.0. When the features equals 0.0, there is none of that variable. Features like height, weight, enzyme activity are ratio variables. Temperature, expressed in F or C, is not a ratio variable. A temperature of 0.0 on either of those scales does not mean 'no heat. However, temperature in Kelvin is a ratio variable, as 0.0 Kelvin really does mean 'no heat'. When working with ratio features, but not interval features, you can look at the ratio of two measurements. A weight of 4 grams is twice a weight of 2 grams, because weight is a ratio variable. A temperature of 100 degrees C is not twice as hot as 50 degrees C, because temperature C is not a ratio features.\n",
        "\n",
        "The picture below (taken from [here](https://www.graphpad.com/support/faq/what-is-the-difference-between-ordinal-interval-and-ratio-variables-why-should-i-care/) summarizes the several attribute types:\n",
        "\n",
        "![alt text](https://lh5.googleusercontent.com/7jyxzQ2ObysJGLFcGB6Zc25AHAswexk68SbOh_KYa4if2P9yRe7lIC8NDUgZEcSGspqpRIGQcMx_qCmrG6sjHegFHy9Sqhp_1z3PFido6d19TKYFq0pMTHDs4OV9l6pP-MTNmeKu)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Inab1T-v7vFz",
        "colab_type": "text"
      },
      "source": [
        "## Feature scaling\n",
        "\n",
        "In general, continuous features (aka, variables or attributes) in a dataset can assume varying ranges of values.\n",
        "\n",
        "From wikipedia:\n",
        "\n",
        "> Feature scaling is a method used to normalize the range of independent variables or features of data. In data processing, it is also known as data normalization and is generally performed during the data preprocessing step.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xCUio5jJajse",
        "colab_type": "text"
      },
      "source": [
        "### Why is scaling needed?\n",
        "\n",
        "The difference in ranges between features in a dataset poses a problem for many popular Data Mining algorithms. Many of these algorithms produce higher quality models or even converge faster (during training) when the attributes are in a relatively similar range of values.\n",
        "\n",
        "The following piece of code (using a simplified version of the [Wine dataset](https://archive.ics.uci.edu/ml/datasets/wine)) illustrates why scaling is needed in some situations. Notice that both `Alcohol` and `Malic acid` are presented in different units, and only because of this, they have different orders of magnitude. For some Data Mining methods, it is desirable that the features are in the same scale."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E4pAXW9GakQL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "1fbe5e1d-f12f-43ed-d8db-4124d78ccc60"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "df = pd.io.parsers.read_csv(\n",
        "    'https://raw.githubusercontent.com/rasbt/pattern_classification/master/data/wine_data.csv',\n",
        "     header=None,\n",
        "     usecols=[0,1,2]\n",
        "    )\n",
        "\n",
        "df.columns=['Class label', 'Alcohol', 'Malic acid']\n",
        "\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Class label</th>\n",
              "      <th>Alcohol</th>\n",
              "      <th>Malic acid</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>14.23</td>\n",
              "      <td>1.71</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>13.20</td>\n",
              "      <td>1.78</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>13.16</td>\n",
              "      <td>2.36</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>14.37</td>\n",
              "      <td>1.95</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>13.24</td>\n",
              "      <td>2.59</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Class label  Alcohol  Malic acid\n",
              "0            1    14.23        1.71\n",
              "1            1    13.20        1.78\n",
              "2            1    13.16        2.36\n",
              "3            1    14.37        1.95\n",
              "4            1    13.24        2.59"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "njTKHcIwKT6m",
        "colab_type": "text"
      },
      "source": [
        "### Scikit-Learn: MinMaxScaler and MaxAbsScaler\n",
        "\n",
        "Scikit-Learn offers transformers (classes) for scaling attributes in a dataset. Among them, we have MaxAbsScaler and MinMaxScaler.\n",
        "\n",
        "- [MinMaxScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html): allows you to dimension the features so that they are between a minimum and maximum value provided, usually between zero and one.\n",
        "\n",
        "- [MaxAbsScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.html): allows the maximum absolute value of each feature to be dimensioned to be the unit.\n",
        "\n",
        "Both of the above scalers have a `feature_range` parameter that allows you to change the range if you don't want 0 to 1 for some reason. If the `feature_range` parameter is not provided, then these transformations are mathematically by the following expressions.\n",
        "\n",
        "min-max scaling: \n",
        "$$\n",
        "x^\\prime= \\frac{x-{\\text{min}}(x)}{{\\text{max}(x)}-{\\text{min}}(x)}\n",
        "$$\n",
        "\n",
        "max-abs scaling: \n",
        "$$\n",
        "x^\\prime= \\frac{x}{{\\text{max}(x)}}\n",
        "$$\n",
        "\n",
        "The image below (taken from the book [Mastering Feature Engineering](https://github.com/changwookjun/StudyBook/blob/master/MachineLearningBooks/Mastering%20Feature%20Engineering.pdf)) illustrates the transformation corresponding to min-max scaling to produce a feature $\\tilde{x}$ from a feature $x$.\n",
        "\n",
        "![alt text](https://miro.medium.com/max/700/1*fnI3SnE9UqupmjQVjqLHeQ.png)\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OVFo7bCTgKmH",
        "colab_type": "text"
      },
      "source": [
        "The two cells below are example of how the MinMaxScaler and MaxAbsScaler classes can be applied."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9M-KMvwZ7H-w",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "outputId": "49e34d3b-dc7f-48e8-d72d-33e3b2e13066"
      },
      "source": [
        "import pandas as pd\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "\n",
        "scaler = MinMaxScaler()\n",
        "\n",
        "df = pd.DataFrame({'A':[14.00,90.20,90.95,96.27,91.21],\n",
        "                   'B':[103.02,107.26,110.35,114.23,114.68],\n",
        "                   'C':['big','small','big','small','small']})\n",
        "\n",
        "print('Original features:\\n', df)\n",
        "columns = ['A', 'B']\n",
        "df[columns] = scaler.fit_transform(df[columns])\n",
        "\n",
        "print('\\nScaled features:\\n', df)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original features:\n",
            "        A       B      C\n",
            "0  14.00  103.02    big\n",
            "1  90.20  107.26  small\n",
            "2  90.95  110.35    big\n",
            "3  96.27  114.23  small\n",
            "4  91.21  114.68  small\n",
            "\n",
            "Scaled features:\n",
            "           A         B      C\n",
            "0  0.000000  0.000000    big\n",
            "1  0.926219  0.363636  small\n",
            "2  0.935335  0.628645    big\n",
            "3  1.000000  0.961407  small\n",
            "4  0.938495  1.000000  small\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IHMEOwLDIz9w",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 259
        },
        "outputId": "5a779a1b-bf0d-4540-cf4d-c1fd9a03bfe3"
      },
      "source": [
        "import pandas as pd\n",
        "from sklearn.preprocessing import MaxAbsScaler\n",
        "\n",
        "scaler = MaxAbsScaler()\n",
        "\n",
        "df = pd.DataFrame({'A':[14.00,90.20,90.95,96.27,91.21],\n",
        "                   'B':[103.02,107.26,110.35,114.23,114.68],\n",
        "                   'C':['big','small','big','small','small']})\n",
        "\n",
        "print('Original features:\\n', df)\n",
        "colunas = ['A', 'B']\n",
        "df[colunas] = scaler.fit_transform(df[colunas])\n",
        "\n",
        "print('Scaled features:\\n', df)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original features:\n",
            "        A       B      C\n",
            "0  14.00  103.02    big\n",
            "1  90.20  107.26  small\n",
            "2  90.95  110.35    big\n",
            "3  96.27  114.23  small\n",
            "4  91.21  114.68  small\n",
            "Scaled features:\n",
            "           A         B      C\n",
            "0  0.145424  0.898326    big\n",
            "1  0.936948  0.935298  small\n",
            "2  0.944739  0.962243    big\n",
            "3  1.000000  0.996076  small\n",
            "4  0.947439  1.000000  small\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z2se92H6K1f3",
        "colab_type": "text"
      },
      "source": [
        "### StandardScaler\n",
        "\n",
        "Standardization is another way of scaling features. This particular type of scaling is performed with the [StandardScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html) class from Scikit-Learn.\n",
        "\n",
        "Standardization first subtracts the *average* value (so the scaled values ​​always have zero mean), and then divides by the *standard deviation* so that the distribution result has unit variance.\n",
        "\n",
        "Unlike min-max scaling, standardization does not limit values ​​to a specific range. However, standardization is much less affected by outliers. For example, suppose the age of person is recorded as 100 (by mistake). In this case, min-max scaling would map all other values ​​from 0 to 16 to 0 to 0.16, for example.\n",
        "\n",
        "Mathematically, standardization is defined by the below expression, where $\\mu$ and $sigma$ are the mean and standard deviation os the corresponding feature, respectively:\n",
        "\n",
        "$$\n",
        "x^\\prime= \\frac{x-\\mu}{\\sigma}\n",
        "$$\n",
        "\n",
        "The image below (taken from the book [Mastering Feature Engineering](https://github.com/changwookjun/StudyBook/blob/master/MachineLearningBooks/Mastering%20Feature%20Engineering.pdf)) illustrates the transformation corresponding to min-max scaling to produce a feature $\\tilde{x}$ from a feature $x$.\n",
        "\n",
        "![alt text](https://miro.medium.com/max/700/1*X0umAvSAyhKYCAvz96iSww.png)\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CZRV_mT7MsLh",
        "colab_type": "text"
      },
      "source": [
        "### Case study: scaling features of the Diabetes dataset\n",
        "\n",
        "Let us practice scaling by applying it to the Pima Indias Diabetes Dataset. Details about this dataset can be found [here](https://www.kaggle.com/uciml/pima-indians-diabetes-database/data#)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "959aVV5cO4MS",
        "colab_type": "text"
      },
      "source": [
        "First, we load the dataset and do some EDA on it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rzE0RAsLMygs",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "b10f3d08-3cf4-4122-86d8-6e7b2c4df4f0"
      },
      "source": [
        "import pandas as pd\n",
        "df_diabetes = pd.read_csv('datasets_228_482_diabetes.csv')\n",
        "df_diabetes.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Pregnancies</th>\n",
              "      <th>Glucose</th>\n",
              "      <th>BloodPressure</th>\n",
              "      <th>SkinThickness</th>\n",
              "      <th>Insulin</th>\n",
              "      <th>BMI</th>\n",
              "      <th>DiabetesPedigreeFunction</th>\n",
              "      <th>Age</th>\n",
              "      <th>Outcome</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6</td>\n",
              "      <td>148</td>\n",
              "      <td>72</td>\n",
              "      <td>35</td>\n",
              "      <td>0</td>\n",
              "      <td>33.6</td>\n",
              "      <td>0.627</td>\n",
              "      <td>50</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>85</td>\n",
              "      <td>66</td>\n",
              "      <td>29</td>\n",
              "      <td>0</td>\n",
              "      <td>26.6</td>\n",
              "      <td>0.351</td>\n",
              "      <td>31</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>8</td>\n",
              "      <td>183</td>\n",
              "      <td>64</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>23.3</td>\n",
              "      <td>0.672</td>\n",
              "      <td>32</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>89</td>\n",
              "      <td>66</td>\n",
              "      <td>23</td>\n",
              "      <td>94</td>\n",
              "      <td>28.1</td>\n",
              "      <td>0.167</td>\n",
              "      <td>21</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>137</td>\n",
              "      <td>40</td>\n",
              "      <td>35</td>\n",
              "      <td>168</td>\n",
              "      <td>43.1</td>\n",
              "      <td>2.288</td>\n",
              "      <td>33</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Pregnancies  Glucose  BloodPressure  ...  DiabetesPedigreeFunction  Age  Outcome\n",
              "0            6      148             72  ...                     0.627   50        1\n",
              "1            1       85             66  ...                     0.351   31        0\n",
              "2            8      183             64  ...                     0.672   32        1\n",
              "3            1       89             66  ...                     0.167   21        0\n",
              "4            0      137             40  ...                     2.288   33        1\n",
              "\n",
              "[5 rows x 9 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 164
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ut_fQTR1Nwb0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 294
        },
        "outputId": "868833b7-5eba-47b0-df88-813c43d08ca1"
      },
      "source": [
        "df_diabetes.info()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 768 entries, 0 to 767\n",
            "Data columns (total 9 columns):\n",
            " #   Column                    Non-Null Count  Dtype  \n",
            "---  ------                    --------------  -----  \n",
            " 0   Pregnancies               768 non-null    int64  \n",
            " 1   Glucose                   768 non-null    int64  \n",
            " 2   BloodPressure             768 non-null    int64  \n",
            " 3   SkinThickness             768 non-null    int64  \n",
            " 4   Insulin                   768 non-null    int64  \n",
            " 5   BMI                       768 non-null    float64\n",
            " 6   DiabetesPedigreeFunction  768 non-null    float64\n",
            " 7   Age                       768 non-null    int64  \n",
            " 8   Outcome                   768 non-null    int64  \n",
            "dtypes: float64(2), int64(7)\n",
            "memory usage: 54.1 KB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_1Jll98dN37N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = df_diabetes.drop(columns=['Outcome'])\n",
        "y = df_diabetes['Outcome']"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QmIyefAzN6VO",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "4a285909-8738-4b63-f538-7cd31b3fa2d8"
      },
      "source": [
        "X.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Pregnancies</th>\n",
              "      <th>Glucose</th>\n",
              "      <th>BloodPressure</th>\n",
              "      <th>SkinThickness</th>\n",
              "      <th>Insulin</th>\n",
              "      <th>BMI</th>\n",
              "      <th>DiabetesPedigreeFunction</th>\n",
              "      <th>Age</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6</td>\n",
              "      <td>148</td>\n",
              "      <td>72</td>\n",
              "      <td>35</td>\n",
              "      <td>0</td>\n",
              "      <td>33.6</td>\n",
              "      <td>0.627</td>\n",
              "      <td>50</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>85</td>\n",
              "      <td>66</td>\n",
              "      <td>29</td>\n",
              "      <td>0</td>\n",
              "      <td>26.6</td>\n",
              "      <td>0.351</td>\n",
              "      <td>31</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>8</td>\n",
              "      <td>183</td>\n",
              "      <td>64</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>23.3</td>\n",
              "      <td>0.672</td>\n",
              "      <td>32</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>89</td>\n",
              "      <td>66</td>\n",
              "      <td>23</td>\n",
              "      <td>94</td>\n",
              "      <td>28.1</td>\n",
              "      <td>0.167</td>\n",
              "      <td>21</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>137</td>\n",
              "      <td>40</td>\n",
              "      <td>35</td>\n",
              "      <td>168</td>\n",
              "      <td>43.1</td>\n",
              "      <td>2.288</td>\n",
              "      <td>33</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Pregnancies  Glucose  BloodPressure  ...   BMI  DiabetesPedigreeFunction  Age\n",
              "0            6      148             72  ...  33.6                     0.627   50\n",
              "1            1       85             66  ...  26.6                     0.351   31\n",
              "2            8      183             64  ...  23.3                     0.672   32\n",
              "3            1       89             66  ...  28.1                     0.167   21\n",
              "4            0      137             40  ...  43.1                     2.288   33\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 167
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1pFFt6YgN9Et",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        },
        "outputId": "15f0a991-b4de-4e2b-eab5-e3c4e672264c"
      },
      "source": [
        "y.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    1\n",
              "1    0\n",
              "2    1\n",
              "3    0\n",
              "4    1\n",
              "Name: Outcome, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 168
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PLOi8dtXOARp",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "8f3617dd-997c-4ac8-f0ed-19b1c4283693"
      },
      "source": [
        "X.Age.hist()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f7dc4d58cf8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 169
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAATB0lEQVR4nO3df6xcZ33n8feHGGiai2LS0CvX8a5T4YLSuBhyFYJAq3vJtgSoairRbKIsdWgq949Uha5Xi+GfwnYjpdqGbKt2o3UJbbqlXLKBKFEC7WbdXLGsFKgdUpzEjXCJQ2OldimOwwU2Wofv/jHH5eZy7RnP/TkP75c0mnOec86c5+uZ+cyZ554zTlUhSWrLS1a7A5KkpWe4S1KDDHdJapDhLkkNMtwlqUHrVrsDABdeeGFt3ry573rf/va3Oe+885a/QyvAWtamVmpppQ6wljPZv3//N6rqVQstWxPhvnnzZvbt29d3vZmZGSYnJ5e/QyvAWtamVmpppQ6wljNJ8tTpljksI0kNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhrUN9yT/EiSLyX5mySPJflI135xki8mOZTkU0le1rW/vJs/1C3fvLwlSJLmG+TI/XngrVX1OmAbcFWSK4DfAW6tqlcDx4EbuvVvAI537bd260mSVlDfcK+e2W72pd2tgLcCd3XtdwDv6qa3d/N0y69MkiXrsSSprwzyn3UkOQfYD7wa+EPgPwMPdUfnJNkEfK6qLk3yKHBVVT3dLfs74I1V9Y15j7kT2AkwPj5+2fT0dN9+zM7OMjY29qK2A0dO9N1uuWzdeP7Q2y5Uy6iylrWnlTrAWs5kampqf1VNLLRsoJ8fqKoXgG1J1gN3A69dbKeqag+wB2BiYqIGuSR3oUt3r999/2K7MrTD100Ova2XVK9NrdTSSh1gLcM6q7NlqupZ4EHgTcD6JKc+HC4CjnTTR4BNAN3y84F/WpLeSpIGMsjZMq/qjthJci7ws8BBeiH/7m61HcA93fS93Tzd8r8q/6NWSVpRgwzLbADu6MbdXwLcWVX3JXkcmE7yn4AvA7d3698O/Pckh4BvAtcsQ78lSWfQN9yr6ivA6xdo/xpw+QLt/xf4pSXpnSRpKF6hKkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkN6hvuSTYleTDJ40keS/K+rv3DSY4keaS7vWPONh9McijJE0netpwFSJJ+0LoB1jkJ7Kqqh5O8Atif5IFu2a1V9btzV05yCXAN8NPATwD/K8lPVdULS9lxSdLp9T1yr6pnqurhbvpbwEFg4xk22Q5MV9XzVfUkcAi4fCk6K0kaTKpq8JWTzcDngUuBfwdcDzwH7KN3dH88yR8AD1XVn3Xb3A58rqrumvdYO4GdAOPj45dNT0/33f/s7CxjY2Mvajtw5MTA/V9qWzeeP/S2C9Uyqqxl7WmlDrCWM5mamtpfVRMLLRtkWAaAJGPAp4H3V9VzSW4Dfhuo7v4W4FcGfbyq2gPsAZiYmKjJycm+28zMzDB/vet33z/oLpfc4esmh952oVpGlbWsPa3UAdYyrIHOlknyUnrB/omq+gxAVR2tqheq6nvAH/H9oZcjwKY5m1/UtUmSVsggZ8sEuB04WFUfndO+Yc5qvwg82k3fC1yT5OVJLga2AF9aui5LkvoZZFjmzcB7gANJHunaPgRcm2QbvWGZw8CvAVTVY0nuBB6nd6bNjZ4pI0krq2+4V9UXgCyw6LNn2OYm4KZF9EuStAheoSpJDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBfcM9yaYkDyZ5PMljSd7XtV+Q5IEkX+3uX9m1J8nvJzmU5CtJ3rDcRUiSXmyQI/eTwK6qugS4ArgxySXAbmBvVW0B9nbzAG8HtnS3ncBtS95rSdIZ9Q33qnqmqh7upr8FHAQ2AtuBO7rV7gDe1U1vB/60eh4C1ifZsOQ9lySdVqpq8JWTzcDngUuBr1fV+q49wPGqWp/kPuDmqvpCt2wv8IGq2jfvsXbSO7JnfHz8sunp6b77n52dZWxs7EVtB46cGLj/S23rxvOH3nahWkaVtaw9rdQB1nImU1NT+6tqYqFl6wZ9kCRjwKeB91fVc70876mqSjL4p0Rvmz3AHoCJiYmanJzsu83MzAzz17t+9/1ns9sldfi6yaG3XaiWUWUta08rdYC1DGugs2WSvJResH+iqj7TNR89NdzS3R/r2o8Am+ZsflHXJklaIYOcLRPgduBgVX10zqJ7gR3d9A7gnjntv9ydNXMFcKKqnlnCPkuS+hhkWObNwHuAA0ke6do+BNwM3JnkBuAp4Opu2WeBdwCHgO8A713SHkuS+uob7t0fRnOaxVcusH4BNy6yX5KkRfAKVUlqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIatG61OzDKNu++f+htd209yfVDbn/45ncOvV9JPxw8cpekBvUN9yQfT3IsyaNz2j6c5EiSR7rbO+Ys+2CSQ0meSPK25eq4JOn0Bjly/xPgqgXab62qbd3tswBJLgGuAX662+a/JjlnqTorSRpM33Cvqs8D3xzw8bYD01X1fFU9CRwCLl9E/yRJQ0hV9V8p2QzcV1WXdvMfBq4HngP2Abuq6niSPwAeqqo/69a7HfhcVd21wGPuBHYCjI+PXzY9Pd23H7Ozs4yNjb2o7cCRE323W4vGz4Wj3x1u260bz1/azizSQs/LqGqlllbqAGs5k6mpqf1VNbHQsmHPlrkN+G2guvtbgF85mweoqj3AHoCJiYmanJzsu83MzAzz1xv2jJPVtmvrSW45MNw//+HrJpe2M4u00PMyqlqppZU6wFqGNdTZMlV1tKpeqKrvAX/E94dejgCb5qx6UdcmSVpBQ4V7kg1zZn8ROHUmzb3ANUlenuRiYAvwpcV1UZJ0tvqOCyT5JDAJXJjkaeC3gMkk2+gNyxwGfg2gqh5LcifwOHASuLGqXlierkuSTqdvuFfVtQs0336G9W8CblpMpyRJi+MVqpLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhrUN9yTfDzJsSSPzmm7IMkDSb7a3b+ya0+S309yKMlXkrxhOTsvSVrYIEfufwJcNa9tN7C3qrYAe7t5gLcDW7rbTuC2pemmJOls9A33qvo88M15zduBO7rpO4B3zWn/0+p5CFifZMNSdVaSNJhUVf+Vks3AfVV1aTf/bFWt76YDHK+q9UnuA26uqi90y/YCH6iqfQs85k56R/eMj49fNj093bcfs7OzjI2NvajtwJETfbdbi8bPhaPfHW7brRvPX9rOLNJCz8uoaqWWVuoAazmTqamp/VU1sdCydYt98KqqJP0/IX5wuz3AHoCJiYmanJzsu83MzAzz17t+9/1nu+s1YdfWk9xyYLh//sPXTS5tZxZpoedlVLVSSyt1gLUMa9izZY6eGm7p7o917UeATXPWu6hrkyStoGHD/V5gRze9A7hnTvsvd2fNXAGcqKpnFtlHSdJZ6jsukOSTwCRwYZKngd8CbgbuTHID8BRwdbf6Z4F3AIeA7wDvXYY+S5L66BvuVXXtaRZducC6Bdy42E7pzDav4t8ZDt/8zlXbt6TBeYWqJDXIcJekBhnuktSgRZ/nrh8uC43379p6ctmvN3CsXzo7HrlLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNWjdYjZOchj4FvACcLKqJpJcAHwK2AwcBq6uquOL66Yk6WwsxZH7VFVtq6qJbn43sLeqtgB7u3lJ0gpa1JH7aWwHJrvpO4AZ4APLsB/9ENm8+/4V2c+urSe5ft6+Dt/8zhXZt7SUUlXDb5w8CRwHCvhvVbUnybNVtb5bHuD4qfl52+4EdgKMj49fNj093Xd/s7OzjI2NvajtwJETQ/d/NY2fC0e/u9q9WBqt17J14/mr05lFWOi9Mqqs5fSmpqb2zxk1eZHFHrm/paqOJPlx4IEkfzt3YVVVkgU/PapqD7AHYGJioiYnJ/vubGZmhvnrzT/KGhW7tp7klgPL8cVp5bVey+HrJlenM4uw0HtlVFnLcBY15l5VR7r7Y8DdwOXA0SQbALr7Y4vtpCTp7Awd7knOS/KKU9PAzwGPAvcCO7rVdgD3LLaTkqSzs5jv0uPA3b1hddYBf15Vf5Hkr4E7k9wAPAVcvfhuSpLOxtDhXlVfA163QPs/AVcuplOSpMXxClVJapDhLkkNMtwlqUGGuyQ1yHCXpAa1cVmhtIxW6ndt5vM3bbQYHrlLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIs2WkNWoxZ+ks9D9KDcqzdNrgkbskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkKdCSnqR1fqhNPA0zKXkkbskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ1atvPck1wF/B5wDvCxqrp5ufYlqQ0LnWO/mJ8vHlSL59cvy5F7knOAPwTeDlwCXJvkkuXYlyTpBy3XkfvlwKGq+hpAkmlgO/D4Mu1Pkoa2UlflLvQtZLm+NaSqlv5Bk3cDV1XVr3bz7wHeWFW/PmedncDObvY1wBMDPPSFwDeWuLurxVrWplZqaaUOsJYz+ZdV9aqFFqzab8tU1R5gz9lsk2RfVU0sU5dWlLWsTa3U0kodYC3DWq6zZY4Am+bMX9S1SZJWwHKF+18DW5JcnORlwDXAvcu0L0nSPMsyLFNVJ5P8OvCX9E6F/HhVPbYED31WwzhrnLWsTa3U0kodYC1DWZY/qEqSVpdXqEpSgwx3SWrQmg33JJuSPJjk8SSPJXlf135BkgeSfLW7f+Vq97WfJD+S5EtJ/qar5SNd+8VJvpjkUJJPdX98XvOSnJPky0nu6+ZHtY7DSQ4keSTJvq5t5F5fAEnWJ7kryd8mOZjkTaNWS5LXdM/FqdtzSd4/anWckuQ3u/f7o0k+2eXAir1X1my4AyeBXVV1CXAFcGP3Ewa7gb1VtQXY282vdc8Db62q1wHbgKuSXAH8DnBrVb0aOA7csIp9PBvvAw7OmR/VOgCmqmrbnHOPR/H1Bb3fcfqLqnot8Dp6z89I1VJVT3TPxTbgMuA7wN2MWB0ASTYCvwFMVNWl9E4suYaVfK9U1UjcgHuAn6V3JeuGrm0D8MRq9+0s6/hR4GHgjfSuVFvXtb8J+MvV7t8A/b+I3hvsrcB9QEaxjq6vh4EL57WN3OsLOB94ku4EiVGuZU7ffw74P6NaB7AR+HvgAnpnJd4HvG0l3ytr+cj9nyXZDLwe+CIwXlXPdIv+ARhfpW6dlW4o4xHgGPAA8HfAs1V1slvlaXoviLXuvwD/AfheN/9jjGYdAAX8zyT7u5/DgNF8fV0M/CPwx91w2ceSnMdo1nLKNcAnu+mRq6OqjgC/C3wdeAY4AexnBd8raz7ck4wBnwbeX1XPzV1WvY+/kTiXs6peqN7XzYvo/bDaa1e5S2ctyc8Dx6pq/2r3ZYm8pareQO/XS29M8q/mLhyh19c64A3AbVX1euDbzBu6GKFa6MahfwH4H/OXjUod3d8FttP74P0J4DzgqpXsw5oO9yQvpRfsn6iqz3TNR5Ns6JZvoHckPDKq6lngQXpfydYnOXUh2Sj8RMObgV9IchiYpjc083uMXh3APx9dUVXH6I3tXs5ovr6eBp6uqi9283fRC/tRrAV6H7YPV9XRbn4U6/jXwJNV9Y9V9f+Az9B7/6zYe2XNhnuSALcDB6vqo3MW3Qvs6KZ30BuLX9OSvCrJ+m76XHp/OzhIL+Tf3a225mupqg9W1UVVtZne1+a/qqrrGLE6AJKcl+QVp6bpjfE+ygi+vqrqH4C/T/KarulKej+vPXK1dK7l+0MyMJp1fB24IsmPdll26jlZsffKmr1CNclbgP8NHOD747sfojfufifwL4CngKur6pur0skBJfkZ4A56fzF/CXBnVf3HJD9J7wj4AuDLwL+tqudXr6eDSzIJ/Puq+vlRrKPr893d7Drgz6vqpiQ/xoi9vgCSbAM+BrwM+BrwXrrXGiNUS/dB+3XgJ6vqRNc2qs/JR4B/Q+/Mvy8Dv0pvjH1F3itrNtwlScNbs8MykqThGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQf8fGCkgCoaPetwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xZHMvrbwPBQq",
        "colab_type": "text"
      },
      "source": [
        "Let us apply standardization to this dataset. For this, let us use Scikit-Learn."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xxkeFFaXNd5m",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "0f0974aa-9eb5-45e1-89e3-8ae6a2ab05af"
      },
      "source": [
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "scaler = StandardScaler()\n",
        "X_scaled = scaler.fit_transform(X)\n",
        "\n",
        "X_scaled = pd.DataFrame(X_scaled, columns=X.columns)\n",
        "X_scaled.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Pregnancies</th>\n",
              "      <th>Glucose</th>\n",
              "      <th>BloodPressure</th>\n",
              "      <th>SkinThickness</th>\n",
              "      <th>Insulin</th>\n",
              "      <th>BMI</th>\n",
              "      <th>DiabetesPedigreeFunction</th>\n",
              "      <th>Age</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.639947</td>\n",
              "      <td>0.848324</td>\n",
              "      <td>0.149641</td>\n",
              "      <td>0.907270</td>\n",
              "      <td>-0.692891</td>\n",
              "      <td>0.204013</td>\n",
              "      <td>0.468492</td>\n",
              "      <td>1.425995</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-0.844885</td>\n",
              "      <td>-1.123396</td>\n",
              "      <td>-0.160546</td>\n",
              "      <td>0.530902</td>\n",
              "      <td>-0.692891</td>\n",
              "      <td>-0.684422</td>\n",
              "      <td>-0.365061</td>\n",
              "      <td>-0.190672</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1.233880</td>\n",
              "      <td>1.943724</td>\n",
              "      <td>-0.263941</td>\n",
              "      <td>-1.288212</td>\n",
              "      <td>-0.692891</td>\n",
              "      <td>-1.103255</td>\n",
              "      <td>0.604397</td>\n",
              "      <td>-0.105584</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.844885</td>\n",
              "      <td>-0.998208</td>\n",
              "      <td>-0.160546</td>\n",
              "      <td>0.154533</td>\n",
              "      <td>0.123302</td>\n",
              "      <td>-0.494043</td>\n",
              "      <td>-0.920763</td>\n",
              "      <td>-1.041549</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-1.141852</td>\n",
              "      <td>0.504055</td>\n",
              "      <td>-1.504687</td>\n",
              "      <td>0.907270</td>\n",
              "      <td>0.765836</td>\n",
              "      <td>1.409746</td>\n",
              "      <td>5.484909</td>\n",
              "      <td>-0.020496</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Pregnancies   Glucose  ...  DiabetesPedigreeFunction       Age\n",
              "0     0.639947  0.848324  ...                  0.468492  1.425995\n",
              "1    -0.844885 -1.123396  ...                 -0.365061 -0.190672\n",
              "2     1.233880  1.943724  ...                  0.604397 -0.105584\n",
              "3    -0.844885 -0.998208  ...                 -0.920763 -1.041549\n",
              "4    -1.141852  0.504055  ...                  5.484909 -0.020496\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 170
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bO8dPFBAPaCw",
        "colab_type": "text"
      },
      "source": [
        "To see the effect of the transformation (scaling), we can call de `describe` function on the scaled dataset:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IJrCOkHQOMPj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 377
        },
        "outputId": "f72eb4aa-3855-4062-ee4d-dfa37f0398e0"
      },
      "source": [
        "X_scaled.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Pregnancies</th>\n",
              "      <th>Glucose</th>\n",
              "      <th>BloodPressure</th>\n",
              "      <th>SkinThickness</th>\n",
              "      <th>Insulin</th>\n",
              "      <th>BMI</th>\n",
              "      <th>DiabetesPedigreeFunction</th>\n",
              "      <th>Age</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "      <td>7.680000e+02</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>2.544261e-17</td>\n",
              "      <td>3.614007e-18</td>\n",
              "      <td>-1.327244e-17</td>\n",
              "      <td>7.994184e-17</td>\n",
              "      <td>-3.556183e-17</td>\n",
              "      <td>2.295979e-16</td>\n",
              "      <td>2.398978e-16</td>\n",
              "      <td>1.857600e-16</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "      <td>1.000652e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>-1.141852e+00</td>\n",
              "      <td>-3.783654e+00</td>\n",
              "      <td>-3.572597e+00</td>\n",
              "      <td>-1.288212e+00</td>\n",
              "      <td>-6.928906e-01</td>\n",
              "      <td>-4.060474e+00</td>\n",
              "      <td>-1.189553e+00</td>\n",
              "      <td>-1.041549e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>-8.448851e-01</td>\n",
              "      <td>-6.852363e-01</td>\n",
              "      <td>-3.673367e-01</td>\n",
              "      <td>-1.288212e+00</td>\n",
              "      <td>-6.928906e-01</td>\n",
              "      <td>-5.955785e-01</td>\n",
              "      <td>-6.889685e-01</td>\n",
              "      <td>-7.862862e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>-2.509521e-01</td>\n",
              "      <td>-1.218877e-01</td>\n",
              "      <td>1.496408e-01</td>\n",
              "      <td>1.545332e-01</td>\n",
              "      <td>-4.280622e-01</td>\n",
              "      <td>9.419788e-04</td>\n",
              "      <td>-3.001282e-01</td>\n",
              "      <td>-3.608474e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>6.399473e-01</td>\n",
              "      <td>6.057709e-01</td>\n",
              "      <td>5.632228e-01</td>\n",
              "      <td>7.190857e-01</td>\n",
              "      <td>4.120079e-01</td>\n",
              "      <td>5.847705e-01</td>\n",
              "      <td>4.662269e-01</td>\n",
              "      <td>6.602056e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>3.906578e+00</td>\n",
              "      <td>2.444478e+00</td>\n",
              "      <td>2.734528e+00</td>\n",
              "      <td>4.921866e+00</td>\n",
              "      <td>6.652839e+00</td>\n",
              "      <td>4.455807e+00</td>\n",
              "      <td>5.883565e+00</td>\n",
              "      <td>4.063716e+00</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "        Pregnancies       Glucose  ...  DiabetesPedigreeFunction           Age\n",
              "count  7.680000e+02  7.680000e+02  ...              7.680000e+02  7.680000e+02\n",
              "mean   2.544261e-17  3.614007e-18  ...              2.398978e-16  1.857600e-16\n",
              "std    1.000652e+00  1.000652e+00  ...              1.000652e+00  1.000652e+00\n",
              "min   -1.141852e+00 -3.783654e+00  ...             -1.189553e+00 -1.041549e+00\n",
              "25%   -8.448851e-01 -6.852363e-01  ...             -6.889685e-01 -7.862862e-01\n",
              "50%   -2.509521e-01 -1.218877e-01  ...             -3.001282e-01 -3.608474e-01\n",
              "75%    6.399473e-01  6.057709e-01  ...              4.662269e-01  6.602056e-01\n",
              "max    3.906578e+00  2.444478e+00  ...              5.883565e+00  4.063716e+00\n",
              "\n",
              "[8 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 171
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vtzKbM3ZPnvC",
        "colab_type": "text"
      },
      "source": [
        "Please, see the result of the describe function above and notice the the means and standard deviations of the attributes were all mapped to 0 and 1, respectively. For example, see below how the number 2.544261e-17 (in scientific notation) is computed:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q9CPeXNuQJSy",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "5fd77a68-5563-4114-8219-aa73ee362e3d"
      },
      "source": [
        "2.544261 / 100000000000000000"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.5442610000000002e-17"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PTu1GIoLRXuG",
        "colab_type": "text"
      },
      "source": [
        "Now, compare the histogram below to the original one."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7-J0l5sJOL6K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X_scaled.Age.hist()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0tdgGp2PRdNZ",
        "colab_type": "text"
      },
      "source": [
        "The code below provides some perspective on what is going on behind the scenes when we use the StandardScaler class of Scikit-Learn."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wx7rwLgXOKT3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "a0b65b47-e0fe-48be-bdf0-8118e5f27868"
      },
      "source": [
        "age_mean = X.Age.mean()\n",
        "age_std = X.Age.std()\n",
        "age_column = X.Age.copy()\n",
        "scaled_column = (age_column - age_mean) / age_std\n",
        "print('Mean = %.2f and scale = %.2f for feature Age' % (scaler.mean_[7], scaler.scale_[7]))\n",
        "print('Mean = %.2f and scale = %.2f for feature Age' % (X.Age.mean(), X.Age.std(ddof=0)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mean = 33.24 and scale = 11.75 for feature Age\n",
            "Mean = 33.24 and scale = 11.75 for feature Age\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gwHA6zDlNgg2",
        "colab_type": "text"
      },
      "source": [
        "Notice that the code above uses the corrected standard deviation value. For details about the difference between the corrected and uncorrected standard deviations, see https://en.wikipedia.org/wiki/Standard_deviation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pL9x2mNnU_FZ",
        "colab_type": "text"
      },
      "source": [
        "### Which feature scaling to apply?\n",
        "\n",
        "There is no hard and fast rule to tell you which feature scaling technique to apply. The choice of using which feature scaling to apply to depends on your problem and the Data Mining algorithm to be used. At the end of the day, the choice boils down to a trial-and-error process: you can fit your model several times, applying different scaling techniques and choose the technique that produces the best results."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BP_CFIQAa2lY",
        "colab_type": "text"
      },
      "source": [
        "## Feature encoding\n",
        "\n",
        "Often, features of a dataset take discrete values ​​(instead of continuous values). For example, a person can have features with the following values:\n",
        "\n",
        "- (\"male, female\"),\n",
        "- (\"from Europe\", \"from the USA\", \"from Asia\"),\n",
        "- (\"use Firefox\", \"use Chrome\", \"use Safari\", \"use Internet Explorer\").\n",
        "\n",
        "Many Data Mining methods expect the features of a dataset to be only numeric. The most common types of descrite encodings are *Ordinal Encoding* and *One-Hot Encoding*. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qZ88PrdBexT6",
        "colab_type": "text"
      },
      "source": [
        "### Ordinal Encoding\n",
        "\n",
        "In this type of encoding, the values os a discrete feature ​​are converted to ordinal integers. This results in a single column of integers per feature. This is a encoding normally applied to ordinal features.\n",
        "\n",
        "![alt text](https://i.imgur.com/tEogUAr.png)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CcCbdQTZeuWx",
        "colab_type": "text"
      },
      "source": [
        "### One-Hot Encoding\n",
        "\n",
        "Sometimes, the values ​​of a discrete feature do not have an intrinsic order. In such cases, a more appropriate coding approach is to use *dummy features*. Dummy features are those that contain values ​​like 1 or 0, representing the fact that a given sample has or does not have a certain property. This is a encoding usually applied to nominal features.\n",
        "\n",
        "![alt text](https://i.imgur.com/TW5m0aJ.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YNl67BhPehC1",
        "colab_type": "text"
      },
      "source": [
        "### Case study: encoding features of the Salary Dataset\n",
        "\n",
        "Let us use Scikit-Learn and Pandas for practicing feature encoding on the Salary dataset. This small dataset contains salary data used in Weisberg's book, consisting of observations on six variables for 52 teachers. The variables are:\n",
        "\n",
        "* sx = Sex, coded: 1 for female and 0 for male\n",
        "* rk = Academic Rank\n",
        "* yr = Number of years in the current rank\n",
        "* dg = highest degree, coded 1 if doctorate, 0 if master's degree\n",
        "* yd = Number of years since the highest grade was earned\n",
        "* sl = salary for the academic year, in dollars.\n",
        "\n",
        "S. Weisberg (1985). Applied Linear Regression, Second Edition. New York: John Wiley and Sons. Page 194."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x6dmTw54a2Ht",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "9f31bb3d-980d-42a8-a151-bf4669be9c74"
      },
      "source": [
        "import pandas as pd\n",
        "df_salary = pd.read_csv('https://data.princeton.edu/wws509/datasets/salary.dat', delim_whitespace=True)\n",
        "df_salary.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sx</th>\n",
              "      <th>rk</th>\n",
              "      <th>yr</th>\n",
              "      <th>dg</th>\n",
              "      <th>yd</th>\n",
              "      <th>sl</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>25</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>35</td>\n",
              "      <td>36350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>13</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>22</td>\n",
              "      <td>35350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>10</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>23</td>\n",
              "      <td>28200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>female</td>\n",
              "      <td>full</td>\n",
              "      <td>7</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>27</td>\n",
              "      <td>26775</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>19</td>\n",
              "      <td>masters</td>\n",
              "      <td>30</td>\n",
              "      <td>33696</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       sx    rk  yr         dg  yd     sl\n",
              "0    male  full  25  doctorate  35  36350\n",
              "1    male  full  13  doctorate  22  35350\n",
              "2    male  full  10  doctorate  23  28200\n",
              "3  female  full   7  doctorate  27  26775\n",
              "4    male  full  19    masters  30  33696"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 175
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rTeE88Ii_1BU",
        "colab_type": "text"
      },
      "source": [
        "Pandas provides an way of doing one-hot encoding, which is the `get_dummies` method. See the example below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X3TeOdiAgqi0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "699e6e95-52b6-42b3-dfd8-27e92f0f1a5e"
      },
      "source": [
        "df_dummy = pd.get_dummies(df_salary['sx'])\n",
        "df_dummy.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>female</th>\n",
              "      <th>male</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   female  male\n",
              "0       0     1\n",
              "1       0     1\n",
              "2       0     1\n",
              "3       1     0\n",
              "4       0     1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rrLlUfUFgbvp",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "2014b975-591e-4bb4-f2e0-aa43b18f1d69"
      },
      "source": [
        "df_salary_new = pd.concat([df_salary, df_dummy], axis=1)\n",
        "df_salary_new.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sx</th>\n",
              "      <th>rk</th>\n",
              "      <th>yr</th>\n",
              "      <th>dg</th>\n",
              "      <th>yd</th>\n",
              "      <th>sl</th>\n",
              "      <th>female</th>\n",
              "      <th>male</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>25</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>35</td>\n",
              "      <td>36350</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>13</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>22</td>\n",
              "      <td>35350</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>10</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>23</td>\n",
              "      <td>28200</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>female</td>\n",
              "      <td>full</td>\n",
              "      <td>7</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>27</td>\n",
              "      <td>26775</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>male</td>\n",
              "      <td>full</td>\n",
              "      <td>19</td>\n",
              "      <td>masters</td>\n",
              "      <td>30</td>\n",
              "      <td>33696</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       sx    rk  yr         dg  yd     sl  female  male\n",
              "0    male  full  25  doctorate  35  36350       0     1\n",
              "1    male  full  13  doctorate  22  35350       0     1\n",
              "2    male  full  10  doctorate  23  28200       0     1\n",
              "3  female  full   7  doctorate  27  26775       1     0\n",
              "4    male  full  19    masters  30  33696       0     1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l9B4Nz0yg4aA",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "197aff14-7636-4227-a3d4-d21afe8fd225"
      },
      "source": [
        "df_salary_new.drop(columns=['sx'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>rk</th>\n",
              "      <th>yr</th>\n",
              "      <th>dg</th>\n",
              "      <th>yd</th>\n",
              "      <th>sl</th>\n",
              "      <th>female</th>\n",
              "      <th>male</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>full</td>\n",
              "      <td>25</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>35</td>\n",
              "      <td>36350</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>full</td>\n",
              "      <td>13</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>22</td>\n",
              "      <td>35350</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>full</td>\n",
              "      <td>10</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>23</td>\n",
              "      <td>28200</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>full</td>\n",
              "      <td>7</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>27</td>\n",
              "      <td>26775</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>full</td>\n",
              "      <td>19</td>\n",
              "      <td>masters</td>\n",
              "      <td>30</td>\n",
              "      <td>33696</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>full</td>\n",
              "      <td>16</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>21</td>\n",
              "      <td>28516</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>full</td>\n",
              "      <td>0</td>\n",
              "      <td>masters</td>\n",
              "      <td>32</td>\n",
              "      <td>24900</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>full</td>\n",
              "      <td>16</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>18</td>\n",
              "      <td>31909</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>full</td>\n",
              "      <td>13</td>\n",
              "      <td>masters</td>\n",
              "      <td>30</td>\n",
              "      <td>31850</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>full</td>\n",
              "      <td>13</td>\n",
              "      <td>masters</td>\n",
              "      <td>31</td>\n",
              "      <td>32850</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>full</td>\n",
              "      <td>12</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>22</td>\n",
              "      <td>27025</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>associate</td>\n",
              "      <td>15</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>19</td>\n",
              "      <td>24750</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>full</td>\n",
              "      <td>9</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>17</td>\n",
              "      <td>28200</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>associate</td>\n",
              "      <td>9</td>\n",
              "      <td>masters</td>\n",
              "      <td>27</td>\n",
              "      <td>23712</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>full</td>\n",
              "      <td>9</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>24</td>\n",
              "      <td>25748</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>full</td>\n",
              "      <td>7</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>15</td>\n",
              "      <td>29342</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>full</td>\n",
              "      <td>13</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>20</td>\n",
              "      <td>31114</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>associate</td>\n",
              "      <td>11</td>\n",
              "      <td>masters</td>\n",
              "      <td>14</td>\n",
              "      <td>24742</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>associate</td>\n",
              "      <td>10</td>\n",
              "      <td>masters</td>\n",
              "      <td>15</td>\n",
              "      <td>22906</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>full</td>\n",
              "      <td>6</td>\n",
              "      <td>masters</td>\n",
              "      <td>21</td>\n",
              "      <td>24450</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>assistant</td>\n",
              "      <td>16</td>\n",
              "      <td>masters</td>\n",
              "      <td>23</td>\n",
              "      <td>19175</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>associate</td>\n",
              "      <td>8</td>\n",
              "      <td>masters</td>\n",
              "      <td>31</td>\n",
              "      <td>20525</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>full</td>\n",
              "      <td>7</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>13</td>\n",
              "      <td>27959</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>full</td>\n",
              "      <td>8</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>24</td>\n",
              "      <td>38045</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>associate</td>\n",
              "      <td>9</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>12</td>\n",
              "      <td>24832</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>full</td>\n",
              "      <td>5</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>18</td>\n",
              "      <td>25400</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>associate</td>\n",
              "      <td>11</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>14</td>\n",
              "      <td>24800</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>full</td>\n",
              "      <td>5</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>16</td>\n",
              "      <td>25500</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>associate</td>\n",
              "      <td>3</td>\n",
              "      <td>masters</td>\n",
              "      <td>7</td>\n",
              "      <td>26182</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29</th>\n",
              "      <td>associate</td>\n",
              "      <td>3</td>\n",
              "      <td>masters</td>\n",
              "      <td>17</td>\n",
              "      <td>23725</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>assistant</td>\n",
              "      <td>10</td>\n",
              "      <td>masters</td>\n",
              "      <td>15</td>\n",
              "      <td>21600</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>associate</td>\n",
              "      <td>11</td>\n",
              "      <td>masters</td>\n",
              "      <td>31</td>\n",
              "      <td>23300</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32</th>\n",
              "      <td>assistant</td>\n",
              "      <td>9</td>\n",
              "      <td>masters</td>\n",
              "      <td>14</td>\n",
              "      <td>23713</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33</th>\n",
              "      <td>associate</td>\n",
              "      <td>4</td>\n",
              "      <td>masters</td>\n",
              "      <td>33</td>\n",
              "      <td>20690</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>34</th>\n",
              "      <td>associate</td>\n",
              "      <td>6</td>\n",
              "      <td>masters</td>\n",
              "      <td>29</td>\n",
              "      <td>22450</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35</th>\n",
              "      <td>associate</td>\n",
              "      <td>1</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>9</td>\n",
              "      <td>20850</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>36</th>\n",
              "      <td>assistant</td>\n",
              "      <td>8</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>14</td>\n",
              "      <td>18304</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>37</th>\n",
              "      <td>assistant</td>\n",
              "      <td>4</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>4</td>\n",
              "      <td>17095</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>38</th>\n",
              "      <td>assistant</td>\n",
              "      <td>4</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>5</td>\n",
              "      <td>16700</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>39</th>\n",
              "      <td>assistant</td>\n",
              "      <td>4</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>4</td>\n",
              "      <td>17600</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>assistant</td>\n",
              "      <td>3</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>4</td>\n",
              "      <td>18075</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>assistant</td>\n",
              "      <td>3</td>\n",
              "      <td>masters</td>\n",
              "      <td>11</td>\n",
              "      <td>18000</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>associate</td>\n",
              "      <td>0</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>7</td>\n",
              "      <td>20999</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>assistant</td>\n",
              "      <td>3</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>3</td>\n",
              "      <td>17250</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>assistant</td>\n",
              "      <td>2</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>3</td>\n",
              "      <td>16500</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45</th>\n",
              "      <td>assistant</td>\n",
              "      <td>2</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>1</td>\n",
              "      <td>16094</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46</th>\n",
              "      <td>assistant</td>\n",
              "      <td>2</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>6</td>\n",
              "      <td>16150</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47</th>\n",
              "      <td>assistant</td>\n",
              "      <td>2</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>2</td>\n",
              "      <td>15350</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>48</th>\n",
              "      <td>assistant</td>\n",
              "      <td>1</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>1</td>\n",
              "      <td>16244</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49</th>\n",
              "      <td>assistant</td>\n",
              "      <td>1</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>1</td>\n",
              "      <td>16686</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50</th>\n",
              "      <td>assistant</td>\n",
              "      <td>1</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>1</td>\n",
              "      <td>15000</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51</th>\n",
              "      <td>assistant</td>\n",
              "      <td>0</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>2</td>\n",
              "      <td>20300</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "           rk  yr         dg  yd     sl  female  male\n",
              "0        full  25  doctorate  35  36350       0     1\n",
              "1        full  13  doctorate  22  35350       0     1\n",
              "2        full  10  doctorate  23  28200       0     1\n",
              "3        full   7  doctorate  27  26775       1     0\n",
              "4        full  19    masters  30  33696       0     1\n",
              "5        full  16  doctorate  21  28516       0     1\n",
              "6        full   0    masters  32  24900       1     0\n",
              "7        full  16  doctorate  18  31909       0     1\n",
              "8        full  13    masters  30  31850       0     1\n",
              "9        full  13    masters  31  32850       0     1\n",
              "10       full  12  doctorate  22  27025       0     1\n",
              "11  associate  15  doctorate  19  24750       0     1\n",
              "12       full   9  doctorate  17  28200       0     1\n",
              "13  associate   9    masters  27  23712       0     1\n",
              "14       full   9  doctorate  24  25748       0     1\n",
              "15       full   7  doctorate  15  29342       0     1\n",
              "16       full  13  doctorate  20  31114       0     1\n",
              "17  associate  11    masters  14  24742       0     1\n",
              "18  associate  10    masters  15  22906       0     1\n",
              "19       full   6    masters  21  24450       0     1\n",
              "20  assistant  16    masters  23  19175       0     1\n",
              "21  associate   8    masters  31  20525       0     1\n",
              "22       full   7  doctorate  13  27959       0     1\n",
              "23       full   8  doctorate  24  38045       1     0\n",
              "24  associate   9  doctorate  12  24832       0     1\n",
              "25       full   5  doctorate  18  25400       0     1\n",
              "26  associate  11  doctorate  14  24800       0     1\n",
              "27       full   5  doctorate  16  25500       1     0\n",
              "28  associate   3    masters   7  26182       0     1\n",
              "29  associate   3    masters  17  23725       0     1\n",
              "30  assistant  10    masters  15  21600       1     0\n",
              "31  associate  11    masters  31  23300       0     1\n",
              "32  assistant   9    masters  14  23713       0     1\n",
              "33  associate   4    masters  33  20690       1     0\n",
              "34  associate   6    masters  29  22450       1     0\n",
              "35  associate   1  doctorate   9  20850       0     1\n",
              "36  assistant   8  doctorate  14  18304       1     0\n",
              "37  assistant   4  doctorate   4  17095       0     1\n",
              "38  assistant   4  doctorate   5  16700       0     1\n",
              "39  assistant   4  doctorate   4  17600       0     1\n",
              "40  assistant   3  doctorate   4  18075       0     1\n",
              "41  assistant   3    masters  11  18000       0     1\n",
              "42  associate   0  doctorate   7  20999       0     1\n",
              "43  assistant   3  doctorate   3  17250       1     0\n",
              "44  assistant   2  doctorate   3  16500       0     1\n",
              "45  assistant   2  doctorate   1  16094       0     1\n",
              "46  assistant   2  doctorate   6  16150       1     0\n",
              "47  assistant   2  doctorate   2  15350       1     0\n",
              "48  assistant   1  doctorate   1  16244       0     1\n",
              "49  assistant   1  doctorate   1  16686       1     0\n",
              "50  assistant   1  doctorate   1  15000       1     0\n",
              "51  assistant   0  doctorate   2  20300       1     0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uyL9y_RQuTc_",
        "colab_type": "text"
      },
      "source": [
        "Now, let us encode the feature `rk`. We will use the following encoding: 1 for assistant professor, 2 for associate professor and 3 for full professor. For this, we are going to use the Pandas `replace` function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dz_vp3SKq7TC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "5267c38a-fef1-435f-d76c-61f6749132db"
      },
      "source": [
        "df_salary.rk.unique()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['full', 'associate', 'assistant'], dtype=object)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 176
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xgJCnzYY5iMx",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 190
        },
        "outputId": "f4984485-38f7-4267-e31a-84f09dd71568"
      },
      "source": [
        "df_salary.rk.values"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['full', 'full', 'full', 'full', 'full', 'full', 'full', 'full',\n",
              "       'full', 'full', 'full', 'associate', 'full', 'associate', 'full',\n",
              "       'full', 'full', 'associate', 'associate', 'full', 'assistant',\n",
              "       'associate', 'full', 'full', 'associate', 'full', 'associate',\n",
              "       'full', 'associate', 'associate', 'assistant', 'associate',\n",
              "       'assistant', 'associate', 'associate', 'associate', 'assistant',\n",
              "       'assistant', 'assistant', 'assistant', 'assistant', 'assistant',\n",
              "       'associate', 'assistant', 'assistant', 'assistant', 'assistant',\n",
              "       'assistant', 'assistant', 'assistant', 'assistant', 'assistant'],\n",
              "      dtype=object)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 177
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BwtIuD4LqygR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mapping = {\"assistant\": 1, \"associate\": 2, \"full\": 3}\n",
        "\n",
        "df_salary.replace({'rk': mapping}, inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xc6o4-eoslay",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "975b59d7-767f-4875-f4bc-6e3942eb63dd"
      },
      "source": [
        "df_salary.rk.unique()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([3, 2, 1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 179
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EJ9wpSyGXlfC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "ab39afe3-a650-4307-83b8-3053ed88cac5"
      },
      "source": [
        "df_salary.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sx</th>\n",
              "      <th>rk</th>\n",
              "      <th>yr</th>\n",
              "      <th>dg</th>\n",
              "      <th>yd</th>\n",
              "      <th>sl</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>male</td>\n",
              "      <td>3</td>\n",
              "      <td>25</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>35</td>\n",
              "      <td>36350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>male</td>\n",
              "      <td>3</td>\n",
              "      <td>13</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>22</td>\n",
              "      <td>35350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>male</td>\n",
              "      <td>3</td>\n",
              "      <td>10</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>23</td>\n",
              "      <td>28200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>female</td>\n",
              "      <td>3</td>\n",
              "      <td>7</td>\n",
              "      <td>doctorate</td>\n",
              "      <td>27</td>\n",
              "      <td>26775</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>male</td>\n",
              "      <td>3</td>\n",
              "      <td>19</td>\n",
              "      <td>masters</td>\n",
              "      <td>30</td>\n",
              "      <td>33696</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       sx  rk  yr         dg  yd     sl\n",
              "0    male   3  25  doctorate  35  36350\n",
              "1    male   3  13  doctorate  22  35350\n",
              "2    male   3  10  doctorate  23  28200\n",
              "3  female   3   7  doctorate  27  26775\n",
              "4    male   3  19    masters  30  33696"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 180
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LvpO9YpBhbhB",
        "colab_type": "text"
      },
      "source": [
        "### Exercise\n",
        "\n",
        "As an exercise, try to encode features of the Adult Dataset. Details about this dataset can be found [here](https://www.kaggle.com/wenruliu/adult-income-dataset).\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OjRrkQKahcv3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "af0310ea-5fa4-49f6-f3bd-48bfefa16fa9"
      },
      "source": [
        "# Source: https://www.oreilly.com/library/view/introduction-to-machine/9781449369880/ch04.html\n",
        "\n",
        "# The file has no headers naming the columns, so we pass header=None\n",
        "# and provide the column names explicitly in \"names\"\n",
        "data = pd.read_csv(\n",
        "    'adult.data', header=None, index_col=False,\n",
        "    names=['age', 'workclass', 'fnlwgt', 'education',  'education-num',\n",
        "           'marital-status', 'occupation', 'relationship', 'race', 'gender',\n",
        "           'capital-gain', 'capital-loss', 'hours-per-week', 'native-country',\n",
        "           'income'])\n",
        "\n",
        "# For illustration purposes, we only select some of the columns\n",
        "df_adult = data[['age', 'workclass', 'education', 'gender', 'hours-per-week',\n",
        "             'occupation', 'income']]\n",
        "\n",
        "df_adult.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>education</th>\n",
              "      <th>gender</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>occupation</th>\n",
              "      <th>income</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>39</td>\n",
              "      <td>State-gov</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Male</td>\n",
              "      <td>40</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>50</td>\n",
              "      <td>Self-emp-not-inc</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Male</td>\n",
              "      <td>13</td>\n",
              "      <td>Exec-managerial</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>38</td>\n",
              "      <td>Private</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>Male</td>\n",
              "      <td>40</td>\n",
              "      <td>Handlers-cleaners</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>53</td>\n",
              "      <td>Private</td>\n",
              "      <td>11th</td>\n",
              "      <td>Male</td>\n",
              "      <td>40</td>\n",
              "      <td>Handlers-cleaners</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>28</td>\n",
              "      <td>Private</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Female</td>\n",
              "      <td>40</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age          workclass  ...          occupation  income\n",
              "0   39          State-gov  ...        Adm-clerical   <=50K\n",
              "1   50   Self-emp-not-inc  ...     Exec-managerial   <=50K\n",
              "2   38            Private  ...   Handlers-cleaners   <=50K\n",
              "3   53            Private  ...   Handlers-cleaners   <=50K\n",
              "4   28            Private  ...      Prof-specialty   <=50K\n",
              "\n",
              "[5 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 181
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z-VFmoqTAk6C",
        "colab_type": "text"
      },
      "source": [
        "Let us see what are the discrete features of this dataset:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xIM4T4hrBMIv",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 259
        },
        "outputId": "729229fa-63f6-4669-8fea-f79a7d72bd77"
      },
      "source": [
        "df_adult.info()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 32561 entries, 0 to 32560\n",
            "Data columns (total 7 columns):\n",
            " #   Column          Non-Null Count  Dtype \n",
            "---  ------          --------------  ----- \n",
            " 0   age             32561 non-null  int64 \n",
            " 1   workclass       32561 non-null  object\n",
            " 2   education       32561 non-null  object\n",
            " 3   gender          32561 non-null  object\n",
            " 4   hours-per-week  32561 non-null  int64 \n",
            " 5   occupation      32561 non-null  object\n",
            " 6   income          32561 non-null  object\n",
            "dtypes: int64(2), object(5)\n",
            "memory usage: 1.7+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GZVxj1KkY0bm",
        "colab_type": "text"
      },
      "source": [
        "Complete the exercise!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u4ahAd0CUmlg",
        "colab_type": "text"
      },
      "source": [
        "## Dealing with missing values"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-kyuqRLmVCzV",
        "colab_type": "text"
      },
      "source": [
        "The difference between the data found in many tutorials and data in the real world is that the latter are rarely clean and homogeneous. In particular, many data sets have missing data. To make things even more complicated, different data sources can indicate missing data in different ways (null, NaN, etc.). For example, we may have to manipulate a data matrix similar to the one presented below:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fA_ZdK4UN8nB",
        "colab_type": "code",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "bfc09c61-0a92-4cc5-fe2c-b9a95f2a0e51"
      },
      "source": [
        "import numpy as np\n",
        "from numpy import nan\n",
        "X = pd.DataFrame(\n",
        "      np.array([[ nan, 0,   3  ],\n",
        "                [ 3,   7,   9  ],\n",
        "                [ 3,   5,   2  ],\n",
        "                [ 4,   nan, 6  ],\n",
        "                [ 8,   8,   1  ]]))\n",
        "X.columns = ['c1', 'c2', 'c3']\n",
        "X.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>c1</th>\n",
              "      <th>c2</th>\n",
              "      <th>c3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>NaN</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>9.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>6.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>8.0</td>\n",
              "      <td>8.0</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    c1   c2   c3\n",
              "0  NaN  0.0  3.0\n",
              "1  3.0  7.0  9.0\n",
              "2  3.0  5.0  2.0\n",
              "3  4.0  NaN  6.0\n",
              "4  8.0  8.0  1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 184
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wex5o-EMapxE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 190
        },
        "outputId": "ab15d210-04da-43b9-8c72-0d333492fc79"
      },
      "source": [
        "X.info()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 5 entries, 0 to 4\n",
            "Data columns (total 3 columns):\n",
            " #   Column  Non-Null Count  Dtype  \n",
            "---  ------  --------------  -----  \n",
            " 0   c1      4 non-null      float64\n",
            " 1   c2      4 non-null      float64\n",
            " 2   c3      5 non-null      float64\n",
            "dtypes: float64(3)\n",
            "memory usage: 248.0 bytes\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lt0ib5R3ON86",
        "colab_type": "text"
      },
      "source": [
        "To apply some Data Mining algorithm to this data, we must first replace the missing data with an appropriate value. This procedure is known as *missing data imputation*.\n",
        "\n",
        "The strategies to perform imputation can be simple (for example, substitution of missing values ​​for the column average) or sophisticated (for example, using a robust imputation model).\n",
        "\n",
        "Scikit-Learn provides the class [`SimpleImputer`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html) to perform simple imputation. This class allows you to complete missing values ​​using some aggregation function applied to the existing values ​​in the column. This choice is made through the `strategy` parameter:\n",
        "\n",
        "* `mean`: the average value,\n",
        "* `most_frequent`: the most frequent value,\n",
        "* `median`: the median value\n",
        "* `constant`: replaces with a constant value (also define the parameter fill_value)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZBU0_2eCPblg",
        "colab_type": "code",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "outputId": "e733be31-7636-4a5b-d5cd-4525f7ee2646"
      },
      "source": [
        "from sklearn.impute import SimpleImputer\n",
        "\n",
        "imp = SimpleImputer(strategy='mean')\n",
        "\n",
        "imp = imp.fit(X)\n",
        "\n",
        "X2 = pd.DataFrame(imp.transform(X), columns=X.columns)\n",
        "\n",
        "print('Original dataset:\\n', X)\n",
        "print('\\nImputed dataset:\\n', X2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original dataset:\n",
            "     c1   c2   c3\n",
            "0  NaN  0.0  3.0\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "3  4.0  NaN  6.0\n",
            "4  8.0  8.0  1.0\n",
            "\n",
            "Imputed dataset:\n",
            "     c1   c2   c3\n",
            "0  4.5  0.0  3.0\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "3  4.0  5.0  6.0\n",
            "4  8.0  8.0  1.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ndh3HcbHtCyv",
        "colab_type": "text"
      },
      "source": [
        "In the example above, notice that the returned values ​​are placed in a Numpy array and we lose all metadata. Anyway, all of these strategies can be performed directly on Pandas, using the `fillna` method to assign missing values. For example, for \"average\", we can use the following code."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sAvwe1I6t-br",
        "colab_type": "code",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "outputId": "9e18183d-d915-43d4-f98f-c3a7f1dc266a"
      },
      "source": [
        "X2 = X.fillna(X.mean())\n",
        "print('Original dataset:\\n', X)\n",
        "print('\\nImputed dataset:\\n', X2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original dataset:\n",
            "     c1   c2   c3\n",
            "0  NaN  0.0  3.0\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "3  4.0  NaN  6.0\n",
            "4  8.0  8.0  1.0\n",
            "\n",
            "Imputed dataset:\n",
            "     c1   c2   c3\n",
            "0  4.5  0.0  3.0\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "3  4.0  5.0  6.0\n",
            "4  8.0  8.0  1.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LTAaw4wiuVft",
        "colab_type": "text"
      },
      "source": [
        "Sometimes, instead of filling missing values, we can decide to drop rows or columns in with undefined values. By using Pandas, we can achieve this. See the examples below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-rnptNKIPkzp",
        "colab_type": "code",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 242
        },
        "outputId": "7e27a216-1bb8-48b0-d506-f9b248454200"
      },
      "source": [
        "X2 = X.dropna()\n",
        "print('Original dataset:\\n', X)\n",
        "print('\\nTransformed dataset:\\n', X2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original dataset:\n",
            "     c1   c2   c3\n",
            "0  NaN  0.0  3.0\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "3  4.0  NaN  6.0\n",
            "4  8.0  8.0  1.0\n",
            "\n",
            "Transformed dataset:\n",
            "     c1   c2   c3\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "4  8.0  8.0  1.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ak9NObxVPx1a",
        "colab_type": "code",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "outputId": "e74742da-c2e3-4fdf-c947-8fa131fb860c"
      },
      "source": [
        "X2 = X.dropna(axis='columns')\n",
        "print('Original dataset:\\n', X)\n",
        "print('\\nTransformed dataset:\\n', X2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original dataset:\n",
            "     c1   c2   c3\n",
            "0  NaN  0.0  3.0\n",
            "1  3.0  7.0  9.0\n",
            "2  3.0  5.0  2.0\n",
            "3  4.0  NaN  6.0\n",
            "4  8.0  8.0  1.0\n",
            "\n",
            "Transformed dataset:\n",
            "     c3\n",
            "0  3.0\n",
            "1  9.0\n",
            "2  2.0\n",
            "3  6.0\n",
            "4  1.0\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}