Assignment-64.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5d974b7b-8553-4f9d-920d-db29cf2705dc",
   "metadata": {},
   "source": [
    "### Q1. You are working on a machine learning project where you have a dataset containing numerical and categorical features. You have identified that  some of the features are highly correlated and there are missing values in some of the columns. You want to build a pipeline that automates the feature engineering process and handles the missing values.\n",
    "\n",
    "Design a pipeline that includes the following steps:\n",
    "* Use an automated feature selection method to identify the important features in the dataset\n",
    "\n",
    "Create a numerical pipeline that includes the following steps:\n",
    "* Impute the missing values in the numerical columns using the mean of the column values\n",
    "* Scale the numerical columns using standardization\n",
    "\n",
    "Create a categorical pipeline that includes the following steps:\n",
    "* Impute the missing values in the categorical columns using the most frequent value of the column\n",
    "* One-hot encode the categorical columns\n",
    "* Combine the numerical and categorical pipelines using a ColumnTransformer\n",
    "* Use a Random Forest Classifier to build the final model\n",
    "* Evaluate the accuracy of the model on the test dataset\n",
    "\n",
    "Note: Your solution should include code snippets for each step of the pipeline, and a brief explanation of each step. You should also provide an interpretation of the results and suggest possible improvements for the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f11a3267-daf2-48aa-ae53-1429ce76ee72",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Below is a Python code snippet that creates a machine learning pipeline to automate feature engineering, handle missing values, and build a Random \n",
    "Forest Classifier model. This pipeline uses scikit-learn and assumes you have a dataset with both numerical and categorical features.\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73b20331-8519-4b25-967f-cb1f9e7a0550",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.feature_selection import SelectFromModel\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "## No actual dataset is used in this code.\n",
    "## This is just the demonstration of the process for creating the pipelines and connecting them together\n",
    "\n",
    "# Load your dataset (replace 'your_dataset.csv' with the actual filename)\n",
    "data = pd.read_csv('your_dataset.csv')\n",
    "\n",
    "# Separate target variable (e.g., 'target') from features\n",
    "X = data.drop(columns=['target'])\n",
    "y = data['target']\n",
    "\n",
    "# Split the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "# Define feature selection model (Random Forest in this case)\n",
    "feature_selector = SelectFromModel(RandomForestClassifier(n_estimators=100, random_state=42))\n",
    "\n",
    "# Numerical pipeline\n",
    "numerical_pipeline = Pipeline([\n",
    "    ('imputer', SimpleImputer(strategy='mean')),\n",
    "    ('scaler', StandardScaler())\n",
    "])\n",
    "\n",
    "# Categorical pipeline\n",
    "categorical_pipeline = Pipeline([\n",
    "    ('imputer', SimpleImputer(strategy='most_frequent')),\n",
    "    ('encoder', OneHotEncoder(handle_unknown='ignore'))\n",
    "])\n",
    "\n",
    "# Combine numerical and categorical pipelines using ColumnTransformer\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('num', numerical_pipeline, X.select_dtypes(include=['number']).columns),\n",
    "        ('cat', categorical_pipeline, X.select_dtypes(exclude=['number']).columns)\n",
    "])\n",
    "\n",
    "# Create the final pipeline with feature selection and Random Forest Classifier\n",
    "pipeline = Pipeline([\n",
    "    ('feature_selector', feature_selector),\n",
    "    ('preprocessor', preprocessor),\n",
    "    ('classifier', RandomForestClassifier(n_estimators=100, random_state=42))\n",
    "])\n",
    "\n",
    "# Fit the pipeline on the training data\n",
    "pipeline.fit(X_train, y_train)\n",
    "\n",
    "# Predict on the test data\n",
    "y_pred = pipeline.predict(X_test)\n",
    "\n",
    "# Evaluate the accuracy of the model\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(f'Accuracy: {accuracy:.2f}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91c3d7d7",
   "metadata": {},
   "source": [
    "#### Explanation of Steps:\n",
    "\n",
    "1. We load the dataset and separate the target variable from the features.\n",
    "2. We split the data into training and testing sets.\n",
    "3. We define a feature selection model using a Random Forest Classifier to identify important features.\n",
    "4. We create separate pipelines for numerical and categorical data. The pipelines handle missing values by imputing with the        mean for numerical columns and the most frequent value for categorical columns. Numerical columns are also standardized.\n",
    "5. We combine the numerical and categorical pipelines using a ColumnTransformer.\n",
    "6. We create the final pipeline, including feature selection and a Random Forest Classifier.\n",
    "7. We fit the pipeline on the training data and make predictions on the test data.\n",
    "8. We evaluate the model's accuracy on the test data.\n",
    "\n",
    "#### Possible Improvements:\n",
    "\n",
    "1. Hyperparameter tuning for the Random Forest Classifier to optimize model performance.\n",
    "2. Cross-validation to get a more robust estimate of model performance.\n",
    "3. Explore different feature selection methods or consider using feature importance scores from the Random Forest.\n",
    "4. Further investigate the dataset to ensure appropriate handling of missing values and feature encoding.\n",
    "5. Consider additional preprocessing steps or feature engineering techniques based on domain knowledge.\n",
    "\n",
    "This pipeline automates many of the common preprocessing and modeling steps, making it easier to experiment and iterate on your machine learning \n",
    "project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72589e1c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2bc394e9-d4de-4ceb-b854-120e05eb3077",
   "metadata": {},
   "source": [
    "### Q2. Build a pipeline that includes a random forest classifier and a logistic regression classifier, and then use a voting classifier to combine their predictions. Train the pipeline on the iris dataset and evaluate its accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "06f82ed7-66dc-4890-b7ad-f76e62ec5e46",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 1.00\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.ensemble import RandomForestClassifier, VotingClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# Load the Iris dataset\n",
    "data = load_iris()\n",
    "X = data.data\n",
    "y = data.target\n",
    "\n",
    "# Split the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "# Create individual classifiers (Random Forest and Logistic Regression)\n",
    "rf_classifier = RandomForestClassifier(n_estimators=100, random_state=42)\n",
    "lr_classifier = LogisticRegression(solver='liblinear', random_state=42)\n",
    "\n",
    "# Create a list of classifiers for the Voting Classifier\n",
    "classifiers = [('Random Forest', rf_classifier), ('Logistic Regression', lr_classifier)]\n",
    "\n",
    "# Create a Voting Classifier using 'hard' voting (majority vote)\n",
    "voting_classifier = VotingClassifier(estimators=classifiers, voting='hard')\n",
    "\n",
    "# Create a pipeline with standardization and the Voting Classifier\n",
    "pipeline = Pipeline([\n",
    "    ('scaler', StandardScaler()),  # Standardize features\n",
    "    ('voting_classifier', voting_classifier)\n",
    "])\n",
    "\n",
    "# Fit the pipeline on the training data\n",
    "pipeline.fit(X_train, y_train)\n",
    "\n",
    "# Predict on the test data\n",
    "y_pred = pipeline.predict(X_test)\n",
    "\n",
    "# Evaluate the accuracy of the model\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(f'Accuracy: {accuracy:.2f}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e7b6655",
   "metadata": {},
   "source": [
    "#### Explanation of Steps:\n",
    "\n",
    "1. We load the Iris dataset, which is a commonly used dataset for classification.\n",
    "2. We split the dataset into training and testing sets.\n",
    "3. We create individual classifiers, namely a Random Forest Classifier and a Logistic Regression Classifier.\n",
    "4. We define a list of classifiers to be used in the Voting Classifier.\n",
    "5. We create a Voting Classifier using \"hard\" voting, which means it predicts the class label by majority vote.\n",
    "6. We create a pipeline that includes standardization (scaling features) and the Voting Classifier.\n",
    "7. We fit the pipeline on the training data.\n",
    "8. We predict on the test data using the pipeline.\n",
    "9. We evaluate the accuracy of the model using scikit-learn's accuracy_score function.\n",
    "\n",
    "The Voting Classifier combines the predictions of the individual classifiers (Random Forest and Logistic Regression) and makes a final prediction based on majority voting. This ensemble approach can improve predictive performance by leveraging the strengths of multiple classifiers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a60774b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}