Supervised Machine Learning Models

Project Title: Spam Email Classification Using K-Nearest Neighbors

Problem Statement

The goal of this project is to classify emails as spam or not spam using the K-Nearest Neighbors (KNN) algorithm. This is a common problem in the field of email filtering and cybersecurity, where identifying and segregating spam emails is crucial for maintaining user productivity and protecting against malicious content.

Dataset Information

The dataset used in this project consists of emails with various attributes that indicate their content and metadata. Each email is labeled as either 'spam' or 'not spam'. The attributes include features extracted from the email content, such as the frequency of certain words and characters.

Steps Taken: 

1. Import Libraries: Necessary libraries such as pandas for data manipulation, seaborn and matplotlib for data visualization, and sklearn for machine learning algorithms and evaluation metrics were imported.

2. Load and Prepare Data: The training and testing datasets were loaded from CSV files. Features (X) and labels (y) were separated. The features were extracted using a regex filter to identify columns with numerical data.

3. Train the KNN Model: The KNN model was initialized and trained using the training dataset. GridSearchCV was used for parameter tuning to find the optimal number of neighbors and other hyperparameters.

4. Evaluate the Model: The model's performance was evaluated using accuracy metric. Visualizations were created to understand the model's performance better.

5. Optimize the Model: The challenge in training a kNN model is to determine the optimal number of neighbors. To find the optimal parameters, GridSearchCV object can be used.

6. Test the Model on Test Dataset: In this phase, we'll evaluate the accuracy of the trained kNN model on the test set. 

7. Evaluate the Model: A good evaluation measure is the confusion matrix that gives the fraction of true positives, true negatives, false positives, and false negatives. Calculate mean accuracy on test dataset

Tools Used- Python, Jupyter Notebook, Pandas, Scikit-learn, Machine Learning, Supervised Learning

Project 2: Implementing Linear Regression, Random Forest and XGBoost to determine best training model 

Project Overview

Introduction:

I worked on this project to explore the process of selecting the appropriate machine learning tool for a given dataset. With numerous options available, such as regression, decision trees, and random forests, the challenge was to determine which model would yield the best results. This dilemma often confused me when I first started into the world of machine learning.

Problem Statement

The objective of this project was to train a machine learning model to predict home prices in California. The goal was to accurately estimate the value of houses using nine different features and one target variable. This project aimed to identify the most effective machine learning algorithm for this regression problem.

Dataset Information

The dataset used in this project contained various features that influence house prices. The features included aspects such as the number of rooms, location, square footage, and other relevant attributes. The target variable was the actual price of the houses.. What I discovered was that different learning algorithms gives me different results on the same dataset. It's crazy, but it's true. XGBoost turned out to be the superstar. Results were as below:

Linear Regression with 56% accuracy 🥉

Random Forest Regressor with 75% accuracy 🥈

XGBoost is the winner with 78% accuracy 🏆

Steps Taken

Understanding the Dataset:. Loaded the dataset and performed an initial exploration to understand the distribution and relationships between features.

EDA- Conducted data cleaning, including handling missing values and outliers. Visualized the data to gain insights into correlations and patterns.

Feature Selection and Feature Importance: It is the process of finding the most important features in the dataset to help predict target. Each feature or column in the dataset impacts the final prediction. Some more than others. I did this on random forest regressor learning algorithm where we have this attribute called "feature importances" that tells the relevance of the features. Using Feature Importance I could reduce the complexity of the model and better explain why my model makes certain predictions.

Model Selection:

Split the dataset into training and testing sets to evaluate the model's performance.

Linear Regression: Implemented a basic linear regression model to establish a baseline for comparison.

Decision Trees: Experimented with decision tree regression to capture non-linear relationships in the data.

Random Forests: Utilized random forest regression to improve performance by averaging multiple decision trees and reducing overfitting.

HyperParameter Tuning- XGBoost is a tunable algorithm unlike linear regression with several hyper parameters. 

Good hyper parameter values can be found by trial and error for a given data set. I found configuring below hyper parameters for the XGBoost learning algorithm improved accuracy of the model to some extent-

1. n_estimators. 

2. max depth. 

3. learning rate. 

4. sub-sample. 

5. column sample

Model Evaluation:

Assessed the performance of each model using metrics such as Root Mean Squared Error (RMSE). Compared the models to determine which provided the most accurate predictions for house prices.

Tools Used- Python, Jupyter Notebook, Pandas, Scikit-learn, Machine Learning, Supervised Learning, Regression