NAIVE BAYES 

A basic and popular machine learning algorithm for classification tasks is Naive Bayes (NB). Its foundation is Bayes' Theorem, which determines a class's probability based on observed data. The premise of feature independence, which regards each feature as independent of the others, is a fundamental characteristic of NB. Even though this presumption might not always hold true with actual data, it streamlines calculations and makes NB incredibly scalable and efficient, particularly for big datasets. NB is a well-liked option for resolving issues including text classification, spam detection, sentiment analysis, and recommendation systems because of its ease of use and efficiency. 

The Naive Bayes algorithm has several variations tailored to different types of data:

1. Multinomial Naive Bayes (MNB):

   • Best suited for discrete data, particularly when features represent counts or frequencies.

   • Commonly used in text classification tasks, where features correspond to the frequency of words or terms in a document. For example, it is widely applied in spam detection by analyzing word occurrences in emails.

   • Assumes non-negative values and models the data using a multinomial distribution.

2. Gaussian Naive Bayes (GNB):

   • Designed for continuous data and assumes the data follows a Gaussian (normal) distribution.

   • It is often applied in scenarios where features are numerical, such as datasets involving measurements, sensor data, or metrics.

   • This variation estimates the likelihood of features belonging to a class using the probability density function of the Gaussian distribution.

3. Bernoulli Naive Bayes (BNB):

   • Suitable for binary data, where features take on values of 1 or 0, indicating the presence or absence of a particular characteristic.

   • This model is frequently used for tasks where binary occurrence matters, such as determining whether specific words are present in a document or not (e.g., keyword detection).

Each variation of Naive Bayes is adapted to the specific structure and type of data, making it a highly versatile and flexible algorithm. Despite its simplicity, NB is known for its robustness, especially when dealing with high-dimensional datasets. It performs well even when the independence assumption is violated to some extent, as long as the class probabilities are reasonably separable.

In summary, Naive Bayes is a powerful tool for solving classification problems due to its computational efficiency, ease of implementation, and adaptability to different data types. Its variations allow practitioners to apply the algorithm across diverse domains, tailoring the model to the nature of the dataset for optimal results.

Dataset Description 

The dataset used in this analysis contains important features related to coal shipments, such as:

• ash-content: A measure of the ash residue in coal.

• heat-content: The heat energy produced when coal is burned.

• price: The price of coal shipments.

• quantity: The amount of coal shipped.

• sulfur-content: A measure of sulfur levels in coal.

A new label, carbon_intensity, was created to classify shipments into three categories: High, Medium, or Low. The classification is based on specific thresholds for sulfur content, ash content, and shipment quantity, reflecting the carbon intensity of the shipments:

• High: Sulfur-content > 2.0, ash-content > 10.0, and quantity > 1,000,000.

• Medium: Sulfur-content > 1.0, ash-content > 5.0, and quantity > 500,000.

• Low: All other shipments.

LINK TO THE DATASET 

Training and Testing Split

The dataset was divided into training and testing subsets to enable unbiased evaluation of the models. The split ratio used was 70% for training and 30% for testing. This ensures that the models are trained on one portion of the data and evaluated on an unseen portion, which prevents overfitting and provides an accurate measure of performance.

Training and Testing Dataset Statistics

• Training Set: 70% of the data

• Testing Set: 30% of the data

• Features: ash-content, heat-content, price, quantity, sulfur-content

• Label: carbon_intensity

IMPORTANCE OF CREATING A DISJOINT SPLIT

1. Prevention of Overfitting:

• When a model is trained on a dataset, it learns patterns and relationships specific to that data.

• If the same data is used for both training and evaluation, the model's performance might appear artificially high because it has already "seen" the test data during training. This is called data leakage and leads to overfitting.

• A disjoint split ensures that the model is tested on completely unseen data, giving a realistic estimate of how it will perform on new, unseen data in the real world.

2. Assessment of Generalization:

• The ultimate goal of a machine learning model is to generalize well to data it has never encountered before.

• By using a disjoint test set, you simulate this real-world scenario and evaluate the model's ability to predict accurately on data outside its training experience.

3. Validation of Model Robustness:

• A model trained and tested on overlapping data might inadvertently memorize specific features or outliers unique to the training set, leading to poor performance on new data.

• A disjoint split ensures the test set serves as an unbiased benchmark, allowing you to identify whether the model's predictions are based on learned patterns or coincidental noise in the training data.

LINK TO THE CODE 

RESULTS

Overview

The results from the three Naive Bayes models—Multinomial Naive Bayes (MNB), Gaussian Naive Bayes (GNB), and Bernoulli Naive Bayes (BNB)—demonstrate varying levels of accuracy and performance based on their suitability for the dataset and the type of data used. This section discusses the results, compares the models, and visualizes the outcomes through confusion matrices and accuracy scores.

SUMMARY

CONCLUSION

The Naive Bayes models provided valuable insights into the classification of carbon intensity levels within the coal shipment dataset:

1. Model Performance:

   • The Gaussian Naive Bayes model achieved the highest accuracy of 69.41%, indicating it is the most suitable for the continuous numerical features in the dataset. This suggests that the distribution of the features aligns well with the Gaussian assumption.

   • The Bernoulli Naive Bayes model achieved an accuracy of 60.05%, performing moderately well when the data was binarized, but its assumptions of binary data limited its effectiveness.

   • The Multinomial Naive Bayes model achieved the lowest accuracy of 48.44%, highlighting its limitation when applied to continuous data, even after handling negative values.

2. Insights into Carbon Intensity:

   • The Gaussian NB model was able to effectively differentiate between the "High," "Medium," and "Low" carbon intensity categories, suggesting that features like sulfur content, ash content, and quantity strongly influence these classifications.

   • The confusion matrices revealed that "Medium" intensity levels were more frequently predicted accurately, while some misclassifications occurred for "High" and "Low" levels, likely due to overlapping feature distributions.

3. Predictive Applications:

   • These models can be used to predict the carbon intensity of future coal shipments based on their properties, enabling proactive decision-making to reduce carbon-intensive shipments.

   • The insights can also help stakeholders in the energy sector identify patterns and trends, such as which shipment characteristics are most strongly associated with high carbon intensity.

4. Limitations and Future Work:

   • While Gaussian NB performed well, further improvements could involve incorporating feature engineering to better capture non-linear relationships or trying other classification models like decision trees or ensemble methods for comparison.

   • Expanding the dataset or incorporating external factors, such as geographic location or supplier information, could also enhance the models' predictive capabilities.

In conclusion, Naive Bayes modeling provided a foundation for understanding the dataset and its classification potential, paving the way for more advanced analysis and actionable insights related to carbon intensity in coal shipments.

FINAL THOUGHTS

The Naive Bayes models provided a foundational understanding of carbon intensity classification in coal shipments. The Gaussian Naive Bayes model demonstrated the best performance, highlighting its suitability for continuous numerical features such as sulfur content, ash content, and shipment quantity. These insights offer actionable opportunities for optimizing shipment processes to reduce emissions and prioritize sustainability. While the models were effective in identifying patterns, further improvements through advanced modeling techniques, feature engineering, and expanded datasets could enhance accuracy and provide deeper insights. This analysis serves as a stepping stone toward data-driven strategies to address carbon intensity in the energy sector.