Embedded Files
Overview:
Naïve Bayes (NB) is a family of supervised probabilistic classifiers based on applying Bayes' Theorem with the "naïve" assumption that every feature is conditionally independent of every other feature given the class label. Despite this simplifying assumption rarely being true in practice, NB models are remarkably effective, fast to train, and perform well even with small datasets. They are widely used in text classification, spam filtering, sentiment analysis, and medical diagnosis.
Sklearn offers four primary flavors of Naïve Bayes, each suited for different data types:
Gaussian NB (GNB): Assumes features follow a continuous normal (Gaussian) distribution. Best used when features are real-valued and roughly bell-shaped, such as GDP per capita or military spending percentages.
Multinomial NB (MNB): Designed for discrete count data. Classically used in text classification (word counts), but also works for any non-negative integer feature counts. This is the most commonly required flavor in assignments and industry.
Bernoulli NB (BNB): Assumes all features are binary (0 or 1). Well-suited for one-hot encoded data or boolean features, such as "did this republic have a conflict? Yes/No."
Categorical NB (CNB): Designed for purely categorical (nominal) features where each feature has a fixed set of possible values. This is the correct choice when working with binned labels like "GDP_High", "GDP_Med", "GDP_Low."
Data Prep:
The dataset used for Naïve Bayes classification was derived from the merged SIPRI/World Bank dataset. A binary label called "Performance" was engineered based on whether a given country-year observation exceeded the median GDP per capita across all republics and all years, producing two balanced classes: High Performer (1) and Low Performer (0). Features include military expenditure as a percentage of GDP, year, encoded historical period, and encoded region. The data was split into an 80% Training Set and a 20% Testing Set using stratified random sampling. These sets must be kept disjoint, meaning the model is trained on data it has never seen during testing to ensure the accuracy metric reflects true generalization rather than memorization.
Why Train-Test Split?
Every supervised model in this project was trained on 80% of the data and evaluated on the remaining 20%, using train_test_split(X, y, test_size=0.2, random_state=42). The two sets must remain completely disjoint meaning no observation can appear in both sets simultaneously. If the model were allowed to evaluate itself on data it already learned from, the resulting accuracy would reflect memorization rather than generalization to unseen data, making it scientifically meaningless. Setting random_state=42 ensures the exact same split is reproducible by anyone attempting to replicate the results. The training set is used exclusively to fit model parameters, while the test set acts as a simulation of real-world unseen data.
Link to Data
https://github.com/rileythejones/-CUBoulder-DS-CSCI-5612-Project/tree/main/Project%20Module%20Three
Code
All three Naïve Bayes models were implemented in Python using the scikit-learn library. The code loads the prepared labeled dataset, performs an 80/20 train-test split, and applies three distinct NB flavors sequentially on the same data. Gaussian NB was applied to the StandardScaler normalized feature matrix, as it assumes continuous normally distributed features. Multinomial NB was applied to a non-negative shifted version of the raw feature matrix, as it requires discrete non-negative input values. Bernoulli NB was applied to a binarized version of the scaled data, converting each feature to a 0 or 1 based on whether it exceeded the mean. Accuracy scores and confusion matrices were computed for each model using the held-out test set exclusively.
Link to Code
https://github.com/rileythejones/-CUBoulder-DS-CSCI-5612-Project/tree/main/Project%20Module%20Three
Results
Gaussian and Multinomial NB tied at 67.53%, while Bernoulli NB slightly underperformed at 66.23%. All three models meaningfully outperform a random baseline of 50% (since this is a binary classification task), which confirms that real, learnable structure exists in the data. The fact that Gaussian and Multinomial NB tied suggests that the features carry the same directional signal regardless of whether the model treats them as continuous or discrete. Bernoulli NB's slightly lower score is expected forcing continuous values like GDP and military spending into binary on/off switches inevitably discards nuance.
These accuracy scores are modest but appropriate for this dataset. With only 15 countries and roughly 400 observations, Naïve Bayes is limited by the small sample size and the strong regional correlation in the data something a richer dataset would address. The confusion matrices above reveal that the models perform best at identifying "Low Performers," and are more uncertain when classifying "High Performers," which tend to be the smaller Baltic states with fewer total observations.
Model Accuracy
Gaussian NB 67.53%
Multinomial NB 67.53%
Bernoulli NB 66.23%
Gaussian NB: The confusion matrix shows the model correctly classified 24 Low Performers and 28 High Performers, while misclassifying 25 (20+5) observations. The off-diagonal cells represent errors, i.e. cases where the model predicted the wrong class.
Multinomial NB: The confusion matrix shows the model correctly classified 30 Low Performers and 22 High Performers, while misclassifying 35 (11+24) observations. The off-diagonal cells represent errors, i.e. cases where the model predicted the wrong class.
Bernoulli NB: The confusion matrix shows the model correctly classified 28 Low Performers and 23 High Performers, while misclassifying 26 (10+16) observations. The off-diagonal cells represent errors, i.e. cases where the model predicted the wrong class.
Conclusions
The Naïve Bayes results carry a somewhat clear message: even a simple probabilistic model, given only four features, can predict whether a post-Soviet republic was thriving or struggling with roughly two-thirds accuracy. The strongest predictors appear to be Region and Period, not military spending alone, which suggests that structural geography being a Baltic state versus a Central Asian republic was nearly deterministic of outcome. The relatively modest accuracy (~67%) also reflects a genuine historical truth: the transition was messy, and some Central Asian countries (such as Kazakhstan) punched above their expected weight due to oil wealth, while some Eastern European states (like Moldova) underperformed despite geographic proximity to successful reformers. A model that was 100% accurate would actually be suspicious it would suggest the transition had a simple, universal explanation. The roughly 33% misclassification rate is the data's way of acknowledging the complexity of the collapse.
Page updated
Google Sites
Report abuse