Decision Tree

Now we will look at the results of all the experiments we did for predictive data mining.

Before Tuning

Experiment 1: Resampling only

60:40 Ratio

From the result below, the accuracy for Python is higher than RapidMiner which is 62.35%. RapidMiner was able to predict the financial level more precisely than Python. The total precison for Python is 62.25% and performance error of 37.65%. For Financial Distress prediction in RapidMiner is quite low which is 56.68% so the model was not able to predict correctly and precisely.

80:20 Ratio

From the result below, the accuracy for Python is higher than RapidMiner by 4.29%. Python was able to predict the financial level more precisely than RapidMiner.

Experiment 2: Resampling and Standardization

60:40 Ratio

From the result below, the accuracy for RapidMiner is higher than Python which is 60.49%. RapidMiner was able to predict the financial level more precisely and correctly than Python. Python model did not have too many wrong predictions compared to RapidMiner, especially for Financial Distress class.

80:20 Ratio

Based on the results, for 80:20 ratio, the accuracy for RapidMiner is higher than Python which is 59.92%. RapidMiner was able to predict the financial level more precisely than Python.

Experiment 3: Resampling, Standardization and Discretization

60:40 Ratio

From the results, the accuracy for Python is higher than RapidMiner which is 62.35%. Python is a better model in this case rather than RapidMiner. The precision for Naive Bayes model using Python is also quite good which is 64.16% total.

80:20 Ratio

For 80:20 ratio, RapidMiner is a better model than Python. This can be seen from the performance accuracy in which RapidMiner has higher percentage; 62.81%. RapidMiner has predicted more true prediction compared to Python.

After Tuning

Experiment 1: Resampling only

60:40 Ratio

For 60:40 ratio after tuning, Python has slightly higher performance accuracy which is 63.37% while RapidMiner has an accuracy of 62.14%. It is only a bit different but Python is still the best model for this experiment and ratio.

80:20 Ratio

From the result below, the accuracy for RapidMiner is better than Python with only small difference. RapidMiner has an accuracy of 62.81% while Python has an accuracy of 62.14%. The amount of true prediction for both Python and RapidMiner do not have much difference but RapidMiner is still slightly better.

Experiment 2: Resampling and Standardization

60:40 Ratio

From the results below, the accuracy for RapidMiner is higher than Python which is 62.35%. The model accuracy for Python is only 59.05%. hence, RapidMiner was able to predict the financial level more precisely than Python for both classes.

80:20 Ratio

Based on the results below, the accuracy for RapidMiner is higher than Python with only 2.73%. As we can see, RapidMiner was able to predict the financial level more precisely than Python.

Experiment 3: Resampling, Standardization and Discretization

60:40 Ratio

From 60:40 ratio, the accuracy for Python is higher than RapidMiner which is 63.79%. RapidMiner and Python has very small difference which is only 0.42% so, it does not that much difference in predicting the target. Hence, the precision in predicting is almost the same.

80:20 Ratio

based on the results below, the accuracy for RapidMiner is higher than Python which is 66.12%. RapidMiner was able to predict the financial level more precisely than Python for this ratio.

Based on the results above, the accuracies for every experiment using Naive Bayes are not that much different. For RapidMiner, the highest accuracy is 62.81% (Experiment 3 80:20) while for Python, the highest one is 65.43% (Experiment 1 60:40). So we did hyperparameter tuning using Grid Search to observe whether the accuracies increase significantly or remain unchanged. After tuning all the models, we learned that most of the performance accuracies did increase significantly. So, after tuning the models did a better job at predicting the financial level precisely. Hyperparameter tuning for this case is indeed successful.