Decision Tree

Now we will look at the results of all the experiments we did for predictive data mining.

Before Tuning

Experiment 1: Resampling only

60:40 Ratio

As we can see from the result below, the accuracy for Python is higher than RapidMiner by 1.24%. Python was able to predict the distress level more precisely than RapidMiner.

80:20 Ratio

As we can see from the result below, the accuracy for Python is higher than RapidMiner by 4.29%. Python was able to predict the distress level more precisely than RapidMiner. RapidMiner predicted the financial distress very low compared to Python, the class precision is only 55.90%.

Experiment 2: Resampling and Standardization

60:40 Ratio

Based on the result below, the accuracy for RapidMiner is higher than Python which is 61.32%. RapidMiner was able to predict the distress level more precisely than Python. However, the precision for Financial Distress is still low compared to Low Financial Distress which is around 57.14% only. The overall precision for Python is 63.06% which is not that bad either.

80:20 Ratio

Based on the result below, the accuracy for RapidMiner is higher than Python which is 59.92%. RapidMiner was able to predict the financial level more precisely than Python but the precision for Financial Distress prediction is still low compared to Low Financial Distress prediction which is 56.59% while the Low Financial Distress has a precision of 70.00%. The overall precision for Python is not that high, 59.68% so it was not able to predict precisely.

Experiment 3: Resampling, Standardization and Discretization

60:40 Ratio

From the result below, the accuracy for Pyhton is higher than RapidMiner which is 62.35%. Python was able to predict the financial level more precisely than Python.

80:20 Ratio

From the result below, the accuracy for RapidMiner is higher than Python which is 64.88%. RapidMiner was able to predict the financial level more precisely than Python. Class precision for Low Financial Distress in RapidMiner is very good which is 80.00% so it was able to predict most correct.

After Tuning

Experiment 1: Resampling only

60:40 Ratio

From the result below, the accuracy for RapidMiner is higher than Python which is 69.96%. RapidMiner was able to predict the financial level more precisely than Python.

80:20 Ratio

From the result below, the accuracy for RapidMiner is higher than Python which is 74.38%. RapidMiner was able to predict the financial level more precisely than Python. Class precision for Low Financial Distress in RapidMiner is very good which is 79.10% so it was able to predict most correct for Low Financial Distress.

Experiment 2: Resampling and Standardization

60:40 Ratio

From the result below, the accuracy for RapidMiner is higher than Python which is 68.72%. RapidMiner was able to predict the financial level more precisely than Python for both Low Financial Distress. Python only has 59.05% of accuracy overall.

80:20 Ratio

From the result below, the accuracy for RapidMiner is higher than Python which is 72.73%. So from the results below, RapidMiner was able to predict the financial level more precisely than Python.

Experiment 3: Resampling, Standardization and Discretization

60:40 Ratio

Based on the result below, the model accuracy for RapidMiner is higher than Python which by 4.93%. RapidMiner was able to predict the financial level more precisely than Python for both classes.

80:20 Ratio

From the result below, the accuracy for RapidMiner is way higher compared to Python which is 70.25%. Pyhton only has a 56.38% of performance accuracy. RapidMiner was able to predict the financial level more precisely than Python. Thus, for this experiment and ratio, RapidMiner is a better model than Python.

Based on all the results, the highest accuracy before tuning is only 68.31% while the lowest is 59.50%. Since the accuracy is not that high, we decided to try tuning it to see whether the accuracy would increase or not. So, we used Grid Search to tune the models. After tuning, we discovered that the most models' accuracies did increase significantly. Now the highest accuracy is 74.38% and the lowest accuracy is 66.26%. Overall, the hyperparameter tuning by using Grid Search did help the model to be more accurate and predicted more precisely than before.