Introduction

1. what tree doing?

• Classification

• Predictions

• Feature Selection

2. Idea behind Trees

• Recursive partitioning (Growing Tree)

• Pruning (Cut Tree )

• dichotomy

3. Terminology

• Decision Node : node made split

• Terminal Node : the node made conclusion

Classification Example

We use iris dataset here.

And first we only consider sepal features. We apply the decision tree in max depth = 2. And we got following result . Then we reflect the result to the scatter plot.

So, basically we can see this algorithms try to split data set into several parts. And they will assign the most majority class into the data belongs to that kind.

Regression Example

Idea is same here. But we get the result of predictions by using the group mean.

Growing Tree

Information Gain Criteria :

• Entropy : How different everything between groups

  • near 1 high entropy - high different between groups

  • near 0 low entropy - similar obs between groups

• Gini : How similar data with groups

  • near 0 within class similar

  • near 1 within class different

Pruning Tree

Why need cutting tree

• Avoid Overfitting

How to choose best cut tree

• Cost Complexity (CCT)

  • • CCT = R + α L

    • R : misclassification rate

    • L : number of leaves

    • α : complexity parameter

Model Performances

This part is similar in Logistic Regression. We are not going focus on this part.

1. Accuracy Rate

2. Confusion Matrix, sensitivity and specificity

3. ROC and AUC.

Model Using

• Classification / Prediction

• Feature importances

  • relative importances in building tress.

  • help us choose predictors in some situations

Pros and Cons