Ensemble Model & Uplift Model

We mainly have two methods of combine different models. One is bagging, and the other method is boosting. The difference between these two methods are the way they combine model.

Bagging

Bagging treats each model equally. They get samples from dataset with replacement. And get results from all model to make final decision.

Random Forest is the example of Bagging

Random Forest

made up by decision trees.
Each samples had a tree. And all the trees classed new record.
final results are go with majority tree result
For example in right pics, there are 6 trees say should be 1 and 3 trees say should be 0. Based on majority rules. The result should be 1

Boosting

Using linear combination to combine all the models.
Increase the weight of those model who have higher accuracy, decrease model weight with those who have lower accuracy.
Example: Ada Boost , Gradient Boost

Ada Boost

Train all model with inital classifier and weight
Add wrong records with new sample, retrain the model and get the new classifier and weight.
Add all wrong recodes with new sample, get the final weight and classifier.

Gradient Boost

Based on Ada Boost, Gradient Boost not add misclassified record, instead, they are trying to model the error and add to the model.

They are trying to minimize the error in each node, and then combine all the model. The process is like graph below.

Pros and Cons

Pros

Useful when predictors are less correlated
Less variance and error, have better result than single model

Cons

Computation Large
In some situation may overfit the data
Blackbox method. Can not really understand relationship betweem target and predictors

Uplift model

Introduction

often used in marketing
to find out the probabilities of cutomer change after they being offered discount or other treatment.
Lift = P ( buy | Treatment ) - P ( buy | No Treatment )

Ideas behind Uplift model

Try to devide customers into 4 parts
Focus more on those persuadable customers. and less focus on sleeping dogs.

What Uplift Model do?

We know the matrix in the left. and we want to transfer left matrix to right matrix.

Page updated

Google Sites

Report abuse