Embedded Files
Growing up, I was a fan of Boost (the drink) and now I’m a fan of boosting my model’s performance!
Growing up, I was a fan of Boost (the drink) and now I’m a fan of boosting my model’s performance!
What is Boosting?
What is Boosting?
Imagine you're learning to swim. Instead of learning everything at once, you start with small steps:
Imagine you're learning to swim. Instead of learning everything at once, you start with small steps:
first floating, then kicking, and finally combining all the skills to become a proficient swimmer.
first floating, then kicking, and finally combining all the skills to become a proficient swimmer.
That’s boosting in a nutshell!
That’s boosting in a nutshell!
How Does It Work?
How Does It Work?
In data science, boosting is an ensemble technique where models are trained sequentially.
In data science, boosting is an ensemble technique where models are trained sequentially.
Each new model focuses on the errors made by the previous ones, gradually improving the overall performance.
Each new model focuses on the errors made by the previous ones, gradually improving the overall performance.
By combining the strengths of multiple weak models (models that perform slightly better than random guessing), boosting creates a strong predictive model.
By combining the strengths of multiple weak models (models that perform slightly better than random guessing), boosting creates a strong predictive model.
Picture a swimming coach (boosting algorithm) who helps you learn to swim in stages.
Picture a swimming coach (boosting algorithm) who helps you learn to swim in stages.
The coach first teaches you to float (initial weak model).
The coach first teaches you to float (initial weak model).
Once you master floating, the coach notices you struggle with kicking, so the next lesson focuses on kicking (the next weak model focuses on errors).
Once you master floating, the coach notices you struggle with kicking, so the next lesson focuses on kicking (the next weak model focuses on errors).
Finally, you learn to coordinate all movements, becoming a proficient swimmer (final strong model).
Finally, you learn to coordinate all movements, becoming a proficient swimmer (final strong model).
In technical terms, boosting algorithms like AdaBoost and Gradient Boosting iteratively add models to correct the mistakes of the previous ones.
In technical terms, boosting algorithms like AdaBoost and Gradient Boosting iteratively add models to correct the mistakes of the previous ones.
Each new model is trained on the residual errors (the difference between the actual and predicted values) of the combined ensemble so far, leading to a strong overall model.
Each new model is trained on the residual errors (the difference between the actual and predicted values) of the combined ensemble so far, leading to a strong overall model.
But here's the catch - your model becomes so good that it overfits!
But here's the catch - your model becomes so good that it overfits!
How do you overcome that?
How do you overcome that?
Dive into boosting and watch your models soar!
Dive into boosting and watch your models soar!
Get in touch at jain.van@northeastern.edu
Get in touch at jain.van@northeastern.edu
Page updated
Google Sites
Report abuse