Tim Salimans

Algoritmica, The Nethelands

Ten Lessons Learned From Ten Machine Learning Challenges

Challenges are an incredible tool for studying, researching, and teaching Machine Learning. Through their focus on a specific problem, their aim of achieving objectively measurable results, and the immediate feedback they give to competitors, they provide a very different experience compared to more traditional ways of engaging with Machine Learning. As a result, Machine Learning challenges can teach us new lessons that we might not have learned from only reading the academic literature…

In this talk I will share the lessons that I have learned from competing in 8 Machine Learning challenges, and from organizing 2 myself. Topics that I will discuss include the enormous effectiveness of combining models through stacking, the wide applicability and success of boosting ensembles of decision trees, the use of probabilistic graphical models and Bayesian methods versus performing empirical error minimization, and the importance of developing a good personal workflow for efficiently solving a prediction problem. Finally, I will discuss what it takes to organize a successful Machine Learning challenge, what opportunities to seize, and what pitfalls to avoid. Throughout I will be illustrating these lessons with my personal experiences in Machine Learning challenges.