Hands-On:
Next Generation AutoML Methods
Abstract
We will have a hands-on tutorial on the next generation of AutoML methods.
First, we will train our own prior-data fitted networks. These are Bayesian models that show potential to be a cornerstone of models for small datasets, as shown with the TabPFN. The TabPFN, one particular PFN, is a model which is able to match performance of much more complicated and >100x slower methods on small datasets. We will also try it out to see what works and what does not.
Finally, we will look into prompting LLMs and interacting with LLMs to do feature preprocessing for us, based on our initial work on this, CAAFE.
Bio
Samuel Müller is a final year PhD student in Frank Hutter's machine learning lab at the University of Freiburg. Before joining the lab's team, he did research work at DeepL and engineering work at Amazon. He studied at ETH Zürich for his bachelor's degree and Cambridge University for his master's degree.
His research is about simplicity. Replacing complicated, engineered methods with simpler more general methods that make good use of computational resources.