Hands-On:
Neural Architecture Search with NASlib
Abstract
NASLib is a modular and flexible framework created with the aim of providing a common codebase to the community to facilitate research on Neural Architecture Search (NAS). It offers high-level abstractions for designing and reusing search spaces, interfaces to 25 benchmarks and evaluation pipelines, enabling the implementation and extension of state-of-the-art NAS methods with a few lines of code. The modularized nature of NASLib allows for easy innovation on individual components (e.g., define a new search space while reusing an optimizer and evaluation pipeline, or propose a new optimizer with existing search spaces). It is designed to be modular, extensible and easy to use. NASLib was developed by the AutoML Freiburg group and with the help of the NAS community, we are constantly adding new search spaces, optimizers and benchmarks to the library.
What you will learn:
The tutorial will provide an introduction to various functionalities of NASLib. The participants will learn how to use different optimizers (black box and one-shot), performance predictors and benchmarks in NASLib and how to write new ones.
Bio
Rhea Sukthanker is a PhD student at the Machine Learning Lab of the University of Freiburg, under the supervision of Frank Hutter. Her research is mainly focused on the topic of Neural Architecture Search (NAS) specifically one-shot NAS for hierarchical search spaces, transformers and discovery of qualitatively new architectures. Before starting her PhD, she completed her Master’s degree at ETH Zurich. During her masters studies she worked at the Computer Vision Lab where her research was mainly focused on NAS for Computer Vision applications.
Arjun Krishnakumar is an incoming Research Engineer at the Machine Learning Lab of the University of Freiburg. Before this he completed his masters thesis at the AutoML lab, supervised by Arber Zela. His main area of interest is Neural Architecture Search, specifically one-shot optimizers. He has been working part time as a student research assistant at the Machine Learning Lab of the University of Freiburg, led by Prof Frank Hutter, since September 2020, and is a co-author of NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy, which was accepted at ICLR 2022.