Bio: I am a professor for Visual Computing at the University of Siegen and an affiliated senior researcher for robust visual learning at the Max-Planck-Institute for Informatics in Saarbrücken. My research focus is at the intersection between Computer Vision and Machine Learning. Specifically, I am interested in building robust and well-performing computer vision models, the relationship between the model architecture and its robustness and in neural architecture search. Before joining the University of Siegen, I was a Juniorprofessor for Computer Vision at the University of Mannheim. I am a member of the ELLIS society.

Abstract: Recently, quite some effort has been dedicated to optimize neural architectures to facilitate ever improved model accuracy. The resulting models are often highly accurate in practice when the data used at test time is sufficiently similar to the training data. However, many state-of-the-art models are highly susceptible to even small domain shifts such as slight data corruptions, which can hamper their deployment in practical settings. As a result, the joint optimization of neural architectures for high accuracy on clean data as well as on data that has undergone some unknown domain shift (usually referred to as robust accuracy) becomes an important research question. It is different from established multi-objective neural architecture search (NAS) approaches such as Hardware aware NAS, (i) because the objective “clean accuracy” and “robust accuracy” are highly entangled. The robust accuracy of a model is usually bounded by its clean accuracy, or, in other words, its high accuracy on clean data is a necessary condition for a reasonable robust accuracy. (ii) A model’s robust accuracy depends on its architecture as well as on the training procedure, such that an architecture that can be trained to perform well under diverse conditions does not necessarily show this behavior when trained conventionally. In this talk, we will discuss multi-objective NAS and its relationship to improving model robustness, and the expected pitfalls when searching for best models to be deployed in practice.

Bio: José Miguel is Professor of Machine Learning at the Department of Engineering in the University of Cambridge, UK. Before joining Cambridge as faculty, he was a postdoctoral fellow in the Harvard Intelligent Probabilistic Systems group at Harvard University, working with Ryan Adams, and before this, also a postdoctoral research associate in the Machine Learning Group at the University of Cambridge (UK), working with Zoubin Ghahramani. Jose Miguel completed his Ph.D. and M.Phil. in Computer Science at the Computer Science Department from Universidad Autónoma de Madrid (Spain), where he also obtained a B.Sc. in Computer Science from this institution, with a special prize to the best academic record on graduation. José Miguel's research focuses on probabilistic machine learning, with a particular interest in deep generative models, Bayesian optimization, approximate inference, Bayesian neural networks and applications of these methods to real-world problems such as data compression and molecular modeling.

Abstract: Machine learning training methods depend plentifully and intricately on hyperparameters, motivating automated strategies for their optimisation. Many existing algorithms restart training for each new hyperparameter choice, at considerable computational cost. Some hypergradient-based one-pass methods exist, but these either cannot be applied to arbitrary optimiser hyperparameters (such as learning rates and momenta) or take several times longer to train than their base models. In this tutorial, I will introduce hypergradient-based methods for hyper-parameter tuning and will extend existing methods to develop an approximate hypergradient-based hyperparameter optimiser which is applicable to any continuous hyperparameter appearing in a differentiable model weight update (such as parameter-specific learning rates or momentum), yet requires only one training episode, with no restarts. I will also provide a motivating argument for convergence to the true hypergradient, and perform tractable gradient-based optimisation of independent learning rates for each model parameter. The resulting method performs competitively from varied random hyperparameter initialisations on several UCI datasets and Fashion-MNIST (using a one-layer MLP), Penn Treebank (using an LSTM) and CIFAR-10 (using a ResNet-18), in time only 2-3x greater than vanilla training.

Bio: Haitham leads Huawei London research centre's reinforcement learning, Bayesian optimisation, and multi-agent system teams and is an Honorary Lecturer at UCL. Before joining Huawei, Haitham held post-doctoral researcher positions at Princeton University and the University of Pennsylvania. His primary research focuses on Bayesian optimisation, reinforcement learning and robotics. He won numerous awards, including the NeurIPS 2020 black-box optimisation challenge, the Conference on Robot Learning (CoRL) Best Paper Systems Paper Award 2020, Huawei Individual Performer Gold Medal 2020, IROS Best Paper Finalist 2021, and the AAMAS Best Blue Sky Paper Award 2021, among others.

Abstract: Sample efficient black-box optimisation is at the heart of many machine learning problems, including but not limited to hyper-parameter tuning, chip design, and antibody discovery. The sheer expense of querying such black boxes challenges the standard big-data paradigm and demands adequate sample efficient solutions. With such applications in mind, this talk will delve into Bayesian optimisation (BO), covering introductory and advanced topics. On the elementary side, we will elaborate on standard BO detailing advances to HEBO - a state-of-the-art algorithm that won the NeurIPS 2020 black-box optimisation challenge. On the advanced side, we will discuss the latest techniques from high-dimensional and meta-BO that allow for further transfer between black-box problems by utilising neural processes. 

Bio: Alexander Tornede is a researcher in the field of Automated Machine Learning (AutoML). He completed his Master's degree with a focus on machine learning at Paderborn University in 2018 and recently defended his Ph.D. under the supervision of Prof. Dr. Eyke Hüllermeier, focusing on Algorithm Selection. Currently, Alexander is a Postdoctoral Researcher in the research group led by Prof. Dr. Marius Lindauer at Leibniz University Hannover, which is part of the automl.org supergroup. Since joining the group in September 2022, he has been actively involved in advancing interactive and explainable AutoML and researches on integrating large language models (LLMs) with AutoML. Furthermore he leads the SMAC development team. In addition to his research endeavors, Alexander serves as one of three general chairs of COSEAL, an international group of researchers dedicated to Algorithm Selection and Configuration and was involved in the organization of the AutoML conferences  2022 and 2023.

Bio: Prof. Dr. Marius Lindauer has been a professor of machine learning at the Leibniz University Hannover since 2019. He received his PhD from the University of Potsdam (Germany) in 2015 under Prof. Dr. Thorsten Schaub and Prof. Dr. Holger Hoos. From 2014 to 2019, he was PostDoc and later junior research group lead at the University of Freiburg under Prof. Dr. Frank Hutter. Besides being a member of ELLIS, CLAIRE, and the platform learning systems (PLS), he is one of the co-heads of automl.org and co-founder of the research network COSEAL, the AutoML conference and the institute of AI at the Leibniz University Hannover. He won several competitions, including SAT solving, ASP solving, and automated machine learning. He gave several invited talks, including at the open data science conference in London and a TEDx talk, and tutorials at renowned AI conferences and summer schools, such as AAAI, IJCAI and ESSAI. In 2022, he was awarded an ERC starting grant, the most prestigious research grant for European young researchers. His research interests include automated machine learning, Bayesian optimization, neural architecture search, interpretability, and reinforcement learning.

Abstract: The fields of both Natural Language Processing (NLP) and Automated Machine Learning (AutoML) have achieved remarkable results over the past years. In NLP, especially Large Language Models (LLMs) have experienced a rapid series of breakthroughs very recently. We envision that the two fields can radically push the boundaries of each other through tight integration. To showcase this vision, we explore the potential of a symbiotic relationship between AutoML and LLMs, shedding light on how they can benefit each other. In particular, we investigate both the opportunities to enhance AutoML approaches with LLMs from different perspectives and the challenges of leveraging AutoML to further improve LLMs.

Bio: Philipp Stratmann is a researcher at Intel Labs, where he explores new neural network architectures for Intel’s research AI accelerator Loihi. Before this, he has worked as a research associate at the Institute for Robotics and Mechatronics of the German Aerospace Center (DLR) and as an intern at NASA’s Jet Propulsion Lab. He holds a PhD in Computer Science from the TU Munich, a MSc in Neuroscience from the University of Oxford, and a BSc in Physics from RWTH Aachen University.

Abstract: In their initial development stages, artificial neural networks (ANN) drew close inspiration from the human brain. Over time, deep learning (DL) has enabled modern computers to surpass the brain's performance in specific tasks, such as board games. However, the human brain still excels in numerous areas, like motor control, while consuming a fraction of the energy required by a single GPU. Neuromorphic computing aims to extend the inspiration from the brain to the design of chips that support spiking neural networks (SNNs), which feature memory-compute integration, fine-grain parallelism, and event-based computing.

This talk will introduce neuromorphic computing using Intel's neuromorphic research processor Loihi as an illustrative example, and unveil methods for translating artificial neural networks (ANNs) into SNNs. Drawing on five years of research from the 180 research institutes in Intel’s Neuromorphic Research Community, the presentation will categorize computations where SNNs have demonstrated orders of magnitude lower latency and energy consumption than state-of-the-art conventional approaches. 

Bio: I am a statistician and ML researcher currently working on improving blood cancer diagnostics at Hemato. Before I transitioned into industry, I completed a Ph.D. at LMU Munich. My research interests center around AutoML, Algorithmic Fairness, model evaluation, and Benchmarking. Specifically, I care about democratizing machine learning and making sure that users who build systems based on (automated) machine learning do so responsibly and to the benefit of all of humanity.

Abstract: In this session, I want to introduce basic concepts in algorithmic fairness that are relevant for machine learning researchers when engaging in work that includes building models whose decisions affect humans. More precisely, I will focus on harms that originate from the use of automated decision-making systems, biases that might be included in such systems, and approaches toward mitigating or reducing the risk of such systems. In the second part, I want to focus on the intersection between Fairness and AutoML with the goal of discussing how fairness can be approached in AutoML, what risks might arise from (over-) simplifying problems and resulting solutions, and what role AutoML can and should play in fairness contexts.