Embedded Files
Day 1
May 23, Thursday
May 23, Thursday
9.30 -10.20
Prof Sven Schewe
Title: Automata for Profit and Pleasure
Abstract: What could be greater fun than toying around with formal structures? One particularly beautiful structure to play with are automata over infinite words, and there is really no need to give any supporting argument for the _pleasure_ part in the title. But how about profit?
When using ω-regular languages as target languages for practical applications like Markov chain model checking, MDP model checking and reinforcement learning, reactive synthesis, or as a target for an infinite word played out in a two player game, the classic approach has been to first produce a deterministic automaton D that recognises that language. This deterministic automaton is quite useful: we can essentially play on the syntactic product of the structure and use the acceptance mechanism it inherits from the automaton as target. This is beautiful and moves all the heavy lifting to the required automata transformations.
But when we want even more profit in addition to the pleasure, the question arises whether deterministic automata are the best we can do. They are clearly good enough: determinism is as restrictive as it gets, and easily guarantees that one can just work on the product. But what we really want is the reverse: we want an automaton, so that we can work on the product, and determinism is just maximally restrictive, and therefore good enough for everything.
We can lift quite a few restrictions and instead turn to the gains we can make when we focus on the real needs of being able to work on the product. For Markov chains, this could be unambiguous automata, for MDPs this could be good-for-MDP automata, and for synthesis and games, it could be good-for-games automata.
We will shed a light on a few nooks and corners of the vast room available open questions and answers, with a bias MDPs analysis in general and reinforcement learning in particular.
10.20 -11.10
Dr Bettina Konighofer
Title: Safe Reinforcement Learning via Shielding (Formal Runtime Enforcement for RL)
Abstract: This presentation focuses on the concept of shielding. Shields are correct-by-construction runtime enforcers that guarantee safe execution automatically correcting any actions that could potentially breach a formal safety specification. Our focus will be on shields that ensure safe exploration for deep reinforcement learning agents. We will explore how shields can be computed for environments that inherit both probabilistic and adversarial behaviour. Furthermore, we will examine recent advances in shielding, including shields that are robust under delayed input observation and shielding under partial observability. Finally, we will discuss recent automata learning approaches that are capable of deriving compact probabilistic models for high-dimensional environments, which can be utilized to compute shields.
Coffee Break 11.10 - 11.30
11.30 - 12.20
Dr Oliver Sutton
Title: Can robustness to adversarial attacks be certified for classifiers learning high dimensional data?
Abstract: An adversarial attack is simply a small change to a piece of input data which produces a large change in a model's output, a common yet undesirable instability found in modern classifiers such as neural networks. An intriguing phenomenon, observed in practice, is that this same instability is not triggered by random corruptions to the input data. We investigate this ‘paradox of apparent stability’ in modern neural networks, and identify a simple comprehensible framework which explains how this behaviour manifests itself. The investigation also reveals that widely used families of techniques for certifying that a classifier is robust against adversarial instabilities are not practically able to do so, despite being prohibitively computationally expensive, especially with high dimensional data. This is because they work by using random perturbations as a proxy for adversarial perturbations. Furthermore, we reveal that there exist large families of well-behaved tasks where neural networks learned by empirical risk minimisation may be unstable to almost any small perturbation on nearly half of the training and validation data -- even though a stable network exists with the same architecture, potentially with arbitrarily similar weights! These results show that there are fundamental difficulties in learning stable classifiers, and that it may not be possible to rigorously understand this problem while working in a conventional distribution-agnostic framework.
Lunch 12.20 - 14.00
14.00 - 14.50
Dr Dominik Wojtczak
Title: Recursive Reinforcement Learning
Abstract: Recursion is the fundamental paradigm to finitely describe potentially infinite objects. As state-of-the-art reinforcement learning (RL) algorithms cannot directly reason about recursion, they must rely on the practitioner's ingenuity in designing a suitable "flat" representation of the environment. The resulting manual feature constructions and approximations are cumbersome and error-prone; their lack of transparency hampers scalability. To overcome these challenges, we develop RL algorithms capable of computing optimal policies in environments described as a collection of Markov decision processes (MDPs) that can recursively invoke one another. Each constituent MDP is characterized by several entry and exit points that correspond to input and output values of these invocations. These recursive MDPs (or RMDPs) are expressively equivalent to probabilistic pushdown systems (with call-stack playing the role of the pushdown stack), and can model probabilistic programs with recursive procedural calls. We introduce Recursive Q-learning -- a model-free RL algorithm for RMDPs -- and prove that it converges for finite, single-exit and deterministic multi-exit RMDPs under mild assumptions.
14.50 - 15.40
Prof Ranko Lazic
Title: Implicit Regularisation of Deep Learning Algorithms
Abstract: Modern machine learning typically involves training deep neural networks that have more parameters than interpolating the data requires. Consequently the loss function has many different global minima, and so even if the training converges to one of them, the question is which one and what does that depend on? This problem is of great practical significance because different outcomes of the neural network training may have quite different properties, such as performance on unseen data and robustness against adversarial perturbations. It is also very interesting theoretically because it requires looking deeply under the hood of the training algorithm which is normally based on gradient descent, to discover how it "implicitly regularises" the neural network parameters. I shall introduce the resulting rapidly developing research area, and point out some past achievements and ongoing challenges.
Coffee Break 15.40 - 16.00
16.00 - 16.50
Dr Matthias Englert
Title: Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs
Abstract: Large neural networks usually have many different ways of fitting or classifying given data. Some of these solutions have more favourable properties than others. For example, some solutions may generalise well to new data that was not seen during training, while others may not generalise. When we train neural networks by gradient based methods, which of these solutions do we converge to? Gaining insight into this question is crucial if we want to understand the success of neural networks. However, even for seemingly simple tasks and networks, proving facts about the training dynamics and the solutions to which the training converges is challenging.
In light of this, we prove that, for the fundamental regression task of learning a single neuron, training a one-hidden layer ReLU network of any width by gradient flow from a small initialisation converges to zero loss and is implicitly biased to minimise the rank of network parameters. By assuming that the training points are correlated with the teacher neuron, we complement previous work that considered orthogonal datasets. Furthermore, we show that there is a surprising distinction between interpolator networks of minimal rank and those of minimal Euclidean norm.
Joint work with Dmitry Chistikov and Ranko Lazic
Workshop Dinner 19:00 - 22:00
at the Georgian Townhouse
https://www.thegeorgiantownhousenorwich.com/
Day 2
May 24, Friday
May 24, Friday
9.30 - 10.20
Prof Yang-Hui He
Title: The AI Mathematician
Abstract: We summarize how AI can approach mathematics in three ways: theorem-proving, conjecture formulation, and language processing. Inspired by initial experiments in geometry and string theory, we present a number of recent experiments on how various standard machine-learning algorithms can help with pattern detection across disciplines ranging from algebraic geometry to representation theory, to combinatorics, and to number theory. At the heart of the programme is the question how does AI help with mathematical discovery.
10.20 - 11.10
Dr Edward Hirst
Title: Machine Learning String Theory Geometries
Abstract: Calabi-Yau manifolds are the most popular compactification spaces for string theories, and the simplest sets of these geometries are constructed via weight systems. The structure of these weight systems are analysed through machine learning techniques, inspiring the design of a dramatically faster approximation for their topological invariants, leading to bulk generation of these geometries in higher dimensions.
Coffee Break 11:10 - 11:30
11.30 - 12.20
Elli Heyes
Title: Generating String Models with Machine Leaning
Abstract: Calabi-Yau n-folds can be obtained as hypersurfaces in toric varieties built from (n+1)-dimensional reflexive polytopes. Calabi-Yau 3-folds are of particular interest in string theory as they reduce 10-dimensional superstring theory to 4-dimensional quantum field theories with N=1 supersymmetry. We generate Calabi-Yau 3-folds by generating 4-dimensional reflexive polytopes and their triangulations using genetic algorithms. We show how, by modifying the fitness function of the genetic algorithm, one can generate Calabi-Yau manifolds with specific properties that give rise to certain string models of particular interest.
12:20 - 13:10
Dr Varun Kanade
Title: What can LLMs learn in context? A study with learning to learn Boolean functions
Abstract: In-context learning is a relatively widely studied phenomenon in recent years, where LLMs can seemingly work well on tasks that they were not trained on, just based on the context given to them. This can be viewed as a meta-learning phenomenon where these models are learning learning algorithms. We will discuss a stylised setting that will allow us to understand to what extent these models are actually learning new function using in-context information. We also discuss how the performance of these models can be improved using curated in-context data. While the talk will mostly discuss empirical observations, we will point to open theory questions that arise.
Based on joint work with Satwik Bhattamishra, Arkil Patel and Phil Blunsom.
Lunch 13.10 - 14.30
Google Sites
Report abuse