Older Talks

Neural Networks and Bayes Rule

Relational or structured reasoning is an important current research challenge. The classical approach to this challenge is a templated graphical model: highly expressive, with well-founded semantics, but (at least naively) difficult to scale up, and difficult to combine with the most effective supervised learning methods. More recently, researchers have designed many different deep network architectures for structured reasoning problems, with almost the flip set of advantages and disadvantages. Can we get the best of both worlds? That is, can we design deep nets that look more like graphical models, or graphical models that look more like deep nets, so that we get a framework that is both practical and "semantic"? This talk will take a look at some progress toward such a hybrid framework.

Keeping our graphs attentive

A multitude of important real-world datasets (especially in biology) come together with some form of graph structure: social networks, citation networks, protein-protein interactions, brain connectome data, etc. Extending neural networks to be able to properly deal with this kind of data is therefore a very important direction for machine learning research, but one that has received comparatively rather low levels of attention until very recently. Attentional mechanisms represent a very promising direction for extending the established convolutional operator on images to work on arbitrary graphs, as they satisfy many of the desirable features for a convolutional operator. Through this talk, I will focus on my work on Graph Attention Networks (GATs), where these theoretical properties have been further validated by solid results on transductive as well as inductive node classification benchmarks. I will also outline some of the earlier efforts towards deploying attention-style operators on graph structures, as well as very exciting recent work that expands on GATs and deploys them in more general circumstances (such as EAGCN, DeepInf, and applications to solving the Travelling Salesman Problem). Time permitting, I will also present some of the relevant related graph-based work in the computational biology and medical imaging domains that I have been involved in.

Intro to Ethics 2

In this presentation, I will introduce to the three main families of normative moral theories: consequentialism, deontologism and virtue ethics.

Dynamics of high-dimensional recurrent networks: how chaos shapes computation in biological and artificial neural networks

Networks of neurons —either biological or artificial— are called recurrent if their connections are distributed and contain feedback loops. Such networks can perform remarkably complex computations, as evidenced by their ubiquity throughout the brain and ever-increasing use in machine learning. They are, however, notoriously hard to control and their dynamics are generally poorly understood, especially in the presence of external forcing. This is because recurrent networks are typically chaotic systems, meaning they have rich and sensitive dynamics leading to variable responses to inputs. How does this chaos manifest in the neural code of the brain? How might we tame sensitivity to exploit complexity when training artificial recurrent networks for machine learning?

Understanding how the dynamics of large driven networks shape their capacity to encode and process information presents a sizeable challenge. In this talk, I will discuss the use of Random Dynamical Systems Theory as a framework to study information processing in high-dimensional, signal-driven networks. I will present an overview of recent results linking chaotic attractors to entropy production, dimensionality and input discrimination of dynamical observables. I will outline insights this theory provides on how cortex performs complex computations using sparsely connected inhibitory and excitatory neurons, as well as implications for gradient-based optimization methods for artificial networks.