Winter / Hiver 2020
Friday, April 24, 2020
Ross Otto (McGill)
Recording: Not available.
How, when, and why do we make reflective versus reflexive choices?
The idea that our choices can arise either from a reflective and cognitively demanding system or a fast and reflexive system finds broad support in psychology and neuroscience. Clearly there are situations in which a given system should control our behavior: making the best possible decision is effortful and time-consuming, but the benefits of deliberative choice may be small relative to its cost. However, little experimental work has addressed what factors play into this tradeoff, or even if these two modes of choice really capture distinct processes. In this talk I examine how our reliance upon reflective versus reflexive choice varies based on factors such as the availability of cognitive resources, acute stress, or time pressure. To do this, I leverage the computational framework of Reinforcement Learning, which yields a set of precise set of testable predictions and data analysis tools—allowing us to formalize how different modes of choice behavior come about, and how these computations might unfold in the brain as measured with fMRI. Finally, I explore new methodologies for addressing these questions, including a quantitative framework for understanding how cognitive effort is allocated in the service of decision-making. Taken together, this work enriches our understanding of how and when people perform reflective versus reflexive choice and when it breaks down, informing both the cognitive psychology and neuroscience of decision-making.
Fairness and Algorithmic Discrimination
AI and machine learning tools are being used with increasing frequency for decision making in domains that affect peoples' lives such as employment, education, policing and loan approval. These uses raise concerns about biases and algorithmic discrimination and have motivated the development of fairness-aware mechanisms in the machine learning (ML) community and the operations research (OR) community, independently. In this talk, I will show how to ensure that the inference and predictions produced by a learned model are fair. Moreover, I will presents methods to ensure fairness in solutions of an optimization problem. I will conclude my talk with my research agenda to build on the complementary strengths of fairness methods in ML and OR and integrate ideas from them into a single system to build trustworthy AI.
Two-player Games in the Era of Machine Learning
Adversarial training, a special case of multiobjective optimization, is an increasingly useful tool in machine learning. For example, two-player zero-sum games are important for generative modeling (GANs) and for mastering games like Go or Poker via self-play. A classic result in Game Theory states that one must mix strategies, as pure equilibria may not exist. Surprisingly, machine learning practitioners typically train a single pair of agents – instead of a pair of mixtures – going against Nash’s principle. I this talk I will put learning in two-player zero-sum games on a firm theoretical foundation. I will first introduce a notion of limited-capacity-equilibrium for games played with neural networks for which, as capacity grows, optimal agents – not mixtures – can learn increasingly expressive and realistic behaviors. I will then focus on the training of such games by countering some common misconceptions about the difficulties of two-player games optimization and proposing to extend techniques designed for variational inequalities to the training of GANs.
Scaling Probabilistically Safe Learning to Robotics
Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems. In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for scaling up such approaches. Furthermore, very little work has been done on the even more difficult problem of safe imitation learning, in which the demonstrator's reward function is not known. This talk focuses on new developments in three key areas for scaling safe learning to robotics: (1) a theory of safe imitation learning; (2) scalable reward inference in the absence of models; (3) efficient off-policy policy evaluation. The proposed algorithms offer a blend of safety and practicality, making a significant step towards safe robot learning with modest amounts of real-world data.
Doing for our robots what nature did for us
We, as robot engineers, have to think hard about our role in the design of robots and how it interacts with learning, both in "the factory" (that is, at engineering time) and in "the wild" (that is, when the robot is delivered to a customer). I will share some general thoughts about the strategies for robot design and then talk in detail about some work I have been involved in, both in the design of an overall architecture for an intelligent robot and in strategies for learning to integrate new skills into the repertoire of an already competent robot.
Modeling Psychotherapy Dialogues with Kernelized Hashcode Representations: A Nonparametric Information-Theoretic Approach
We propose a novel dialogue modeling framework, the first-ever nonparametric kernel functions based approach for dialogue modeling, which learns kernelized hashcodes as compressed text representations; unlike traditional deep learning models, it handles well relatively small datasets, while also scaling to large ones. We also derive a novel lower bound on mutual information, used as a model-selection criterion favoring representations with better alignment between the utterances of participants in a collaborative dialogue setting, as well as higher predictability of the generated responses. As demonstrated on three real-life datasets, including prominently psychotherapy sessions, the proposed approach significantly outperforms several state-of-art neural network based dialogue systems, both in terms of computational efficiency, reducing training time from days or weeks to hours, and the response quality, achieving an order of magnitude improvement over competitors in frequency of being chosen as the best model by human evaluators.
On the road to individualized medicine: machine learning in the era of ‘omics’ data
Individualized medicine (IM) promises to revolutionize patient care by providing personalized treatments based on an individual’s molecular and clinical characteristics. However, we are still far from achieving the goals of IM. For example, in the case of cancer which is the leading cause of death in Canada, the majority of patients only receive (often inadequate) ‘standard of care’ treatment for their cancer type, independent of their tumours’ unique molecular profile. Even when a patient is originally responsive to a drug, they may develop drug resistance and thus face a relapse of cancer. Predicting the patients’ clinical drug response to different treatments and identifying biomarkers of drug sensitivity that can be targeted to overcome drug resistance are two major challenges in moving towards individualized medicine. Machine learning (ML) methods are a natural solution to address these issues, however the complexity of the underlying biological mechanisms and the unique characteristics of the heterogeneous, high-dimensional, multi-modal, and noisy data prohibits us from using off-the-shelf ML algorithms. In this talk, I will describe some recent approaches we developed to address these issues and will describe some important remaining challenges in this domain and our plans to address them.