Past Seminars: 2020

Abstract: While the trend in machine learning has tended towards more complex hypothesis spaces, it is not clear that this extra complexity is always necessary or helpful for many domains. In particular, models and their predictions are often made easier to understand by adding interpretability constraints. These constraints shrink the hypothesis space; that is, they make the model simpler. Statistical learning theory suggests that generalization may be improved as a result as well. However, adding extra constraints can make optimization (exponentially) harder. For instance it is much easier in practice to create an accurate neural network than an accurate and sparse decision tree. We address the following question: Can we show that a simple-but-accurate machine learning model might exist for our problem, before actually finding it? If the answer is promising, it would then be worthwhile to solve the harder constrained optimization problem to find such a model. In this talk, I present an easy calculation to check for the possibility of a simpler model. This calculation indicates that simpler-but-accurate models do exist in practice more often than you might think. Time-permitting, I will then briefly overview our progress towards the challenging problem of finding optimal sparse decision trees.

Bio: Cynthia Rudin is a professor of computer science, electrical and computer engineering, and statistical science at Duke University. Previously, Prof. Rudin held positions at MIT, Columbia, and NYU. Her degrees are from the University at Buffalo and Princeton University. She is a three-time winner of the INFORMS Innovative Applications in Analytics Award. She has served on committees for INFORMS, the National Academies, the American Statistical Association, DARPA, the NIJ, and AAAI. She is a fellow of both the American Statistical Association and Institute of Mathematical Statistics. She was a Thomas Langford Lecturer at Duke University for 2019-2020.

Abstract: This talk will revolve around two performance criteria that have been studied in the context of episodic reinforcement learning: an old one, Best Policy Identification (BPI) [Fiechter, 1994], and a new one, Reward Free Exploration (RFE) [Jin et al., 2020]. We will see that a variant of the very first BPI algorithm can actually be used for the more challenging reward free exploration problem. This Reward-Free UCRL algorithm, which adaptively explores the MDP and adaptively decides when to stop exploration, requires fewer exploration episodes than state-of-the art algorithms. We will then present alternative algorithms for the BPI objective and discuss the relative complexity of BPI and RFE.

Bio: Emilie Kaufmann is a CNRS researcher in the CRIStAL laboratory at University of Lille. She is also a member of the Inria Scool team (formely SequeL), whose expertise is in sequential decision making. She worked a lot on the stochastic multi-armed bandit problem, in particular towards getting a better understanding of the difference between rewards maximization and pure-exploration problems. She also recently worked on exploration for reinforcement learning.

Abstract: Research in artificial general intelligence aims to create agents that can learn from their own experience to solve arbitrary tasks in complex and dynamic settings. To do so effectively and efficiently, such an agent must be able to predict how its environment will change both dependently and independently of its own actions. General value functions (GVFs) are one approach to representing such relationships. A single GVF poses a predictive question defined by three components: a behavior (policy), a prediction timescale, and a prediction target (cumulant). Estimated answers to these questions can be learned efficiently from the agent’s own experience using temporal-difference learning methods. The agent’s collection of GVF questions and corresponding answers can be viewed as forming a predictive model of the agent’s interaction with its environment. Ultimately, such a model may enable an agent to understand its environment and make decisions therein.

Bio:Craig Sherstan is currently a Research Scientist at Sony AI, and defending his dissertation as a PhD student in Computing Science at the University of Alberta. He is supervised by Patrick Pilarski as part of the Bionic Limbs for Improved Natural Control (BLINC) Lab and the Reinforcement Learning and Artificial Intelligence Lab (RLAI), and is a Vanier Scholar. Craig’s research has focused on developing agents which can continually and incrementally construct predictive representations of themselves and their world.

Abstract: With improvements in computer vision techniques and hardware, some of the problems of manual assessment of histology images, such as inter- and intra-observer variability, inability to assess subtle visual features, and the time taken to examine whole slides are being alleviated by computational pathology. A key module in several computational pathology pipelines is the one that segments nuclei. Accurate nuclei segmentation could facilitate downstream analysis of tissue samples for assessing not only cancer grades or stages but also for predicting tumor recurrence, treatment effectiveness and for quantifying intra-tumor heterogeneity. Identifying different types of nuclei, such as epithelial, neutrophils, lymphocytes, macrophages, etc., could yield information about the host immune response that could advance our understanding of the mechanisms governing treatment resistance and adaptive immunity in cancers of various organs. This talk will give an overview of state-of-the art machine learning algorithms for nuclei segmentation and classification from H&E stained tissue images while providing insights into the process of creating one of the largest nuclei segmentation datasets and organizing two international competitions on this theme. I will also discuss a few nuclei segmentation use-cases including automatic staging of colorectal tumors, prostate cancer recurrence prediction and, intratumor heterogeneity

Bio: Neeraj Kumar just began as a postdoctoral fellow in the Greiner lab, University of Alberta. His research interests include computational pathology, medical image processing, machine learning for healthcare and medicine, and bioinformatics. Previously, he was a research associate at the center for computational imaging and personalized diagnostics at Case Western Reserve University, Cleveland. He completed his Ph.D. in Electronics and Electrical Engineering from the Indian Institute of Technology Guwahati, in 2017. After his Ph.D., he joined as a research fellow in cancer education and career development program of the department of pathology, University of Illinois at Chicago, to address clinically and biologically relevant cancer research questions by capitalizing on his expertise in image processing and machine learning. He was also a visiting researcher at the Institute of Research in Communications and Cybernetics at Ecole Centrale Nantes, Nantes, France, and at Beckman Institute of the University of Illinois at Urbana- Champaign, Champaign. He is a recipient of the numerous prestigious awards including the R25 trainee award from NCI (NIH, USA), Erasmus Mundus Heritage Fellowship, Microsoft Research India PhD fellowship, and the best poster award at University of Illinois Cancer Center’s annual research symposium.

Abstract: In popular imagination, household robots that we can instruct to "bring me my red mug from the kitchen" or ask “where are my glasses?” are common. For a robot to execute such an instruction or answer such a question, it needs to parse and interpret natural language, understand the 3D environment it is in (e.g. what objects exist and how they are described), navigate to locate the target object, and then formulate an appropriate response. While there has been previous work on the language-to-vision grounding problem in the 2D domain, there is much less work on methods operating with 3D representations such as required by the scenarios in these examples. As a first step in this direction, we introduce the new task of 3D object localization in scenes using natural language descriptions. As input, we assume a point cloud of a scanned 3D scene along with a freeform text description of a specified target object. Through crowdsourcing, we collect a dataset of natural language descriptions of objects in the ScanNet dataset and create a benchmark with several baseline methods for this challenging task of predicting the 3D bounding box of a referred object based on a natural language description. I will conclude by briefly summarizing various other ongoing projects in the area of grounded natural language to 3D interactive environments.

Bio: Dr. Angel Xuan Chang is an Assistant Professor of Computer Science at Simon Fraser University. Dr. Chang’s research focuses on the intersection of natural language understanding, computer graphics, and AI. Her research connects language to 3D representations of shapes and scenes and addresses grounding of language for embodied agents in indoor environments. She has worked on methods for synthesizing 3D scenes and shapes, 3D scene understanding, and helped to create various datasets for 3D deep learning (ShapeNet, ScanNet, Matterport3D). She received her Ph.D. in Computer Science from Stanford, under the supervision of Chris Manning. Dr. Chang received the SGP 2018 Dataset Award for her work on the ShapeNet dataset. She is a recipient of the TUM-IAS Hans Fischer Fellowship and a Canada CIFAR AI Chair.

Abstract: Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems. In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for scaling up such approaches. Furthermore, very little work has been done on the even more difficult problem of safe imitation learning, in which the demonstrator's reward function is not known. This talk focuses on new developments in three key areas for scaling safe learning to robotics: (1) a theory of safe imitation learning; (2) scalable reward inference in the absence of models; (3) efficient off-policy policy evaluation. The proposed algorithms offer a blend of safety and practicality, making a significant step towards safe robot learning with modest amounts of real-world data.

Bio: Scott Niekum is an Assistant Professor and the director of the Personal Autonomous Robotics Lab (PeARL) in the Department of Computer Science at UT Austin. He is also a core faculty member in the interdepartmental robotics group at UT. Prior to joining UT Austin, Scott was a postdoctoral research fellow at the Carnegie Mellon Robotics Institute and received his Ph.D. from the Department of Computer Science at the University of Massachusetts Amherst. His research interests include imitation learning, reinforcement learning, and robotic manipulation. Scott is a recipient of the 2018 NSF CAREER Award and 2019 AFOSR Young Investigator Award.

Abstract: While artificial intelligence holds the promise of providing value in almost every discipline, the two of most immediate value are medicine/health and law. In this presentation, we summarize what some of the motivation and misconceptions of AI applied to law, and provide a glimpse of the work of the XAI lab in navigating paths to creating value from AI in the full spectrum of of the judicial process, from access to justice to support for judgement decisions

Bio: Randy Goebel is currently professor of Computing Science in the Department of Computing Science at the University of Alberta, Associate Vice President (Research) and Associate Vice President (Academic), and Fellow and co-founder of the Alberta Machine Intelligence Institute (AMII). He received the B.Sc. (Computer Science), M.Sc. (Computing Science), and Ph.D. (Computer Science) from the universities of Regina, Alberta, and British Columbia, respectively. Professor Goebel's theoretical work on abduction, hypothetical reasoning and belief revision is internationally well known, and his recent research is focused on the formalization of visualization and explainable artificial intelligence (XAI). He has been a professor or visiting professor at the University of Waterloo, University of Regina, University of Tokyo, Hokkaido University, Multimedia University (Malaysia), National Institute of Informatics, and a visiting researcher at NICTA (now Data 61) in Australia, and DFKI and VW Data:Lab in Germany. He has worked on optimization, algorithm complexity, systems biology, and natural language processing, including applications in legal reasoning and medical informatics.

Abstract: This talk will present two of our recent results regarding sample-based reinforcement learning. The first half is about using distributional tools to analyze the behaviour of sample-based algorithms, such as TD(0) and Q-Learning, when the step-size is kept constant. We find that lifting the analysis to distributions results in significantly simpler proofs, and sheds new insights on the learning process. The second half revisits representation learning in the linear function approximation setting to answer the question: are there representations (i.e., mappings from states to features) which are stable, in the sense that lead to convergent RL even in the presence of the deadly triad? Surveying classic representation learning schemes (proto-value functions, Krylov bases, etc.) we find that not all schemes are created equal with respect to stability. From an auxiliary tasks perspective, we find that simply predicting immediate next features and expected future rewards may actually lead to stable representations -- giving formal evidence of the usefulness of these common heuristics.

Bio: Marc G. Bellemare leads the reinforcement learning efforts at Google Brain in Montreal and holds a Canada CIFAR AI Chair at the Quebec Artificial Intelligence Institute (Mila). He received his Ph.D. from the University of Alberta, where he developed the highly-successful Arcade Learning Environment with Michael Bowling and Joel Veness. He was a research scientist at DeepMind from 2013 to 2017, during which time he made major contributions to deep reinforcement learning, in particular pioneering the distributional method. Marc G. Bellemare is also a CIFAR Learning in Machines & Brains Fellow and an adjunct professor at McGill University.

Abstract: General value functions represent the world as a collection of predictive questions using the semantics of value functions. A fundamental open question with GVFs is determining what questions should be asked. One of the key parameters of a GVF is the prediction timescale. In this talk I will present an approach called Γ-nets which allows a single function approximation network to be trained and queried for arbitrary fixed timescales, removing the need to determine GVF timescale ahead of time. This presentation will be an extended version of the talk I will be presenting at AAAI in New York next week.

Bio: Craig Sherstan is currently a Research Scientist at Sony AI, and defending his dissertation as a PhD student in Computing Science at the University of Alberta. He is supervised by Patrick Pilarski as part of the Bionic Limbs for Improved Natural Control (BLINC) Lab and the Reinforcement Learning and Artificial Intelligence Lab (RLAI), and is a Vanier Scholar. Craig’s research has focused on developing agents which can continually and incrementally construct predictive representations of themselves and their world.

Abstract: 48Hour Discovery Inc. is an University of Alberta-based company - founded in 2017, 48Hour Discovery Inc. aims to facilitate new drug discovery in the pharmaceutical and biotechnology industries with our advanced platform based on patented barcoded libraries, stream-lined workflow and open-access data management. The 48Hour Discovery Cloud houses a database of user curated content of >10,000 screens and provides easy-to-use search functions, data visualization and analysis. Together with their team of Laboratory and Data Scientists, 48 Hour Discovery is continually building new technologies that make ligand discovery faster, easier, and more transparent.

Bio: Dr. Ratmir Derda, CEO, and Associate Professor: Ratmir Derda received his B.Sci. in Physics from Moscow Institute of Physics and Technology in 2001 and Ph.D. in Chemistry from the University of Wisconsin-Madison in 2008. From 2008 to 2011, he was a postdoctoral researcher at Harvard University. Dr. Derda has been recognized for his work in research, business, and mentorship - with recent awards by University of Alberta, and featured as a Top 40 under 40 local leader in Edmonton.