Organizers

Organizing Committee:

Andrew Barto is Professor Emeritus of Computer Science, University of Massachusetts, Amherst, having retired in 2012. He served as Chair of the UMass Department of Computer Science from 2007 to 2011. He received his B.S. with distinction in mathematics from the University of Michigan in 1970, and his Ph.D. in Computer Science in 1975, also from the University of Michigan. He is Co-Director of the Autonomous Learning Laboratory and an Associate Member of the Neuroscience and Behavior Program of the University of Massachusetts. He currently serves as an associate editor of Neural Computation, as a member of the Advisory Board of the Journal of Machine Learning Research, as a member of the editorial boards Adaptive Behavior, and Frontiers in Decision Neuroscience. Professor Barto is a Fellow of the American Association for the Advancement of Science, a Fellow and Senior Member of the IEEE, and a member of the Society for Neuroscience. He received the 2004 IEEE Neural Network Society Pioneer Award for contributions to the field of reinforcement learning, and the IJCAI-17 Award for Research Excellence for groundbreaking and impactful research in both the theory and application of reinforcement learning. He has published over one hundred papers or chapters in journals, books, and conference and workshop proceedings. He is co-author with Richard Sutton of the book "Reinforcement Learning: An Introduction," MIT Press 1998, which has received over 24,000 citations.

Doina Precup received her B.Sc. from the Computer Science Department, Technical University Cluj-Napoca, Romania, in 1994, and the M.S. and PhD from the Department of Computer Science, University of Massachusetts, Amherst, in 1997 and 2000 respectively. In July 2000 she joined the School of Computer Science at McGill University, as a tenure-track assistant professor. Doina Precup's research interests lie mainly in the field of machine learning. She is especially interested in the learning problems that face a decision-maker interacting with a complex, uncertain environment. Doina uses the framework of reinforcement learning to tackle such problems. Her current research is focused on developing better knowledge representation methods for reinforcement learning agents. Doina Precup is also more broadly interested in reasoning under uncertainty, and in the applications of machine learning techniques to real-world problems.

Shie Mannor is a professor of Electrical Engineering at the Technion, Israel Institute of Technology. Shie graduated from the Technion with a PhD in Electrical Engineering in 2002. He was a Fulbright postdoctoral scholar at LIDS at MIT from 2002 to 2004. He was at the Department of Electrical and Computer Engineering at McGill University from July 2004 until August 2010, where he held the Canada Research Chair in Machine Learning. Shie has been with the Department of Electrical Engineering at the Technion since 2008 where he is currently a professor. His research interests include machine learning, planning and control, and networks. Shie has published over 70 journal papers and over 130 conference papers and holds 8 patents. He is an associate editor of Operations Research and of Mathematics of Operations Research and an action editor of the Journal of Machine Learning Research (JMLR). His research awards include several best paper awards, the Henri Taub Prize for Academic Excellence, an ERC Starting Grant, an HP Faculty Award and a Horev Fellowship.

Tom Schaul is a senior research scientist at DeepMind. His research interests include (modular/hierarchical) reinforcement learning, (stochastic/black-box) optimization with minimal hyperparameter tuning, and (deep/recurrent) neural networks. His favorite application domain are games. He grew up in Luxembourg and studied computer science in Switzerland (with exchanges at Waterloo and Columbia), where he obtained an MSc from the EPFL in 2005. He holds a PhD from TU Munich (2011), which he did under the supervision of Jürgen Schmidhuber at the Swiss AI Lab IDSIA. From 2011 to 2013 he was a postdoc at the Courant Institute of NYU, in the lab of Yann LeCun.

Roy Fox is a postdoc at UC Berkeley working with Ken Goldberg in the Laboratory for Automation Science and Engineering (AUTOLAB), and with Ion Stoica in the Real-Time Intelligent Secure Execution lab (RISELab). His research interests include reinforcement learning, dynamical systems, information theory, automation, and the connections between these fields. His current research focuses on automatic discovery of hierarchical control structures in deep reinforcement learning and in imitation learning of robotic tasks. Roy holds a MSc in Computer Science from the Technion, under the supervision of Moshe Tennenholtz, and a PhD in Computer Science from the Hebrew University, under the supervision of Naftali Tishby. He was an exchange PhD student with Larry Abbott and Liam Paninski at the Center for Theoretical Neuroscience at Columbia University, and a research intern at Microsoft Research.

Carlos Florensa is a PhD student in the Robotics Learning Lab at UC Berkeley, under the supervision of Prof. Pieter Abbeel. His main interest is learning under minimum supervision. In particular, his current research focuses on solving robotics tasks where only high-level instructions are provided, yielding challenging sparse reward problems. In 2015 he obtained a doubled degree in Mathematics and Industrial Engineering, with Honors, at the Center for High Interdisciplinary Training in the UPC (Spain). During his undergrad he performed research at international institutions such as Argonne National Lab (USA), the EPFL (Switzerland), ICFO (Spain) and CMU (USA). Currently he is a LaCaixa Fellow.

Advisory Committee:

Ken Goldberg is an artist, inventor, and UC Berkeley Professor. He is Chair of the Industrial Engineering and Operations Research Department, with secondary appointments in EECS, Art Practice, the School of Information, and Radiation Oncology at the UCSF Medical School. Ken is Director of the CITRIS "People and Robots" Initiative and the UC Berkeley AUTOLAB where he and his students pursue research in geometric algorithms and machine learning for robotics and automation in surgery, manufacturing, and other applications. Ken developed the first provably complete algorithms for part feeding and part fixturing and the first robot on the Internet. Despite agonizingly slow progress, Ken persists in trying to make robots less clumsy. He has over 250 peer-reviewed publications and 8 U.S. Patents. He co-founded and served as Editor-in-Chief of the IEEE Transactions on Automation Science and Engineering. Ken's artwork has appeared in 70 exhibits including the Whitney Biennial and films he has co-written have been selected for Sundance and nominated for an Emmy Award. Ken was awarded the NSF PECASE (Presidential Faculty Fellowship) from President Bill Clinton in 1995, elected IEEE Fellow in 2005 and selected by the IEEE Robotics and Automation Society for the George Saridis Leadership Award in 2016.

Pieter Abbeel (BS/MS EE KU Leuven, 2000; PhD CS Stanford, 2008, Advisor: Andrew Ng) is professor at UC Berkeley (EECS, BAIR) since 2008 and is a Research Scientist at OpenAI since 2016. Pieter has developed apprenticeship learning algorithms which have enabled advanced helicopter aerobatics, including maneuvers such as tic-tocs, chaos and auto-rotation, which only exceptional human pilots can perform. His group has enabled the first end-to-end completion of reliably picking up a crumpled laundry article and folding it and has pioneered deep reinforcement learning for robotics, including learning locomotion and visuomotor skills. His work has been featured in many popular press outlets, including BBC, New York Times, MIT Technology Review, Discovery Channel, SmartPlanet and Wired. His current research focuses on robotics and machine learning with particular focus on deep reinforcement learning, deep imitation learning, deep unsupervised learning, and AI safety. Pieter has won various awards, including the Sloan Research Fellowship, the Air Force Office of Scientific Research Young Investigator Program (AFOSR-YIP) award, the Okawa Research Grant, the 2011 TR35, the IEEE Robotics and Automation Society (RAS) Early Career Award, and the Dick Volz Best U.S. Ph.D. Thesis in Robotics and Automation Award. Pieter was awarded the NSF PECASE (Presidential Early Career Awards for Scientists and Engineers) from President Barack Obama in 2016.

Roberto Calandra is a Postdoctoral Scholar at UC Berkeley in the Berkeley Artificial Intelligence Research Laboratory (BAIR) under the supervision of Sergey Levine. His scientific interests are at the conjunction of Machine Learning and Robotics, in what is know as Robot Learning. Specifically he is interested in learn and leveraging models of the world in order to provide robots with the capability of predicting their actions. Some of the research topics that he is currently interested in include: Deep Reinforcement Learning, Bayesian Optimization and Dynamics Modeling. Previously, he completed his Ph.D. in the Intelligent Autonomous Systems Lab at TU Darmstadt, under the supervision of Jan Peters and Marc Deisenroth.