Abstract

The combination of abundant data and large-scale neural networks has transformed many fields of artificial intelligence, including vision and language. This has not been the case in robotics. Robots are real machines, embodied in the world, and operating in real-time. Their hardware is evolving rapidly. As the hardware evolves, new data is collected and old data is discarded. At present, data is seldom stored or shared. Like many machines, robots break down routinely, thus exacerbating this data paucity problem. Robots also come in a variety of shapes and attempt to solve a myriad of tasks. This large diversity in multimodal actuation and perception, and in tasks causes unique generalization difficulties for machine learning approaches. Moreover we not only demand that robots understand images and language, but we also expect them to be capable of fine motor control and of safe interaction with people. The goal of this talk is to raise awareness of these challenges, and to outline some initial attempts at solving them, using approaches such as deep learning, meta-learning, offline reinforcement learning, preference learning, interaction and reward learning.

Bio

I am a lead scientist at DeepMind. I learned to use neural networks to control real machines during my undergraduate studies at the University of the Witwartersrand, South Africa. This was followed by a PhD in information engineering at Cambridge University, and a postdoc in AI at UC Berkeley. I enjoyed working as a full professor at the universities of British Columbia and Oxford before joining DeepMind. During that time, I also helped create and grow tech companies in web-scale machine learning and natural language understanding with deep networks. I am passionate about solving intelligence and robotics as I believe the ultimate survival of humans depends on this. I also believe this positive outlook will only be made possible by improving diversity, equity and inclusion in our fields. In particular, the abnormally low representation of women, Africans, and many other disadvantaged groups in engineering and science, in all roles but especially in leadership, is precluding us from having access to a precious pool of perspectives and vibrant ideas. This lack of inclusion and diversity is not only a missed opportunity, but it also poses existential dangers that we can no longer be complacent about.

Homepage: https://twitter.com/nandodf?lang=en

Google Scholar: https://scholar.google.com/citations?user=nzEluBwAAAAJ&hl=en

Affiliation: DeepMind

Abstract

Quality Diversity (QD) algorithms are those that seek to produce a diverse set of high-performing solutions to problems. I will describe them and a number of their positive attributes. I will summarize how they enable robots, after being damaged, to adapt in 1-2 minutes in order to continue performing their mission. I will next describe our QD-based Go-Explore algorithm, which dramatically improves the ability of deep reinforcement learning algorithms to solve previously unsolvable problems wherein reward signals are sparse, meaning that intelligent exploration is required. Go-Explore solved all unsolved Atari games, including Montezuma’s Revenge and Pitfall, considered by many to be a grand challenge of AI research. I will next motivate research into open-ended algorithms, which seek to innovate endlessly, and introduce our POET algorithm, which generates its own training challenges while learning to solve them, automatically creating a curricula for robots to learn an expanding set of diverse skills. Finally, I’ll argue that an alternate paradigm—AI-generating algorithms (AI-GAs)—may be the fastest path to accomplishing our field’s grandest ambition of creating general AI, and describe how QD and Open-Ended algorithms will be essential ingredients of AI-GAs.

Bio

Jeff Clune is an Associate Professor of computer science at the University of British Columbia and a research manager at OpenAI. Before those roles, he was a Senior Research Manager and founding member of Uber AI Labs, which was formed after Uber acquired a startup he helped lead. Jeff focuses on deep learning, including deep reinforcement learning. Prior to Uber, he was the Loy and Edith Harris Associate Professor in Computer Science at the University of Wyoming. Before that he was a Research Scientist at Cornell University and received degrees from Michigan State University (PhD, master’s) and the University of Michigan (bachelor’s). More on Jeff’s research can be found at JeffClune.com or on Twitter (@jeffclune).

Homepage: http://jeffclune.com/

Google Scholar: https://scholar.google.com/citations?hl=en&user=5TZ7f5wAAAAJ

Affiliation: University of British Columbia & OpenAI

Abstract

Robots working with humans and in human environments need to learn from and adapt to their users. As robots enter human environments, they will need to interact with humans in a variety of roles: as students, teachers, collaborators and assistants. In these roles, robots will need to adapt to users' individual preferences and capabilities, which may not be known ahead of the interaction. In this talk, I will describe approaches for robot learning during interaction, considering robots in different roles and in a variety of applications, including rehabilitation, collaboration in industrial settings, and in education and entertainment.

Bio

Prof. Dana Kulić conducts research in robotics and human-robot interaction (HRI), and develops autonomous systems that can operate in concert with humans, using natural and intuitive interaction strategies while learning from user feedback to improve and individualize operation over long-term use. Dana Kulić received the combined B. A. Sc. and M. Eng. degree in electro-mechanical engineering, and the Ph. D. degree in mechanical engineering from the University of British Columbia, Canada, in 1998 and 2005, respectively. From 2006 to 2009, Dr. Kulić was a JSPS Post-doctoral Fellow and a Project Assistant Professor at the Nakamura-Yamane Laboratory at the University of Tokyo, Japan. In 2009, Dr. Kulić established the Adaptive System Laboratory at the University of Waterloo, Canada, conducting research in human robot interaction, human motion analysis for rehabilitation and humanoid robotics. Since 2019, Dr. Kulić is a professor and director of Monash Robotics at Monash University, Australia. In 2020, Dr. Kulić was awarded the ARC Future Fellowship. Her research interests include robot learning and human-robot interaction.

Homepage: https://www.monash.edu/engineering/robotics

Google Scholar: https://scholar.google.com/citations?hl=en&user=sL0KJlQAAAAJ

Affiliation: Monash University, Australia

Abstract

Tiny machine learning (tinyML) is a fast-growing and emerging field at the intersection of machine learning (ML) algorithms and low-cost embedded systems. It enables on-device analysis of sensor data (vision, audio, IMU, etc.) at ultra-low-power consumption (<1mW). Moving machine learning compute close to the sensor(s) allows for an expansive new variety of always-on ML use-cases, especially in size, weight and power (SWaP) constrained robots. This talk introduces the broad vision behind tinyML, and specifically, it focuses on exciting new applications that tinyML enables for cheap and lightweight on-device robot learning. Although tinyML for robotics has rich possibilities, there are still numerous technical challenges to address. Tight onboard processor, memory and storage constraints, coupled with embedded software fragmentation, and a lack of relevant large-scale tinyML sensor datasets and benchmarks pose a substantial barrier to developing novel robotics applications. To this end, the talk touches upon the myriad research opportunities for unlocking the full potential of "tiny robot learning," spanning from algorithm design to automatic hardware synthesis.

Bio

Vijay Janapa Reddi is an Associate Professor at Harvard University, VP and a founding member of MLCommons (mlcommons.org), a nonprofit organization aiming to accelerate machine learning (ML) innovation for everyone. He also serves on the MLCommons board of directors and is a Co-Chair of the MLCommons Research organization. He led the MLPerf Inference ML benchmark for datacenter, edge, mobile and IoT systems. Before joining Harvard, he was an Associate Professor at The University of Texas at Austin in the Electrical and Computer Engineering department. His research sits at the intersection of machine learning, computer architecture and runtime software. He specializes in building computing systems for tiny IoT devices, as well as mobile and edge computing. Dr. Janapa-Reddi is a recipient of multiple honors and awards, including the National Academy of Engineering (NAE) Gilbreth Lecturer Honor (2016), IEEE TCCA Young Computer Architect Award (2016), Intel Early Career Award (2013), Google Faculty Research Awards (2012, 2013, 2015, 2017, 2020), Best Papers at the 2020 Design Automation Conference (DAC), 2005 International Symposium on Microarchitecture (MICRO), 2009 International Symposium on High-Performance Computer Architecture (HPCA), IEEE’s Top Picks in Computer Architecture awards (2006, 2010, 2011, 2016, 2017,2021). He has been inducted into the MICRO and HPCA Hall of Fame (in 2018 and 2019, respectively). Dr. Janapa-Reddi is passionate about widening access to applied machine learning for STEM, Diversity, and using AI for social good. He designed the Tiny Machine Learning (TinyML) series on edX, a massive open online course (MOOC) that sits at the intersection of embedded systems and ML that thousands of global learners access and audit free of cost. He was also responsible for the Austin Hands-on Computer Science (HaCS) deployed in the Austin Independent School District for K-12 CS education. Dr. Janapa-Reddi received a Ph.D. in computer science from Harvard University, an M.S. from the University of Colorado at Boulder and a B.S from Santa Clara University.

Homepage: https://scholar.harvard.edu/vijay-janapa-reddi

Google Scholar: https://scholar.google.com/citations?hl=en&user=gy4UVGcAAAAJ

Affiliation: Harvard University

Abstract

Autonomous quadrotors will soon play a major role in search-and-rescue, delivery, and inspection missions, where a fast response is crucial. However, their speed and maneuverability are still far from those of birds and human pilots. Agile flight is particularly important: since drone battery life is usually limited to 20-30 minutes, drones need to fly faster to cover longer distances. However, to do so, they need faster sensors and algorithms. Human pilots take years to learn the skills to navigate drones. What does it take to make drones navigate as good or even better than human pilots? Autonomous, agile navigation through unknown, GPS-denied environments poses several challenges for robotics research in terms of perception, planning, learning, and control. In this talk, I will show how the combination of both model-based and machine learning methods united with the power of new, low-latency sensors, such as event cameras, can allow drones to achieve unprecedented speed and robustness by relying solely on onboard computing.

Bio

Davide Scaramuzza is a Professor of Robotics and Perception at the University of Zurich, where he does research at the intersection of robotics, computer vision, and machine learning, using standard cameras and event cameras, and aims to enable autonomous, agile navigation of micro drones in search and rescue applications. For his research contributions, he won prestigious awards, such as a European Research Council (ERC) Consolidator Grant, the IEEE Robotics and Automation Society Early Career Award, a Google Research Award, and two Qualcomm Innovation Fellowships. In 2015, he cofounded Zurich-Eye, today Facebook Zurich, which developed the visual-inertial SLAM system running in Oculus Quest VR headsets. He was also the strategic advisor of Dacuda, today Magic Leap Zurich. Many aspects of his research have been prominently featured in wider media, such as The New York Times, BBC News, Discovery Channel.

Homepage: http://rpg.ifi.uzh.ch/people_scaramuzza.html

Google Scholar: https://scholar.google.com/citations?user=SC9wV2kAAAAJ&hl=en

Affiliation: University of Zurich

Abstract

There are few things more frustrating than a machine that repeats the same mistake over and over again. To contend with a complex and uncertain world, robots must learn from their mistakes and rapidly adapt to their environment. In this tutorial I will discuss several different ways in which ideas from machine learning and model predictive control (MPC) can be leveraged to build intelligent, adaptive robotic systems. I’ll start by discussing how to learn models for MPC that perform well on a given control task. Next, I’ll introduce an online learning perspective on MPC that unifies well-known algorithms and provides a prescriptive way to generate new ones. Finally, I will discuss how MPC can be combined with model-free reinforcement learning to build fast, reactive systems that can improve their performance over time.

Bio

Byron Boots is an Associate Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington where he directs the UW Robot Learning Laboratory. He is also a Principal Research Scientist in the Seattle Robotics Lab at NVIDIA, and is co-chair of the IEEE Robotics and Automation Society Technical Committee on Robot Learning. Byron’s group performs fundamental and applied research in machine learning, artificial intelligence, and robotics with a focus on developing theory and systems that tightly integrate perception, learning, and control. He has received several awards including "Best Paper" Awards from ICML, AISTATS, RSS, and IJRR. He is also the recipient of the RSS Early Career Award, the NSF CAREER Award, and the Outstanding Junior Faculty Research Award from the College of Computing at Georgia Tech. Byron received his PhD from the Machine Learning Department at Carnegie Mellon University.

Homepage: https://homes.cs.washington.edu/~bboots/

Google Scholar: https://scholar.google.com/citations?hl=en&user=kXB8FBoAAAAJ

Affiliation: University of Washington

Abstract

This talk will describe how ground, aerial, and marine robots have been used in disasters, most recently the coronavirus pandemic. In the first year of the pandemic, 338 instances of robots in 48 countries protecting healthcare workers from unnecessary exposure, handling the surge in demand for clinical care, preventing infections, restoring economic activity, and maintaining individual quality of life have been reported. The uses span six sociotechnical work domains and 29 different use cases representing different missions, robot work envelopes, and human-robot interaction dyads. The dataset also confirms a model of adoption of robotics technology for disasters. Adoption favors robots that maximize the suitability for established use cases while minimizing risk of malfunction, hidden workload costs, or unintended consequences as measured by the NASA Technical Readiness Assessment metrics. Regulations do not present a major barrier but availability, either in terms of inventory or prohibitively high costs, does. The model suggests that in order to be prepared for future events, roboticists should partner with responders now, investigate how to rapidly manufacture complex, reliable robots on demand, and conduct fundamental research on predicting and mitigating risk in extreme or novel environments.

Bio

Dr. Robin R. Murphy is the Raytheon Professor of Computer Science and Engineering at Texas A&M University, a TED speaker, and an IEEE and ACM Fellow. She helped create the fields of disaster robotics and human-robot interaction, deploying robots to 29 disasters in five countries including the 9/11 World Trade Center, Fukushima, the Syrian boat refugee crisis, Hurricane Harvey, and the Kilauea volcanic eruption. Murphy’s contributions to robotics have been recognized with the ACM Eugene L. Lawler Award for Humanitarian Contributions, a US Air Force Exemplary Civilian Service Award medal, the AUVSI Foundation’s Al Aube Award, and the Motohiro Kisoi Award for Rescue Engineering Education (Japan). She has written the best-selling textbook Introduction to AI Robotics (2nd edition 2019) and the award-winning Disaster Robotics (2014), plus serving as an editor for the science fiction/science fact focus series for the journal Science Robotics. She co-chaired the White House OSTP and NSF workshops on robotics for infectious diseases and recently co-chaired the National Academy of Engineering/Computing Community Consortium workshop on robots for COVID-19.

Homepage: https://people.engr.tamu.edu/robin.r.murphy/index.html

Google Scholar: https://scholar.google.com/citations?hl=en&user=SIcRijoAAAAJ

Affiliation: Texas A&M University

Abstract

One extraordinary property of child development is that they spend a lot of time spontaneously exploring their environment. They are not doing it because they are externally imposed tasks: rather they are driven by different forms of what psychologists call intrinsic motivation, and what we may call curiosity in everyday language. In particular, children are autotelic: during exploratory play, they invent and pursue their own problems.

Such intrinsically motivated exploration has been argued to be key in child development, in particular for solving problems with sparse rewards, as well as learning world models and discovering open-ended skill repertoires.

A number of teams in the world, including mine, have worked now for a while on modelling these capabilities and studying how they can be transferred in AI. This has recently led to the development of a diversity of techniques enabling machines, including robots, to efficiently learn by themselves large repertoires of skills in complex environments. These algorithms can be grouped in the general framework of Intrinsically Motivated Goal Exploration Processes, including implementations based on population-based techniques (Baranes and Oudeyer, 2013; Forestier et al.,2017) and goal-conditioned Deep RL techniques (Nair et al., 2018; Colas et al., 2019; Blaes et al., 2019; Pong et al., 2019, see also Colas, Karch et al. 2020 for a review).

In this tutorial, I will explain what are the basic components of such architectures and illustrate how they work in diverse contexts. I will in particular focus on mechanisms enabling to learn to represent goals, to learn goal-achievement functions, and to sample them using automatic curriculum learning to enable efficient acquisition of skill repertoires in environments with various kinds of distractors.I will also discuss how various approaches have considered goal types ranging from low-level images (Nair et al., 2018) or sensorimotor configuration (Colas et al., 2019) to abstract language-based goals (Colas et al., 2020). Finally, I will also discuss how one can formalize quality-diversity algorithms (Cully and Demiris, 2017; Ecoffet et al., 2021) as a special kind of intrinsically motivated goal exploration process.

Bio

Dr. Pierre-Yves Oudeyer is Research Director (DR1) at Inria and head of the Inria and Ensta-ParisTech FLOWERS team (France). Before, he was a permanent researcher in Sony Computer Science Laboratory for 8 years (1999-2007). He studied theoretical computer science at Ecole Normale Supérieure in Lyon, and received his Ph.D. degree in artificial intelligence from the University Paris VI, France. He has been studying lifelong autonomous learning, and the self-organization of behavioural, cognitive and cultural structures, at the frontiers of artificial intelligence, machine learning, cognitive sciences and educational technologies. He has been developing models of intrinsically motivated learning, pioneering curiosity-driven learning algorithms working in real world robots, and developed theoretical frameworks to understand better human curiosity and autonomous learning. He also studied mechanisms enabling machines and humans to discover, invent, learn and evolve communication systems. He has published two books, more than 100 papers in international journals and conferences, holds 8 patents, gave several invited keynote lectures in international conferences, and received several prizes for his work in developmental robotics and on the origins of language. In particular, he is laureate of the Inria-National Academy of Science young researcher prize in computer sciences, and of an ERC Starting Grant EXPLORERS. He is also editor of IEEE CIS Newsletter on Cognitive and Developmental Systems where he organizes interdisciplinary dialogs in cognitive science, AI and robotics, as well as associate editor of IEEE Transactions on Cognitive and Developmental Systems and Frontiers in Neurorobotics. He has been chair of IEEE CIS Technical Committee on Cognitive and Developmental Systems in 2015-16. He is also working actively for the diffusion of science towards the general public, through the writing of popular science articles and participation in radio and TV programs as well as science exhibitions.

Homepage: http://www.pyoudeyer.com and http://flowers.inria.fr

Google Scholar: https://scholar.google.com/citations?hl=en&user=gCqGj4sAAAAJ

Affiliation: Inria

Abstract

Social learning helps humans and animals rapidly adapt to new circumstances, and drives the emergence of complex learned behaviors. This tutorial focuses on Social Reinforcement Learning, RL algorithms that leverage multi-agent social learning to improve single-agent learning and generalization, multi-agent coordination, and human-AI interaction. We will cover how to use multi-agent training to generate a curriculum of increasingly complex learning tasks, driving agents to learn more complex behavior, and improving zero-shot transfer to unknown, single-agent test tasks. We will discuss how social learning from agents that are present in the environment can provide similar benefits, and enhance human-AI interaction. Finally, we will discuss the problem of learning to coordinate with other agents, review some of the key challenges, and introduce several proposed approaches. We will show how techniques like social influence, which maximizes mutual information between agents’ actions, can improve coordination without depending on assumptions like centralized training or shared rewards. The tutorial aims to demonstrate that multi-agent social learning---whether through competition, cooperation, or merely co-existence---can enhance RL agents’ ability to acquire interesting behavior, generalize to new environments, and interact with people.

Bio

Natasha Jaques holds a joint position as a Research Scientist at Google Brain and Postdoctoral Fellow at UC Berkeley. Her research focuses on Social Reinforcement Learning---developing multi-agent RL algorithms that can improve single-agent learning, generalization, coordination, and human-AI collaboration. Natasha received her PhD from MIT, where she worked on Affective Computing and deep/reinforcement/machine learning. Her work has received the best demo award at NeurIPS 2016, best paper at the NeurIPS workshops on ML for Healthcare and Cooperative AI, and an honourable mention for best paper at ICML 2019. She has interned at DeepMind, Google Brain, and is an OpenAI Scholars mentor. Her work has been featured in Science Magazine, Quartz, the MIT Technology Review, Boston Magazine, and on CBC radio. Natasha earned her Masters degree from the University of British Columbia, and undergraduate degrees in Computer Science and Psychology from the University of Regina.

Homepage: https://natashajaques.ai/

Google Scholar: https://scholar.google.ca/citations?user=8iCb2TwAAAAJ&hl=en

Affiliation: Google Brain & UC Berkeley