ICRA2021 Virtual Workshop
Robot Learning in Real-world Applications: Beyond Proof of Concept

Title: Learning-based navigation and control of quadrupeds and excavators

Abstract: In the recent years, we saw a tremendous progress in the use of machine learning tools in the field of robotics. While it is common practice for perception and classification problems, recent works have shown very promising results in the use of machine learning to control complex systems in challenging environments and overcome the performance (limitations) of classical control approaches. In this talk, I will give some insights into the use of machine learning and in particular, reinforcement learning to control very complex machines such as electric quadrupeds or hydraulic excavators. I will show the challenges that we had to overcome to transfer control policies learned for simulation into real world applications and present come comparisons to classical methods used before. Moreover, I will show some of our works that enable these machine to autonomously navigate cluttered environments and diverse terrain.

Short bio: Marco is an assistant professor for robotic systems at ETH Zurich and co-founder of ANYbotics AG, a Zurich-based company developing legged robots for industrial applications. Marco’s research interests are in the development of novel machines and actuation concepts together with the underlying control, planning, and learning algorithms for locomotion and manipulation. His works find application from electrically actuated quadrupeds like ANYmal to large-scale autonomous excavators used for digital fabrication and disaster mitigation. Marco is part of the National Centre of Competence in Research (NCCR) Robotics and NCCR Digital Fabrication and PI in various international projects (e.g. EU Thing, NI) and challenges (e.g. DARPA SubT).

Title: Deep Predictive Learning: Real-Time Motion Adaptation for Prediction Error Minimization

Abstract: It is almost impossible to design or learn generic optimal models for robots operating in the complex real world. Therefore, it is essential for real robots to have a mechanism to adapt the prediction and actions in real time to minimize the prediction error of the model. We have proposed a framework for "deep predictive learning" based on this idea, and have developed multiple robot applications in collaboration with several companies. In this talk, I will introduce the actual applications of these robots and an overview of our new "moonshot" project, AIREC (AI-driven Robot for Embrace and Care), which aims to develop general-purpose robots using deep predictive learning.

Short bio: Tetsuya Ogata received the B.S., M.S., and D.E. degrees in mechanical engineering from Waseda University, in 1993, 1995, and 2000, respectively. He was a Research Associate with Waseda University from 1999 to 2001. From 2001 to 2003, he was a Research Scientist with the RIKEN Brain Science Institute. From 2003 to 2012, he was an Associate Professor with the Graduate School of Informatics, Kyoto University. Since 2012, he has been a Professor with the Faculty of Science and Engineering, Waseda University. Since 2017, he is a Joint-Appointed Research Fellow with the Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology. Since 2020, he is a director of the Institute of AI and robots, Waseda University. His current research interests include human-robot interaction, dynamics of human-robot mutual adaptation, and inter-sensory translation in robot systems with neuro-dynamical models.

Title: Challenges and future trends of autonomous industrial robots in the MRO value chain

Abstract: With a projected value of 220Bl USD by 2025 and a yearly growth rate of 17% the MRO (Maintenance, Repair and Overhaul) industry bears extensive potential for EU economic growth. This forecast is seriously cut down by the high human injury and death toll rates, mainly associated to harsh working environment and scarcity of unmanned support solutions. The future of industrial autonomous robots features socially intelligent and collaborative hyper-animals, capable of providing a clear response to key MRO barriers by targeting zero injuries for humans while enabling drastic reduction of plants downtime. These robots will leverage unprecedented skills: i) walk and climb with great dexterity and speed across various workspaces indoor/outdoor, being enabled by redundant and reconfigurable locomotion system; ii) autonomous and collaborative execution of tasks from inspection and corrosion removal to soldering and coating, enabled by robot’s capability to embed various processing units; iii) multimodal sensing platform to capture heterogeneous, topological and temporal multi-scale proprioceptive and context related information; iv) AI-based attention and cognitive systems to build full situational consciousness of both internal and external states including the interpretation of the human health and emotional state and v) AI-based long-life learning policies and behavioural decision-maker system integrated with hierarchical control architecture for safe and reliable planning and execution of robot’s MRO missions. The successful accomplishment of such challenging objectives relies upon the establishment of new transdisciplinary competences and the synergy between multiple stakeholders with complementary skills operating in the MRO value chain.

Short bio: Anna Valente is a Professor of Industrial Robotics. She got a PhD in Manufacturing Technologies and Production Systems at the Politecnico di Milano and a Post-doctorate in interoperability for adaptive factories from University of Bath, UK. Since 2006, she has been working in cooperation with big research institutions and industrial stakeholders operating in the manufacturing value chain. She is currently Head of the Laboratory for Automation, Robots and Machines with SUPSI-DTI-ISTePS where the core research deals with the design, engineering and prototyping up to TRL 7 industrial solutions integrating advanced process chains to realize high value added products. She is the author of two books and more than 100 papers on system (re)configuration, robotics and control platforms. Since 2012 she has been coordinating several European Funded Projects under FP7 and H2020 Frameworks. In 2019 she was awarded with the Woman-Led Innovation and the Grand Prix for Innovation from European Commission. She is an associate member of CIRP - The International Academy for Production Engineering. She serves the Innosuisse Agency as expert since 2018 and she is member of the Swiss Science Council since 2020.

Title: Reinforcement learning approach to real-world robot control: the challenges for beyond proof of concept

Abstract: Reinforcement learning enables robots to learn desirable behaviors from empirical samples collected by trial and error. However, since it requires many samples, it is not easy to apply it to real-world robotic systems, where the sample cost (i.e., the time and human burden required to collect empirical samples) is high. In this talk, I will introduce our recent development on sample-efficient reinforcement learning methods and their applications to real-world robots such as cloth manipulation robots, a small boat, a waste crane in the garbage incineration plant, etc. I will also discuss various possibilities and directions to enhance the practicality of reinforcement learning approaches further.

Short bio: Takamitsu Matsubara received his Ph.D. in information science from the Nara Institute of Science and Technology, Nara, Japan, in 2007. From 2005 to 2007, he was a research fellow (DC1) of the Japan Society for the Promotion of Science. From 2013 to 2014, he was a visiting researcher of the Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands. He is currently an associate professor at the Nara Institute of Science and Technology and a visiting researcher at the ATR Computational Neuroscience Laboratories, Kyoto, Japan. His research interests are machine learning and control theory for robotics.

Title: Guiding exploration for real-world robotic reinforcement learning

Abstract: In terms of safety and learning efficiency, exploration remains one of the main challenges for robotic reinforcement learning. In this presentation, I will present two methods that address this challenge. First, "generalized state-dependent exploration", which reduces jerky motions for safer exploration on real robotic systems. Second, "shared control templates", which use prior knowledge about relevant frames of reference to guide exploration. Applications to tendon-driven robots and assistive systems highlight the relevance of these methods to learning on complex physical robotic systems. In this context, I will also present the VeriDream project, which aims to identify and resolve issues that arise when applying open-source machine learning software to industrial use cases. Finally, if time permits, I would like to initialize a discussion on what it means to do "real-world" experiments.

Short bio: Freek Stulp is the head of the department of Cognitive Robotics at the Institute of Robotics and Mechatronics at the German Aerospace Center (DLR) near Munich. There, he leads a team of 25 researchers who focus on interactive robotic skill learning and explainable/transferable knowledge representations for robots. Before, he was an assistant professor at the Ecole Nationale Superieure de Techniques Avancees (ENSTA-Paris), and held post-doc fellowships at the University of Southern California (Los Angeles) and the Advanced Telecommunications Research Institute International (Kyoto). He received his doctorate degree in Computer Science from the Technische Universität München. With Evangelos Theodorou and Stefan Schaal, he is the winner of the King-Sun Fu Best Paper Award of the IEEE Transactions on Robotics for the year 2012, for a paper on reinforcement learning of robotic manipulation skills.

Title: Data-efficient learning for robotics

Abstract: Is there a systematic way in which we can incorporate insights from cognitive science and neuroscience into the models we build in machine learning? Can this combination lead to models that exhibit strong generalization, and robots that understand everyday concepts? In this talk, I will describe the approach Vicarious is taking in getting to yes on these questions. First, I will describe Recursive Cortical Network (RCN), a probabilistic generative model for vision, and explain the neuroscience and cognitive science insights that led to this model. Using those insights, RCN was able to fundamentally beat the defense of text-based CAPTCHAs and outperform deep neural nets on several benchmark tasks while being orders of magnitude data efficient. RCN is currently deployed on our robots in warehouses doing real-world work, with the same data efficiency and flexibility. I will then describe a Visual Cognitive Computer that uses RCN as a fundamental building block and learns concepts as programs on this computer. Learning concepts enable a simple way to communicate tasks to robots. Third, I will show recent work on how cognitive maps can be learned from purely ordinal data, giving robots an easy way to represent real and conceptual spaces.

Short bio: Dr. Dileep George is co-founder and CTO of Vicarious AI, where he leads the development of a general AI layer for robotics, enabling their applications in high-mix, high-changeover settings. Vicarious has pioneered brain-inspired and data-efficient AI algorithms. They gained world-wide attention for fundamentally breaking text-based CAPTCHAs with very little training data, and for teaching robots to understand abstract concepts. Dileep has authored several influential papers, published in Science, Science Robotics,. NeurIPS, ICML, and CVPR, and his research has been featured in NYT, WSJ and NPR. Before founding Vicarious, Dileep was CTO of Numenta, an AI company he co-founded with Jeff Hawkins and Donna Dubinsky during his graduate studies at Stanford. Before Numenta, Dileep was a Research Fellow at the Redwood Neuroscience Institute. Dileep’s research on hierarchical models of the brain earned him a PhD in Electrical Engineering from Stanford University. He has an MS in Electrical Engineering from Stanford and a B.Tech in Electrical Engineering from IIT Mumbai.