Learning and Inference in Robotics:

Integrating Structure, Priors and Models

Robotics: Science and Systems workshop on June 29, 2018

Schedule

08:55 - 09:00 | Organizers | Introduction

09:00 - 09:30 | Drew Bagnell | There Is No Deep Reinforcement Learning

09:30 - 10:00 | Angela Schoellig | Combining Models and Data for Enhanced Robot Control and Decision Making

10:00 - 10:30 | Poster Session and Coffee Break | #2, #3, #5, #7, #8, #9, #10, #11, #12, #16, #17

10:30 - 11:00 | Spotlight Contributed Talks | #1, #4, #6, #13, #14, #15

11:00 - 11:30 | Jeannette Bohg | On the Role of Structure in Learning for Robot Manipulation [slides][talk]

11:30 - 12:00 | Martin Riedmiller | Towards less Structure and Priors in Deep RL for Robotics [talk]

12:00 - 13:30 | Lunch Break

13:30 - 14:00 | Marc Toussaint | Inference vs Optimization Formulations of Planning and Imitation Learning

14:00 - 14:30 | Kostas Daniilidis | Building equivariant networks [talk]

14:30 - 15:00 | Poster Session and Coffee Break | #1, #4, #6, #13, #14, #15, #18, #19, #20, #21

15:00 - 15:30 | Luca Carlone | Robust and Verifiable Robot Perception for Safety-Critical Applications [slides][talk]

15:30 - 16:00 | Zico Kolter | Convexity in a deep world [talk]

16:00 - 17:00 | Panel Discussion

Invited Talk Abstracts

09:00 - 09:30 | Drew Bagnell | There Is No Deep Reinforcement Learning

We'll explore the exciting recent advances in Reinforcement Learning, and note comparisons and contrasts with part achievements in the field. We take note that Reinforcement Learning and neural networks have been closely tied for decades (i.e., "Neuro-dynamic Programming"), and that advances an RL and advances in neural architectures have been nearly completely orthogonal. We identify key issues that remain incompletely addressed in RL (with neural networks or otherwise), from stability of bootstrapping to the use of value functions and policies. In the last part of the talk, we zoom into one of these still-open issues critical for RL sample efficiency in robotics. We attempt to understand the trade-offs between black-box (parameter space) and white-box (action space) explorations. We identify a set of key parameters via theory and experiment that suggest when white-box methods are likely to be more sample efficient and visa-versa. This is joint work with Anirudh Vemula, Wen Sun, Max Likhachev, and others.

09:30 - 10:00 | Angela Schoellig | Combining Models and Data for Enhanced Robot Control and Decision Making

In contrast to computers and smartphones, the promise of robotics is to build devices that can physically interact with the world. The interaction of robots with the physical world may be as simple as moving on roads or in the air, and can be as complex as physically collaborating with humans. Envisioning robots to work in human-centered and interactive environments challenges current robot algorithm design, which has been largely based on a-priori knowledge about the system and its environment. In this talk, I will show how we combine models and data to achieve safe and high-performance robot behavior in the presence of uncertainties and unknown effects. Our work focuses on learning robot control and decision making strategies, and enables robots to adapt their behavior as they move in the world. In this talk, I will highlight how we use "structure" and prior knowledge to (i) appropriately place the learning in the overall system architecture (including choosing the inputs and outputs of the learning module), (ii) enable online learning (i.e., enable the robot to move in the real world and start gathering data), (iii) design data-efficient algorithms, and (iv) provide provable performance and safety guarantees for the learning. An important characteristic of our work is that we try to identify where unknown effects affect the robot behavior but that we do not make any strong assumptions about the unknown effects themselves. We demonstrate our algorithms on self-flying and -driving vehicles, as well as on mobile manipulators. More information and videos at: www.dynsyslab.org.

11:00 - 11:30 | Jeannette Bohg | On the Role of Structure in Learning for Robot Manipulation [slides][talk]

Recent approaches in robotics follow the insight that perception is facilitated by interaction with the environment. First, this creates a rich sensory signal that would otherwise not be present. Second, knowledge of the sensory dynamics upon interaction allows prediction and decision-making over a longer time horizon. To exploit these benefits of Interactive Perception for capable robotic manipulation, a robot requires both: methods for processing rich, sensory feedback and feedforward predictors of the effect of physical interaction. In the first part of this talk, I will present a method for processing rich, sensory feedback to perform motion-based segmentation of an unknown number of simultaneously moving objects. The underlying model estimates dense, per-pixel scene flow that is then followed by clustering in motion trajectory space. We show how this outperforms state-of-the-art in scene flow estimation and multi-object segmentation. In the second part, I will present a method for predicting the effect of physical interaction with objects in the environment. The underlying model combines an analytical physics model and a learned perception part. In extensive experiments, we show how this hybrid model outperforms purely learned models in terms of generalisation. In both projects, we found that introducing structure greatly reduces training data, eases learning and provides extrapolation. Based on these findings, I will discuss the role of structure in learning for robot manipulation.

11:30 - 12:00 | Martin Riedmiller | Towards less Structure and Priors in Deep RL for Robotics [talk]

Being able to do reinforcement learning from scratch on real robots is the long-term goal on our research agenda. A particular challenge in these scenarios is that methods need to be highly data-efficient, since data-collection on real robots is time intensive and often expensive. Bringing in structure, priors or models is often the way of choice to overcome this problem. I will discuss several examples of real world applications of RL where bringing in structure played a crucial role for the solution. I will then also discuss downsides of these approaches, and show some recent developments that try to reduce the amount of prior knowledge to a minimum, while still be able to solve complex tasks from scratch.

13:30 - 14:00 | Marc Toussaint | Inference vs Optimization Formulations of Planning and Imitation Learning

Formulating planning/control, learning and perception coherently as probabilistic inference is intriguing. But in practice, optimization formulations seem to offer somewhat different ways to express the structure, e.g. in sequential manipulation planning or learning manipulations from demonstration.  In this talk I discuss such examples and eventually raise questions on how to reconcile this with a fully probabilistic formulation.

14:00 - 14:30 | Kostas Daniilidis | Building equivariant networks [talk]

Traditional convolutional networks exhibit unprecedented robustness to intraclass nuisances when trained on big data. Geometric transformations like rotations has been tackled with data augmentation, too. Several approaches have shown recently that data augmentation can be avoided if networks are structured such that feature representations are transformed the same way as the input, a desirable property called equivariance. We show in this talk that global equivariance can be achieved for the case of 2D scaling, rotation, and translation as well as 3D rotations. We show state of the art results using an order of magnitude lower capacity than competing approaches.

15:00 - 15:30 | Luca Carlone | Robust and Verifiable Robot Perception for Safety-Critical Applications [slides][talk]

Real-time computational perception is a crucial building block of modern robots and autonomous vehicles. When robotics technologies are used in safety-critical applications (e.g., intelligent transportation, disaster response, military applications of drones) failures in robot perception may result in human casualties. Therefore, it is of paramount importance to develop verification methods as well as perception techniques with provable performance guarantees, that can enable failure detection and mitigation. In this talk, I discuss our work on fast provably-correct solvers for Simultaneous Localization and Mapping (SLAM) and cover recent extensions to the case of SLAM with outliers resulting from perceptual failures. Then, I expand the discussion beyond SLAM, and explore the opportunity of using “structure” and model-based techniques to equip learning-based perception methods with formal performance guarantees.

15:30 - 16:00 | Zico Kolter | Convexity in a deep world [talk]

Spotlight Contributed Talks

Contributed Papers