Soft hands are robotic systems that embed compliant elements in their mechanical design. This enables an effective adaptation with the items and the environment, and ultimately, an increase in their grasping performance. These hands come with clear advantages in terms of ease-to-use and robustness if compared with classic rigid hands, when operated by a human. However, their potential for autonomous grasping is still largely unexplored, due to the lack of suitable control strategies. In this talk, I will present an approach to enable soft hands to autonomously grasp objects, starting from the observations of human strategies. The approach combines minimalistic multi-modal sensing, the purposeful exploitation of the intrinsic softness of the artificial hands and deep learning techniques. Applications to reflex grasping in advanced human-robot-interaction and autonomous grasping are presented and discussed.

We introduce the latest advances in robotic walking under two respects. On one hand, passive walking dynamics is deeply studied with the help of an optimization framework for the design and analysis of biped walkers. The framework goes into great detail in capturing the dynamic constraints imposed by the contact dynamics. On the other hand, useful heuristics are investigated in order to efficiently warm-start the optimization problem. This is the subject of the European projects MEMMO and IREPA.

In this talk I will present recent work from our group on modeling contact with Coulomb friction. While numerous approximations of Coulomb friction are widely used in practice, a complete treatment remains elusive: the maximum dissipation principle, part of the Coulomb model stating that friction uses up as much energy as possible during contact slip, imposes non-convex and non-smooth constraints that are difficult to integrate in a computationally efficient solver. For quasi-static grasp analysis, this talk will present a relaxation that allows the relevant friction constraints to be solved as a Mixed Integer Program, as well as an algorithm that successively refines this relaxation locally in order to efficiently obtain solutions to arbitrary accuracy. It is, to the best of our knowledge, the first time that a model has been proposed that can handle three-dimensional frictional constraints that include the maximum dissipation principle, up to arbitrary accuracy and in a computationally efficient fashion. This framework allows us to solve general queries regarding quasi-static grasp stability, and could have applicability for contact models in other settings, such as dexterous manipulation or locomotion.

Learning a control policy from offline simulations is a tempting idea, but success in this area has proven to be elusive for complex robots like our humanoids. One particular challenge is the way we handle contact between the robot and the world. A controller which decides when and where to make contact must be able to reason about a complex mixture of discrete and continuous states, making optimization of such a controller difficult even when running offline and in simulation. By solving enough example optimizations offline, we might hope to learn a general policy, but collecting samples of the optimal policy is difficult: local optimization methods tend to be fast but may produce samples which are only locally optimal, while mixed-integer optimizations can provide globally optimal samples but may take an extremely long time to do so.

In this talk, I will discuss how we balanced online and offline computation to take advantage of the specific guarantees provided by a mixed-integer representation of contact. Using offline optimizations we learn an approximation of the global cost-to-go for a simulated robot, and this approximation provides enough fidelity to inform the behavior of a short-horizon model-predictive controller which can run at real-time. Although neither the offline value function nor the online MPC is sufficient to balance the robot on its own, combining both allows the controller to make effective use of multiple contact points to balance the robot.

Robust planning in real-world robotics applications requires the capability of adjusting the plan using state measurements continuously and in real-time. Many of today's approaches for online motion planning for legged systems have achieved this capability through task decomposition and model reduction. Such simplifications have been vital in making the computation of motion planning fast in finding solutions in real-time. However, simplifications generally come at the cost of limiting the maneuverability of the robot. Moreover, transferring solutions found by whole-body optimization-based methods to robotic hardware poses a challenging task. The presence of unmodeled dynamics often renders such methods impractical to employ on hardware since the optimization tends to fully exploit the provided model to perform dynamic tasks and often parts of this solution relies on unrealistic model assumptions such as perfect actuation.

In this talk, I will present our latest results on the application of the Model Predictive Control (MPC) for whole-body motion planning and control of legged robots. Our constrained nonlinear MPC approach is a Dynamic Programming approach with Gauss-Newton approximation. I will discuss our method for employing a multi-threading scheme for computing the optimal solution, which, significantly improves the throughput of the MPC loop. Furthermore, I will describe how we deal with model errors when deploying the MPC on hardware, and discuss our method for incorporating actuator bandwidth limitations into the MPC. Finally, I will explain our approach for implementing MPC feedback strategy (in contrast to the commonly-used feedforward strategy) on real hardware which bridges the gap between low update-rate MPC and high-rate execution of torque commands using only an onboard computer with moderate computational power.

There has been a recent surge in enthusiasm for using machine learning to learn robot control from experience, but this enthusiasm is often tempered by the difficulty of obtaining satisfactory performance in seemingly simple tasks. In this research we explore the structure of optimal decisions in nonlinear optimal control, and how discontinuities in this structure pose inherent difficulties for neural network learning. We propose a new supervised, discontinuity-sensitive training scheme, based on Mixture of Experts (MoE) models, which composes multiple models, each of which performs well over a region where output is continuous. Experiments indicate dramatic improvements in control performance for dynamic underactuated systems compared to standard neural networks. This is joint work with Gao Tang.

In this talk, I will discuss my work on closing the loop in robotic manipulation, moving towards robots that can better perceive their environment and react to unforeseen situations. I will present real-time control strategies for dynamical systems that involve frictional contact interactions. I will begin the talk by discussing model-based control architectures that effectively deal with the combinatorial complexity associated with determining optimal sequences of contact modes. I will show that formulating the search for optimal modes separately from the search for optimal control inputs, by leveraging machine learning methods, can reduce the online computational requirements to solving a convex quadratic program. Finally, I will describe my current efforts on developing a framework for tactile dexterity, which leverages contact sensing for dexterous manipulation using a dual-arm manipulation robotic system.

Humanoid robots are complex dynamical systems with many degrees of freedom and control inputs, such that the generation of whole-body motions is a very challenging task. Among the difficulties to solve are redundancy, feasibility, underactuation, changing contacts, and dynamic stability control. In this talk I will present some of the research performed in my group on the generation of whole-body motions, in particular walking motions, for humanoid robots. Optimal control approaches based on physical models of the robot can help to solve all the above mentioned difficulties but their solution is often too time consuming. On the other hand, model-free learning methods are not able to explore the entire space due to the high number of dimensions and the immediate loss of stability and the danger to break the robot. We propose to combine optimal control methods with learning methods in two different ways:

• to start learning over the real system from the converged solution in order to remove the inevitable model-reality mismatch
• to use optimal control solutions for a complex system as training data to learn movement primitives that can then be used to synthesize new motions in a very short time.

I will also discuss the inverse control approach which is similar to inverse reinforcement learning and which allows to identify behavior models (in terms of optimization cost functions) from human motion capture recordings. These can then be applied to robot models to generate bio-inspired behavior in humanoid robots by optimal control. We have applied this optimization-based transfer of behaviors from humans to robots using models of different levels of complexity.

DeepMind is working on some of the world’s most complex and interesting research challenges, with the ultimate goal of solving artificial general intelligence (AGI). We ultimately want to develop an AGI capable of dealing with a variety of environments. A truly general AGI needs to be able to act on the real world and to learn tasks on real robots. Robotics at DeepMind aims at endowing robots with the ability to learn how to perform complex manipulation tasks. This talk will give an introduction to DeepMind with specific focus on robotics, control or reinforcement learning.

Soft Robots, i.e. robots that embed compliant elements in their links and/or joint with fixed and/or adjustable torque deflection characteristic, are becoming more and more popular in the Robotics research community, e.g. SORO (Soft Robotics) has today the highest Impact Factor (5.057) among the Robotics Journals. Thanks to their enriched and in some cases adaptable dynamics, Soft Robots promise to be more resilient, efficient in cyclic motions, and adaptable to unstructured environments than conventional rigid robots. Optimal Planning and Control Techniques have been adopted to theoretically and experimentally prove which robot compliance is better depending on the task and the competitive advantage w.r.t. the rigid case, but these results are confined to one-DoF proof of concepts. The transition to multi-DoF systems is extremely challenging because to exploit the full potential of Soft Robots a careful plan and control of robot motion taking into account the full robot dynamics is mandatory. A wrong motion planning and control could not only compromise robot performance but also substantially adulterate its compliant nature.

This talk first reviews the hardest challenges to plan and control Soft Robots motion, then highlight how optimal planning and control techniques are one of the best candidate to tackle this challenges, and finally will presents some of the most recent findings in motion control, manipulation, and locomotion of soft robots.

Algorithms for trajectory optimization through contacts have achieved performances hardly imaginable a few years ago, but relatively high computation times make their online application still unfeasible. Trajectory libraries represent an effective middle ground where, after a suitable problem parameterization, selected problem instances are solved offline and stored, reducing the online workload to slight local adaptations. In the construction of these libraries, modeling robots as fully nonlinear systems confronts us with the weakness of the results available in nonlinear parametric optimization, and relegates us to the use of intrinsically local and insufficiently robust solvers.

In this talk I will argue that, oftentimes, PieceWise-Affine (PWA) models capture the essential dynamic features of a robot interacting with the environment through contacts. As opposed to the generic nonlinear case, parametric optimal control of PWA systems (a.k.a. explicit hybrid MPC) lends itself to much stronger theoretical results, both in the construction and the use of trajectory libraries. I will present various approaches for approximate explicit hybrid MPC and I will discuss how to tackle from multiple sides the principal bottleneck of these techniques, namely, sample complexity.