Speakers

Recent Progress on Certifiable Perception Algorithms

Abstract: Many perception and state estimation problems in robotics and computer vision can be formulated as optimization problems. In many of these problems, failure to converge to the optimal solution of the optimization problem leads to computing inaccurate and unreliable state estimates. In safety-critical applications, from self-driving to drones and service robots, these failures can cascade to system-level failure and put human life at risk. This talk reviews the idea of certifiable perception algorithms, which are able to solve hard optimization problems arising in robot perception and certify the optimality of the resulting solution. I will also discuss recent advances in this research area, including faster solvers and applications where measurements are collected over time.

Geometry-aware Bayesian optimization for efficient robot skill learning

Abstract: Fast and data-efficient skill learning is a key challenge in robotics, where robots often need to generalize previously-learned policies to unforeseen settings. In this context, Bayesian optimization has emerged as a powerful tool to efficiently optimize parametric policies in challenging robotics scenarios. However, improving the performance, scalability, and safety of Bayesian optimization techniques remains a central goal for robotics applications.

This talk introduces the idea of incorporating inductive bias into Bayesian optimization algorithms by leveraging task-specific information on the geometry of the search space. Many robotic parameters belong to non-Euclidean spaces with inherent geometric properties. For instance, orientations can be viewed as elements of the unit sphere or the special orthogonal group; control gains, inertia, and manipulability ellipsoids lie in the manifold of symmetric-positive-definite matrices; and robot joint configurations belong to the torus. First, I will discuss how geometry-awareness can be brought into Bayesian optimization by leveraging Riemannian manifold theory, both in the surrogate model and in the optimization of the acquisition function. Second, I will discuss how this framework can be extended to optimize high-dimensional non-Euclidean objective functions. Finally, I will show that leveraging geometry as inductive bias for Bayesian optimization generally leads to significant performance improvements in a variety of benchmarks and robotics applications. 

Certifiable and Distributed Nonconvex Optimization

InDi: An Indirect/Direct method for Optimal Control Problems

Abstract: Optimal control problems are widely present in engineering and with an increasing number of successful applications. Direct methods emerge as the most popular approach to solve these problems because they are simpler to implement and require less symbolic computation. Instead, the indirect approach requires more symbolic manipulation, but through the Pontryagin maximum principle, it allows for a better understanding of the solution's structure, and the control part is captured more naturally.

This work introduces a tailored derived numerical method for solving optimal control problems which can be seen interchangeably as derived from a direct approach or from an indirect approach. The search for a connection between the solution of the discretization of direct methods with that obtained with the discretization of indirect methods is not a new idea. The principle of covector mapping (CMP) has demonstrated a mapping that makes the solution of the direct method equivalent to that of the indirect method when using pseudospectral collocation techniques. Similarly to CMP, this study introduces  a mapping approach for finite difference scheme that connects the discretized equations of indirect methods with those of direct transcription applied to implicit dynamics. Revisiting the discretization of the indirect method as that obtained from a direct transcription allows us to hybridize the solution techniques and algorithms of the two families of methods by exploiting their respective strengths.

This innovative formulation holds great promise for the indirect method, facilitating the adoption of regularization techniques commonly associated with direct approaches, all while retaining the explicit solution of the control as a function of the states and co-states within a bi-level optimization framework that significantly improves the numerical performance. This flexible formulation allows the introduction of numerical techniques derived from direct methods, as proximal methods, with the aim of obtaining a robust solver while maintaining the efficient structure of Newton's damped method used for the solution of the indirect method. Moreover, the proposed mapping extends its utility to encompass implicit dynamics, thereby providing a comprehensive solution strategy for a broader array of optimal control problems and applications.

Graphs of Convex Sets

Abstract: We've recently introduced Graphs of Convex Sets  (GCS) as a general framework for optimizing mixed continuous + combinatorial optimization problems that are naturally specified on a graph, and have talked mostly so far about its applications to motion planning. In this talk, I'll try to give the broader view of GCS, which includes natural extensions of many network flow problems, and which connects deeply to many results in systems and control.

Numerical methods for solving nonsmooth optimal control problems

Abstract: In control applications contact and friction phenomena are often modeled as nonsmooth dynamical systems. In order to optimally control such systems or to implement MPC controllers, nonsmooth optimal control problems have to be solved. This involves three steps: (re)formulating the problem, discretizing it, and solving it. In contrast to standard smooth trajectory optimization, these steps are highly nontrivial. In this talk, we review recent developments in all three steps and show with counterexamples why naive approaches must fail. For the first step, we investigate the time-freezing reformulation - which simplifies systems with state jumps; for the second, finite elements with switch detection (FESD) - which ensures high accuracy and correct sensitivities; and we discuss how to efficiently solve mathematical programs with complementarity constraints - which frequently arise in the third step. Examples show how special care is needed in all three steps to obtain efficient and meaningful solutions.

Care and Feeding of MPC: Model-Predictive Control of the Electric Atlas

Abstract: The Atlas humanoid robot can walk, jump, run, and manipulate, and it can combine those behaviors seamlessly to create flashy dance moves, parkour sequences, or dynamic throws. In this presentation, I'll discuss how we've built the generations of model-predictive control that power Atlas's behaviors, as well as the challenges of deploying MPC in reality. Finally, I'll show how we're developing new behaviors on top of that controller to allow the new electric Atlas to start doing useful work in the world. 

NLP Sampling:
Combining MCMC and NLP Methods for Diverse Constrained Sampling 

Abstract: I will discuss relations between typical MCMC (e.g. Langevin, manifold & polytope sampling) and constrained optimization techniques, and argue for a unified approach. My original motivation is robustness in task-and-motion planning.

Differentiable optimization for robotics

Abstract: Optimization is a crucial technology for robotics and provides functionality such as optimal control, motion planning, state estimation, alignment, manipulation, tactile sensing, pose tracking, and safety mechanisms. These solvers are often integrated with learned models that estimate and predict non-trivial parts of the world. *Differentiable optimization* enables the learned model to receive a learning signal from these downstream optimization problems. This signal encourages the model to improve on regions that are important for the optimization problem to work well, rather than making accurate predictions under a supervised loss. This talk will overview the foundations, applications, and recent advancements on these topics.

Differentiable simulation

Abstract: Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and for adapting simulators to reality. However, existing approaches to differentiable simulation have focused on scenarios with well-behaved gradients, such as systems with mostly smooth dynamics. In this talk, Rika will outline the challenges arising with contact-rich interactions, deformables and fluids, where simulators yield rugged optimization landscapes. Rika will present an approach that combines global search and local optimization to overcome the above challenges, and also touch upon other interesting directions that benefit from differentiable simulation, such as differentiable rendering and optimizing tool morphologies. Then, Yang will present another perspective on tackling the optimization challenges, which employs large language models (LLMs) to devise stage-by-stage plans, executed via a closed-loop predictive control using differentiable physics for deformables.

Differentiable simulation

Abstract: Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and for adapting simulators to reality. However, existing approaches to differentiable simulation have focused on scenarios with well-behaved gradients, such as systems with mostly smooth dynamics. In this talk, Rika will outline the challenges arising with contact-rich interactions, deformables and fluids, where simulators yield rugged optimization landscapes. Rika will present an approach that combines global search and local optimization to overcome the above challenges, and also touch upon other interesting directions that benefit from differentiable simulation, such as differentiable rendering and optimizing tool morphologies. Then, Yang will present another perspective on tackling the optimization challenges, which employs large language models (LLMs) to devise stage-by-stage plans, executed via a closed-loop predictive control using differentiable physics for deformables.

Meta-Learning for Robot Estimation and Control via Bilevel Optimization

Abstract: Nonlinear optimization plays a pivotal role in robot state estimation, planning, and control, which ensures precise and efficient operations. However, their effectiveness hinges on the meticulous tuning of various hyperparameters, such as weighting matrices, reference waypoints, and horizon lengths. In dynamic and complex environments, real-time adaptation of these hyperparameters becomes indispensable for the robust functioning of the robots. To this end, we develop efficient algorithms for generating these hyperparameters using deep neural networks and thereby achieve fast environment adaptation and robust control in challenging robotic tasks. They are essentially solving a bilevel optimization, but utilize the inherent sparsity in the low-level dynamics-constrained optimization problem. We will present several interesting applications in quadrotor control, including aerodynamic disturbance estimation, agile flight control, and cooperative transport by distributed learning.