Accepted Papers

Abstract: We provide a concise overview of our progress in identifying and certifying globally optimal solutions of state estimation problems in robotics. We give a summary of the theoretical background, providing pointers for novices in the field to learn about this topic. We then given an overview of the problems we have treated, putting them in a unified framework to make it easier to find, compare, and build upon them. We discuss our methods for simplifying the generation and analysis of these solutions, and finally, present advances to allow to embed them in real-world robotics pipelines. We conclude with a discussion of important open research problems.

Paper: Link

Abstract: Recent successes in applying reinforcement learning (RL) for robotics has shown it is a viable approach for constructing robotic controllers. However, RL controllers can produce many collisions in environments where new obstacles appear during execution. This poses a problem in safety-critical settings. We present a hybrid approach, called iKinQP-RL, that uses an Inverse Kinematics Quadratic Programming (iKinQP) controller to correct actions proposed by an RL policy at runtime. This ensures safe execution in the presence of new obstacles not present during training. Preliminary experiments illustrate our iKinQP-RL framework completely eliminates collisions with new obstacles while maintaining a high task success rate.

Paper: Link

Slides: Link

Video: Link

Abstract: Ensuring safety for learning-based techniques, especially reinforcement learning, is a critical step toward deploying robot learning in the real world. This problem poses several challenges, such as deriving safety policies from given constraints, tackling challenges in dynamic environments, designing constraints that consider long-term safety, and addressing uncertainty. ATACOM attempts to construct a Constraint Manifold and a safe action space tangent to this manifold, exploiting the prior knowledge of robot dynamics and differentiable constraints. This paper provides an overview of the ATACOM method of handling various types of constraints, including learnable ones. We show that ATACOM effectively incorporates prior knowledge into learning-based methods to solve high-dimensional tasks with complex constraints.

Paper: Link

Video: Link

Abstract: Recent advancements in multiple robotics domains have paved the way for the widespread deployment of autonomous mobile robots. In industrial environments, for example, these robots autonomously transport goods, while in residential settings, robotic vacuums are increasingly common. Despite the rapid progress in technologies enabling these advancements, the development of tools and workflows for the verification and validation of autonomous operations has lagged. A significant challenge is integrating safety analysis early in the development process, often leading to safety assessments being conducted late in the development life cycle. This delay increases the risk of insufficiently thorough analyses. This paper introduces an innovative approach to assess the safety of mobile robots through a comprehensive Model-based Safety Analysis (MBSA) method. This method seamlessly integrates safety assessment artifacts into the design process, improving consistency and automation, reducing manual tasks, and minimizing errors. Additionally, simulation data is utilized to confirm the assumptions that underlie the safety analysis, ensuring their validity. The approach is demonstrated through the modeling and simulation of a wheeled robot system. The paper details the methodology, its practical application, and concludes with insights gained from the implementation.

Paper: Link

Slides: Link

Video: Link

Abstract: This paper presents an overview of the Burer-Monteiro method (BM), a technique that has been applied to solve robot perception problems to certifiable optimality in real-time. BM is often used to solve semidefinite programming relaxations, which can be used to perform global optimization for non-convex perception problems. Specifically, BM leverages the low-rank structure of typical semidefinite programs to dramatically reduce the computational cost of performing optimization. This paper discusses BM in certifiable perception, with three main objectives: (i) to consolidate information from the literature into a unified presentation, (ii) to elucidate the role of the linear independence constraint qualification (LICQ), a concept not yet well-covered in certifiable perception literature, and (iii) to share practical considerations that are discussed among practitioners but not thoroughly covered in the literature. Our general aim is to offer a practical primer for applying BM towards certifiable perception.

Paper: Link

Abstract: An outstanding challenge for robotic systems like autonomous vehicles is ensuring safe interaction with humans without sacrificing performance. Existing safety methods often neglect the robot’s ability to learn and adapt at runtime, leading to overly conservative behavior. This paper proposes a new closed-loop paradigm for synthesizing safe control policies that explicitly account for the robot’s evolving uncertainty and its ability to quickly respond to future scenarios as they arise, by jointly considering the physical dynamics and the robot’s learning algorithm. We leverage adversarial reinforcement learning for tractable safety analysis under high-dimensional learning dynamics and demonstrate our framework’s ability to work with both Bayesian belief propagation and implicit learning through large pre-trained neural trajectory predictors.

Paper: Link

Poster: Link

Video: Link

Abstract: Modern neural trajectory predictors in autonomous driving are developed using imitation learning (IL) from driving logs. Although IL benefits from its ability to glean nuanced and multi-modal human driving behaviors from large datasets, the resulting predictors often struggle with out-of-distribution (OOD) scenarios and with traffic rule compliance. On the other hand, classical rule-based predictors, by design, can predict traffic rule satisfying behaviors while being robust to OOD scenarios, but these predictors fail to capture nuances in agent-to-agent interactions and human driver's intent. In this paper, we present RuleFuser, a posterior-net inspired evidential framework that combines neural predictors with classical rule-based predictors to draw on the complementary benefits of both, thereby striking a balance between performance and traffic rule compliance. The efficacy of our approach is demonstrated on the real-world nuPlan dataset where RuleFuser leverages the higher performance of the neural predictor in in-distribution (ID) scenarios and the higher safety offered by the rule-based predictor in OOD scenarios.

Paper: Link

Abstract: Learning-based neural network (NN) control policies have shown impressive empirical performance in a wide range of tasks in robotics and control. However, formal (Lyapunov) stability guarantees over the region-of-attraction (ROA) for NN controllers with nonlinear dynamical systems are challenging to obtain, and most existing approaches rely on expensive solvers such as sums-of-squares (SOS), mixed-integer programming (MIP), or satisfiability modulo theories (SMT). In this paper, we demonstrate a new framework for learning NN controllers together with Lyapunov certificates using fast empirical falsification and strategic regularizations. We propose a novel formulation that defines a larger verifiable region-of-attraction (ROA) than shown in the literature, and refines the conventional restrictive constraints on Lyapunov derivatives to focus only on certifiable ROAs. The Lyapunov condition is rigorously verified post-hoc using branch-and-bound with scalable linear bound propagation-based NN verification techniques. The approach is efficient and flexible, and the full training and verification procedure is accelerated on GPUs without relying on expensive solvers for SOS, MIP, nor SMT. The flexibility and efficiency of our framework allow us to demonstrate Lyapunov-stable output feedback control with synthesized NN-based controllers and NN-based observers with formal stability guarantees, for the first time in literature.

Paper: Link

Abstract: In this paper, we seek to learn a robot policy guaranteed to satisfy state constraints. Typical RL algorithms only encourage constraint satisfaction through reward shaping. However, such soft constraints cannot offer safety guarantees. To address this gap, we propose POLICEd RL, a novel RL algorithm explicitly designed to enforce affine hard constraints in closed-loop with a black-box environment. Our key insight is to make the learned policy be affine around the unsafe set and to use this affine region as a repulsive buffer to prevent trajectories from violating the constraint. We prove that such policies guarantee constraint satisfaction. Our results demonstrate the capacity of POLICEd RL to enforce hard constraints in robotic tasks while significantly outperforming existing methods.

Paper: Link

Abstract: Grounding the common-sense reasoning of Large Language Models (LLMs) in physical domains remains a pivotal yet unsolved problem for embodied AI. Whereas prior works have focused on leveraging LLMs directly for planning in symbolic spaces, this work uses LLMs to guide the search of task structures and constraints implicit in multi-step demonstrations. Specifically, we borrow from manipulation planning literature the concept of mode families, which group robot configurations by specific motion constraints, to serve as an abstraction layer between the high-level language representations of an LLM and the low-level physical trajectories of a robot. By replaying a few human demonstrations with synthetic perturbations, we generate coverage over the demonstrations’ state space with additional successful executions as well as counterfactuals that fail the task. Our explanation-based learning framework trains an end-to-end differentiable neural network to predict successful trajectories from failures and as a by-product learns classifiers that ground low-level states and images in mode families without dense labeling. The learned grounding classifiers can further be used to translate language plans into reactive policies in the physical domain in an interpretable manner. We show our approach improves the interpretability and reactivity of imitation learning through 2D navigation and simulated and real robot manipulation tasks. Website: https://yanweiw.github.io/glide/

Paper: Link

Abstract: Robot behavior policies trained via imitation learning are prone to failure under conditions that deviate from their training data. Thus, algorithms that monitor learned policies at test time and provide early warnings of failure are necessary to facilitate scalable deployment. We propose a runtime monitoring framework that splits the detection of failures into two complementary categories: 1) Erratic failures, which we propose to detect using a statistical measure of temporal action consistency, and 2) task progression failures, where we use VLMs to detect when the policy confidently and consistently takes actions that do not solve the task. Our approach has two key strengths. First, because learned policies exhibit diverse failure modes, combining complementary detectors leads to significantly higher accuracy at failure detection. Second, using a statistical temporal action consistency measure ensures that we quickly detect when multi- modal, generative policies exhibit erratic behavior at negligible computation cost. In contrast, we only use VLMs to detect failure modes that are less time-sensitive. We demonstrate our approach in the context of diffusion policies trained on a multi-modal domain and simulated bi-manual robotic manipulation domains. The resultant system, unifying temporal consistency detection with VLM runtime monitoring, detects 15% more failures than using either of the two detectors, thus highlighting the importance of assigning specialized detectors to complementary categories of failure.

Paper: Link

Poster: Link

Slides: Link

Video: Link

Abstract: Quadrotors are among the most agile flying robots. Despite recent advances in learning-based control and computer vision, autonomous drones still rely on explicit state estimation. On the other hand, human pilots only rely on a first-person-view video stream from the drone onboard camera to push the platform to its limits and fly robustly in unseen environments. To the best of our knowledge, we present the first vision-based quadrotor system that autonomously navigates through a sequence of gates at high speeds while directly mapping pixels to control commands. Like professional drone-racing pilots, our system does not use explicit state estimation and leverages the same control commands humans use (collective thrust and body rates). We demonstrate agile flight at speeds up to 40km/h with accelerations up to 2g. This is achieved by training vision-based policies with reinforcement learning (RL). The training is facilitated using an asymmetric actor-critic with access to privileged information. To overcome the computational complexity during image-based RL training, we use the inner edges of the gates as a sensor abstraction. This simple yet robust, task-relevant representation can be simulated during training without rendering images. During deployment, a Swin-transformer-based gate detector is used. Our approach enables autonomous agile flight with standard, off-the-shelf hardware. Although our demonstration focuses on drone racing, we believe that our method has an impact beyond drone racing and can serve as a foundation for future research into real-world applications in structured environments.

Paper: Link

Poster: Link

Video: Link

Abstract: Trajectory planning under kinodynamic constraints is fundamental for advanced robotics applications that require safe, dexterous, reactive, and rapid skills in complex environments. This paper introduces a new approach to safe reinforcement learning by combining learning-to-plan methods with reinforcement learning, resulting in a novel integration of black-box learning of motion primitives and optimization. We show that the proposed method outperforms state-of-the-art in challenging scenarios such as planning to hit in robot air hockey.

Paper: Link

Slides: Link

Video: Link

Abstract: Rapid advances in perception have enabled large pre-trained models to be used out of the box for transforming high-dimensional, noisy, and partial observations of the world into rich occupancy representations. However, the reliability of these models and consequently their safe integration onto robots remains unknown when deployed in environments unseen during training. In this work, we address this challenge by rigorously quantifying the uncertainty of pre-trained perception systems for object detection via a novel calibration technique based on conformal prediction. As a result, the calibrated perception system can be used in combination with any safe planner to provide an end-to-end statistical assurance on safety in unseen environments. We evaluate the resulting approach, Perceive with Confidence (PwC), with experiments in simulation and on hardware where a quadruped robot navigates through previously unseen indoor, static environments. These experiments validate the safety assurances for obstacle avoidance provided by PwC and demonstrate up to 40% improvements in empirical safety compared to baselines. Videos and code can be found at https://perceive-with-confidence.github.io.

Paper: Link

Poster: Link

Slides: Link 

Video: Link

Abstract: Estimating the region of attraction (RoA) for robot controllers is crucial for safe operation and integration. Many existing methods, limited by the need for closed-form expressions, do not suit data-driven controllers, while trajectory-based methods are overly data-intensive. We previously showed that Morse Graphs—directed acyclic graphs capturing non-linear dynamics—can efficiently estimate RoAs without an analytical model, though they falter with high-dimensional systems. This paper introduces Morse Graph-aided discovery of Regions of Attraction in a learned Latent Space (MORALS). Combining auto-encoding neural networks with Morse Graphs, MORALS efficiently predicts attractors and their RoAs for data-driven controllers in complex systems like a 67-dim humanoid robot and a 96-dim three-fingered manipulator. It projects the dynamics into a latent space and constructs Morse Graphs representing the bistability of the underlying dynamics to detect stable versus unstable behaviors. Evaluation shows data-efficient RoA estimation in high-dimensional robotic datasets.

Paper: Link

Abstract: The primary goal of reinforcement learning is to develop decision-making policies that prioritize optimal performance without considering risk or safety. In contrast, safe reinforcement learning aims to mitigate or avoid unsafe states. This paper presents a risk-sensitive Q-learning algorithm that leverages optimal transport theory to enhance the agent safety. By integrating optimal transport into the Q-learning framework, our approach seeks to optimize the policy’s expected return while minimizing the Wasserstein distance between the policy’s stationary distribution and a predefined risk distribution, which encapsulates safety preferences from domain experts. We validate the proposed algorithm in a Gridworld environment. The results indicate that our method significantly reduces the frequency of visits to risky states and achieves faster convergence to a stable policy compared to the traditional Q-learning algorithm.

Paper: Link

Abstract: In this work, we address extending classical Hamilton-Jacobi (HJ) reachability analysis to hybrid dynamical systems. We characterize the reachable sets for hybrid systems through a generalized value function defined over discrete and continuous states of the hybrid system. We also provide a numerical algorithm to compute this value function and obtain the reachable set.

Our framework can compute reachable sets for hybrid systems consisting of multiple discrete modes, each with its own set of nonlinear continuous dynamics, discrete transitions that can be directly commanded or forced by a discrete control input, while still accounting for control bounds and adversarial disturbances in the state evolution. Along with the reachable set, the proposed framework also provides an optimal continuous and discrete controller to ensure system safety. We demonstrate our framework in several simulation case studies, as well as on a real-world testbed.

Paper: Link

Poster: Link

Slides: Link

Video: Link

Abstract: We propose semidefinite trajectory optimization (STROM), a framework that computes fast and certifiably optimal solutions for nonconvex trajectory optimization problems defined by polynomial objectives and constraints. STROM employs sparse second-order Lasserre's hierarchy to generate semidefinite program (SDP) relaxations of trajectory optimization. Different from existing tools (e.g., YALMIP and SOSTOOLS in Matlab), STROM generates chain-like multiple-block SDPs with only positive semidefinite (PSD) variables. Moreover, STROM does so two orders of magnitude faster. Underpinning STROM is cuADMM, the first ADMM-based SDP solver implemented in CUDA and runs in GPUs. cuADMM builds upon the symmetric Gauss-Seidel ADMM algorithm and leverages GPU parallelization to speedup solving sparse linear systems and projecting onto PSD cones. In five trajectory optimization problems (inverted pendulum, cart pole, vehicle landing, flying robot, and car back-in), cuADMM computes optimal trajectories (with certified suboptimality below 1%) in minutes (when other solvers take hours or run out of memory) and seconds (when others take minutes). Further, when warmstarted by data-driven initialization in the inverted pendulum problem, cuADMM delivers real-time performance: providing certifiably optimal trajectories in 0.66 seconds despite the SDP has 49,500 variables and 47,351 constraints.

Paper: Link

Poster: Link

Video: Link

Abstract: The rapid advancement of robotics necessitates robust tools for developing and testing safe control architectures in dynamic and uncertain environments. Ensuring safety and reliability in robotics, especially in safety-critical applications, is crucial, driving substantial industrial and academic efforts. In this context, we extend CBFkit, a Python/ROS2 toolbox, which now incorporates a planner using reach-avoid specifications as a cost function. This integration with the Model Predictive Path Integral (MPPI) controllers enables the toolbox to satisfy complex tasks while ensuring formal safety guarantees under various sources of uncertainty using Control Barrier Functions (CBFs). CBFkit is optimized for speed using JAX for automatic differentiation and jaxopt for quadratic program solving. The toolbox supports various robotic applications, including autonomous navigation, human-robot interaction, and multi-robot coordination. The toolbox also offers a comprehensive library of planner, controller, sensor and estimator implementations. Through a series of examples, we demonstrate the enhanced capabilities of CBFkit in different robotic scenarios.

Paper: Link

Poster: Link 

Slides: Link

Video: Link

Abstract: This paper proposes an approach for solving a reach-avoid motion planning problem given a mobile robot with black-box dynamics. We consider the specific case where obstacles are near the robot’s goal, causing a near danger (i.e., narrow gap) scenario. We apply neural networks to learn a space of motion plans, which the robot tracks with a feedback controller. This results in a lumped type of uncertainty that we call tracking error to represent the fact that the robot cannot perfectly track any plan. We show how our recent work on Piecewise Affine Reachability Computation (PARC) combines elegantly with feedforward, rectified linear unit (ReLU)-activated neural network planners to discover safe, goal-reaching trajectories near danger. For this short paper, we demonstrate the approach on a 2-D double integrator as a preliminary example.

Paper: Link

Poster: Link

Slides: Link

Video: Link

Abstract: Quadrotor flight is an extremely challenging problem due to the limited control authority encountered at the limit of handling. Model Predictive Contouring Control (MPCC) has emerged as a promising model-based approach for time optimization problems such as drone racing. However, the standard MPCC formulation used in quadrotor racing introduces the notion of the gates directly in the cost function, creating a multi-objective optimization that continuously trades off between maximizing progress and tracking the path accurately. This paper introduces three key components that enhance the state-of-the-art MPCC approach for drone racing. First and foremost, we provide safety guarantees in the form of a track constraint and terminal set. The track constraint is designed as a spatial constraint which prevents gate collisions while allowing for time optimization only in the cost function. Second, we augment the existing first principles dynamics with a residual term that captures complex aerodynamic effects and thrust forces learned directly from real-world data. Third, we use Trust Region Bayesian Optimization (TuRBO), a state-of-the-art global Bayesian Optimization algorithm, to tune the hyperparameters of the MPCC controller given a sparse reward based on lap time minimization. The proposed approach achieves similar lap times to the best-performing RL policy and outperforms the best model-based controller while satisfying constraints. In both simulation and real world, our approach consistently prevents gate crashes with 100% success rate, while pushing the quadrotor to its physical limits reaching speeds of more than 80km/h.

Paper: Link

Abstract: Legged robots navigating cluttered environments must be jointly agile for efficient task execution and safe to avoid collisions with obstacles or humans. Existing studies either develop conservative controllers (< 1.0 m/s) to ensure safety, or focus on agility without considering potentially fatal collisions. This paper introduces Agile But Safe (ABS), a learning-based control framework that enables agile and collision-free locomotion for quadrupedal robots. ABS involves an agile policy to execute agile motor skills amidst obstacles and a recovery policy to prevent failures, collaboratively achieving high-speed and collision-free navigation. The policy switch in ABS is governed by a learned control-theoretic reach-avoid value network, which also guides the recovery policy as an objective function, thereby safeguarding the robot in a closed loop. The training process involves the learning of the agile policy, the reach-avoid value network, the recovery policy, and an exteroception representation network, all in simulation. These trained modules can be directly deployed in the real world with onboard sensing and computation, leading to high-speed and collision-free navigation in confined indoor and outdoor spaces with both static and dynamic obstacles.

Paper: Link

Poster: Link

Video: Link

Abstract: We address the problem of large-scale traffic management for autonomous vehicles. We derive a safety risk prediction framework for autonomous vehicle interactions at road intersections using Graph Neural Networks (GNN) as well as a unique “traffic signal” control plan for each intersection. The plan controls the status and duration of the traffic lights at road intersections while addressing the safety implications of the chosen plan. 

Initially, due to existing limitations on the availability of Urban Air Mobility (UAM) vehicle trajectory datasets, we develop safe traffic control algorithms for autonomous vehicle interactions based on both simulation and real-world data obtained from ground vehicles, applicable to traffic management of autonomous ground vehicles. Next, we present an approach that adapts the obtained results to scenarios involving electrical vertical takeoff and landing vehicles (eVTOLs) for aerial transportation in urban areas by extending the concept of 2 dimensional highways into a 2.5-dimensional air-highway network on which vehicles travel.

To the best of our knowledge, we propose the first framework that integrates both safety constraints as well as adaptive traffic signal control reinforcement learning based protocols for safe and efficient movement coordination on air highways.

Paper: Link

Poster: Link

Slides: Link

Video: Link

Abstract: Safe autonomous driving critically depends on how well the ego-vehicle can predict the trajectories of neighbouring vehicles. To this end, several trajectory prediction algorithms have been presented in the existing literature which outputs a multi-modal distribution of obstacle trajectories instead of a single deterministic prediction to account for the underlying uncertainty. However, existing planners cannot handle the multi-modality based on just sample-level information of the predictions. 

With this motivation, this paper proposes a trajectory optimizer that can leverage the distributional aspects of an arbitrarily complex distribution, in particular, an output distribution represented as a deep neural network, in a computationally tractable and sample-efficient manner. The core of our approach is built on embedding distribution in Reproducing Kernel Hilbert Space (RKHS) , which we leverage in two ways. First, we propose an RKHS embedding approach to select probable samples from the obstacle trajectory distribution. Second, we rephrase chance-constrained optimization as distribution matching in RKHS and propose a novel sampling-based optimizer for its solution.

Paper: Link

Poster: Link

Slides: Link

Video: Link

Abstract: With the advent of large foundation model based planning, there is a dire need to ensure their output aligns with the stakeholder’s intent. When these models are deployed in the real world, the need for alignment is magnified due to the potential cost to life and infrastructure due to unexpected faliures. Temporal Logic specifications have long provided a way to constrain system behaviors and are a natural fit for these use cases. In this work, we propose a novel approach to factor in signal temporal logic specifications while using autoregressive transformer models for trajectory planning. We also provide a trajectory dataset for pretraining and evaluating foundation models. Our proposed technique acheives 74.3 % higher specification satisfaction over the baselines.

Paper: Link