Vision-based Agile Robotics

Autonomous drones play a crucial role in inspection, agriculture, logistics, and search-and-rescue missions and promise to increase productivity by a factor of 10. However, they still lag behind human pilots in speed, versatility, and robustness. What does it take to fly autonomous drones as agile as or even better than human pilots? Autonomous, agile navigation through unknown, GPS-denied environments poses several challenges for robotics research regarding perception, learning, planning, and control. In this talk, I will show how the combination of model-based and machine-learning methods, united with the power of new, low-latency sensors, such as event cameras, can allow drones to achieve unprecedented speed and robustness by relying solely on onboard computing. This can result in better productivity and safety of future autonomous aircraft. 

Big Robot Models that Train Fast and Run Fast

Vision-language-action models (VLAs) have become a dominant paradigm for training scalable robot foundation models. But the first generation of VLAs (RT-2, OpenVLA, …) came with severe limitations: they were slow to train, even slower to inference, and struggled to solve dexterous manipulation tasks beyond simple pick and place. In this talk I will summarize our work on a new generation of VLA models that train quicker, inference fast, and are able to solve complex manipulation tasks like folding a basket of laundry or cleaning up unseen kitchens. The key: advances in how we can effectively integrate robot actions into multi-modal LLMs.

Cooperative Exploration in Constrained Environments.

This talk will discuss the multi-agent autonomy architecture of NASA's Cooperative Autonomous Distributed Robotic Explorers (CADRE) mission, a technology demonstration that will deliver a team of autonomous rovers to the Moon's Reiner Gamma region in the coming year.

Multi-robot systems hold great promise to address key questions in planetary science. They can observe phenomena of interest from multiple locations at the same time; produced detailed three-dimensional images of the subsurface through seismic and radar surveys; and offer increased resilience compared to monolithic explorers, enabling bolder exploration. Autonomy is an enabling technology for multi-robot systems: it allows robots to work together as a team, building on each other's abilities, with no humans in the loop, a critical capability when light-speed delays and low bandwidth make teleoperation infeasible. But how do we design, build, test, and fly algorithms for a team of autonomous robots? How should the robots decide who (if any) should be in charge; when they should drive, and when they should recharge; how to explore an unknown region together; and how to collect measurements in formation? And, critically, how do the robots do this with a cellphone-grade CPU, stringent power and thermal constraints, limited and uncertain inter-agent bandwidth, and even more limited bandwidth to operators on Earth?

In this talk, we will walk through CADRE's autonomy stack, explore the resource trade-offs that informed the design of its multi-agent autonomy architecture, and end by speculating about the future of multi-robot systems for planetary exploration.

Towards Resource-Constrained Robot Mapping and Motion learning using Physics Priors

This talk will outline the use of physics priors towards creating efficient, resource constrained robot mapping and motion learning methods. These methods require minimal to no expert data and achieve high efficiency in both training and inference while effectively operating in complex, high-dimensional environments under various constraints. Recent advancements in robot motion learning include methods based on imitation and offline reinforcement learning, which are known to necessitate a significant amount of expert trajectories and entail high training times. In contrast, this presentation will introduce a new class of self-supervised, physics-informed neural motion policy learners. These methods aim to directly solve Partial Differential Equations (PDEs) that govern robot motion without depending on expert data or requiring extensive training resources. Additionally, the talk will discuss how these PDEs can create a new robot-motion-friendly and compact mapping feature. We demonstrate that this new mapping feature is better suited for fast robot motion generation than existing mapping features, such as occupancy maps or Sign Distance Fields. This talk will demonstrate that these new physics-informed approaches outperform state-of-the-art imitation learning and offline reinforcement learning methods in terms of scalability, training efficiency, data efficiency, computational planning speed, path quality, and success rates.

Talk Details Pending

Talk Details Pending

Beyond the demo: ensuring robotic reliability with onboard compute constraints.

Drones have evolved from toys, to tools, to now serving as critical infrastructure in industries like first response and inspection. Positioned on rooftops in automated docks for immediate deployment, their reliability is crucial. Skydio has conducted well over 2 million customer flights, is set to do 1 million this year alone, and has become an indispensable part of everyday operations.

This talk leverages Skydio's experience to discuss what it takes to deploy autonomous drones at scale, while operating within onboard compute constraints. Using the example of the fully autonomous dock landing subsystem essential for remote operations, this talk will highlight practical principles and necessary trade-offs essential for reliable performance in these demanding and safety-critical environments.