MyoSymposium @Neurips'25

Summary: 

Achieving human-level speed and performance in robotics remains a grand challenge. In our lab at Google DeepMind, we've developed a robot agent capable of playing competitive table tennis against human opponents.

While our initial systems mastered cooperative rallies, the transition to competitive, adversarial play required a fundamental shift in our approach. In this talk, I will briefly overview the project's journey but will focus specifically on the key innovations and lessons learned that enabled this leap from cooperation to competition.

I will detail the main changes to our deep reinforcement learning policy architecture, sim2real pipeline, and real-time adaptation recipe that were necessary to win points. These changes allowed our agent to win 45% of matches against a range of human players. I will conclude by summarizing the core takeaways for designing athletic intelligent systems and for the future of dexterous, real-time robotics at large.

Bio:

Pannag Sanketi (Linked-In, X) is a Senior Staff Engineer and Tech-Lead Manager on the Robotics team at Google DeepMind in Mountain View, California. His research focuses on the intersection of robotics, optimal control, and deep learning. Over the last seven years, he has led high-impact projects in robot learning, including X-Embodied foundation models (Open XE / RT-X) and agile robots capable of playing table tennis and catching objects.

His research, published at top conferences like CoRL, ICRA, NeurIps, RSS, IROS, L4DC. His Open XE / RT-X work received the Best Conference Paper award at ICRA 2024, the Robo-DM paper received the Best Robot Learning Paper award at ICRA 2025 and the table tennis project was a finalist for the Best Robot Learning Paper award at ICRA 2025. His work has been featured in major media outlets, including NBC, CBS, Wired, TechCrunch, and IEEE Spectrum.

Prior to his focus on robotics, Pannag was a Tech-Lead in the Android team for over seven years. There, he developed AI features for core products (Camera, Photos, Pixel Watch) and founded MLKit, one of Google's largest on-device machine learning frameworks. His earlier career includes roles at sports-tech startups (SportVision, PixBlitz) and as a post-doctoral associate at the Smith-Kettlewell Eye Research Institute, where he developed computer vision algorithms to aid the visually impaired.

Pannag holds a PhD (2009) and MS (2005) in Nonlinear Control Systems from UC Berkeley, and a B.Tech in Mechanical Engineering from IIT-Madras (2003).

Summary:

Understanding and controlling human movement requires a self model that captures how the body perceives and acts within the world. This talk presents a whole-body sensory-musculoskeletal model that integrates anatomically detailed musculoskeletal dynamics with multimodal sensory feedback. We develop deep reinforcement learning and model predictive control methods that enable coordinated, whole-body behaviors. The system reproduces natural human movements across locomotion and manipulation tasks, while uncovering internal musculoskeletal and sensory dynamics that are difficult to measure experimentally. This work provides a computational framework for exploring human sensorimotor intelligence and developing embodied agents capable of self-consistent interaction with the world.

Bio:

Yanan Sui (YananSui.com) is an Associate Professor at Tsinghua University. His research focuses on neuro-musculo-skeletal modeling and control, with applications in robotics and brain-machine interaction. He received degrees from Tsinghua and Caltech, and conducted postdoctoral research at Caltech and Stanford. His work on safe optimization has been featured in textbooks at Stanford and other universities, and his research on human preference optimization led to a Best Conference Paper Award and a Best Paper Award on Human-Robot Interaction at the International Conference on Robotics and Automation (ICRA). His methods have been successfully applied to clinical treatment of neural injuries. He has served as a committee member and senior area chair for leading AI conferences. For his contributions at the intersection of AI, robotics, and neural engineering, he was named one of MIT Technology Review’s Innovators Under 35 in China.

Summary:

Soccer presents a long-standing challenge for robotics, requiring the orchestration of perception, planning, and control across scales ranging from milliseconds to minutes. In this talk, we review several years of progress at DeepMind in tackling this open-ended problem, beginning with combining low-level motion priors with strategic game play, through tackling robust sim-to-real transfer, and finally integrating these complex behaviors into single policies that display both agility and strategy, purely from egocentric vision on real world compute constrained hardware.

Bio:

Ben Moran is a Research Engineer at Google DeepMind in London, working on robotics with Nicolas Heess, following a career in automated trading.  Ben is also a PhD candidate at UCL, supervised by Zhibin Li.  His research interests cover robot agility, 3D and real-to-sim for robotics, and his work has been published in Science Robotics, ICRA and ICML and at the RoboCup Symposium.  His team's project on humanoid robot soccer at DeepMind garnered considerable attention, with coverage from 45 media outlets including the BBC and CBS "60 Minutes".