Benchmarking Autonomous Service Robotics in Real Life

About This Workshop

Autonomous service robots are increasingly prevalent in real-world settings, assuming various roles to supplement human capabilities. Mobile platforms such as Walker, H1, Cobiot, TIAGo, and Figure-01 have captured the interest of non-expert users, eager to integrate robots into everyday environments. With this growing presence, researchers must consider various aspects of integrating robots within inhabited spaces, including factors such as verbal and nonverbal communication, social behavior and navigation, or robot morphologies. The expansion of research in this domain poses the challenge of developing new benchmarks to assess performance realistically beyond the confines of research laboratories. This workshop aims to foster dialogue surrounding the future of research and its benchmarking in autonomous service robotics. Using keynotes, participants, and organizers’ experiences of real-world robotic competitions, we plan to steer the conversation to address core components of recent advancements and put forth a set of voluntary draft guidelines for autonomous service roboticists concerning test procedures and benchmarking. The sessions' results will be synthesized into a set of draft frameworks for setting realistic testing scenarios and utilizing them in testing and benchmarking. This workshop seeks to enhance the reporting standards within real-world autonomous robotics research and propel the field toward more structured research practices.