Robot competitions, including the EU FP7 funded projects RoCKin and euRathlon, and robotics challenges such as EuRoC, are becoming widely accepted as a complementary approach to advance robotics R&D. Competitions stimulate innovation in a compelling way by providing researchers and developers with realistic mock up scenarios to test and compare their robots.
At the same time benchmarking is recognised as a major challenge in robotics – without benchmarks it is impossible to properly compare robot solutions – and several efforts have attempted to define and publish benchmarks (e.g., RawSeeds, BRICS, EURON SIG and IEEE TC on Good Experimental Methodology).
This workshop includes a panel of speakers – each leading competition challenge and/or benchmarking activities – to publically explore and debate the synergies between competitions, challenges and benchmarks. Ideally, competition scenarios, challenges and benchmarks should be co-created in order to both (a) utilise and test existing benchmarks in competitions and (b) use competitions to develop and extend benchmarks.