Robot competitions/challenges aim to create an opportunity to bring together the most advanced robot technologies globally and to challenge the human limits.
By solving actual problems in the society, the robot competitions/challenges deepen people’s understanding of robots and induce positive discussions that would lead to concrete uses and applications of robots. In this workshop, we introduce several robot competitions/challenges all around the world such as euRathlon, EuRoC, RoCKIn, DRC, RoboCup, ARGOS, etc. and exchange information of these events for future organizers of robot competitions/challenges.

What's New
  • The WS will be held on 10th October.
  • ROOM : Grand Ballroom

Main Organizer
Dr.Tomomasa Sato, University of Tokyo and NEDO (New Energy and Industrial Technology Development Organization, JAPAN)
Prof. Bruno Siciliano, University of Naples Federico II
Prof. Henrik I. Christensen, Georgia Institute of Technology
Prof. Minoru Asada, Osaka University
Elena Messina, U.S. National Institute of Standards & Technology
Prof. Hiroyuki Okada, Tamagawa University
Dr. Kazuhito Yokoi, AIST
Prof. Satoshi Tadokoro, Tohoku University

Topics of interest
  • Robot Competition 
  • Robot Challenge 
  • Performance Evaluation & Benchmarking of Robot Systems

Intended audience
Because in this workshop, almost all major robot competitions and challenges in the world will be presented, all researchers and students, who wants to join some of robotics challenges and competitions or create new robotics challenges or competitions, should have a strong interest in this workshop.

Invited Speakers
  1. Dr. Gill Prat, Toyota Motors
  2. Prof. Bruno Siciliano, University of Naples Federico II
  3. Prof. Pedro U. Lima, Instituto Superiot Tecnico
  4. Dr. Tomomasa Sato, U. Tokyo and NEDO
  5. Dr. Giuseppe Loianno ,U. Penn
  6. Prof. Henrik I. Christensen, UC San Diego
  7. Dr. Amit Kumar Pandey, Softbank Robotics
  8. Dr. Marco Hutter,ETH  Zürich
  9. Andrew Tinka, Amazon
  10. Dr. Itsuki Noda, National Institute of Advanced Industrial Science and Technology (AIST)
  11. Adam Jacoff(NIST)  and Elena Messina, NIST


October 10th, 2016
ROOM: Grand 
 Time    Talk
  9:00-9:10 Opening
  9:10-10:00 Special Talk 1: DARPA Challenges: How challenge is useful for innovation by Gill Pratt (Toyota Research Institute)
10:00-10:30 Coffee Break
10:30-11:00 Talk 2:European Robotics Challenges (EuRoC) by Bruno Siciliano (University of Naples Federico II)
Talk 3: The European Robotics League by Pedro U. Lima (Universidade de Lisboa) and Alan Winfield (University of the West of England, Bristol)
11:30-12:00 Talk 4: World Robot Summit in 2020 Hosted by Japan ~World Robot Challenge & World Robot Expo~ by Tomomasa Sato (U. Tokyo and NEDO)
12:00-13:00 Lunch Break
13:00-13:30 Talk 5: MBZIRC competition: Challenges, Results and Social Impacts of the 2017
competition by Giuseppe Loianno (U. Penn)
13:30-14:00 Talk 6:US FIRST Competition by Henrik I Christensen (UC San Diego)
14:00-14:30 Talk 7:Challenges for next generation of HRI and Socially Intelligent Robot competitions? Is Human-Robot Competition an Answer? by Amit Kumar Pandey (Softbank Robotics)
14:30-15:00 Talk 8: ARGOS Challenge – Legged Robots for Inspection of Industrial Sites by Marco Hutter (ETH  Zürich)
15:00-15:30 Talk 9: Amazon Picking Challenge by Andrew Tinka (Amazon)
15:30-16:00 Coffee Break
16:00-16:30 Talk 10:RoboCup: a grand challenge toward 2050 by Itsuki Noda (RoboCup Federation, and Artificial Intelligence Research Center, AIST)
16:30-17:00 Talk 11: An Ecosystem Approach to Performance Evaluations And Benchmarks by Adam Jacoff(NIST) and Elena Messina (U. S. National Institute of Standards & Technology)
17:00-17:50 Panel Discussion by All speakers
17:50-18:00 Closing


Special Talk 1: DARPA Challenges: How challenge is useful for innovation
Gill Pratt (Toyota Research Institute)
The DARPA Challenges, including the Grand and Urban Challenges, and the Robotics Challenge, have catalyzed the development of their fields. Despite the seminal nature of these challenges, public perception of what the challenges tested and what they actually tested have been somewhat different. This talk will review these differences, and other lessons from the DARPA challenges and suggest some lessons for the future.

Talk 2:
European Robotics Challenges (EuRoC)
Bruno Siciliano (University of Naples Federico II)
Exploiting synergies across robotics and manufacturing stakeholders will speed up the process of bringing innovative technologies from research labs to industrial end-users. As an enabler in this context, the EuRoC project has launched three industry-relevant challenges:
* Reconfigurable Interactive Manufacturing Cell
* Shop Floor Logistics and Manipulation
* Plant Servicing and Inspection
It aims at sharpening the focus of European manufacturing through a number of application experiments, while adopting an innovative approach which ensures comparative performance evaluation. Each challenge was launched via an open call and is structured in 3 stages. 45 Contestants were selected using a challenge in a simulation environment. Matching up the best Contestants with industrial end users, 15 Challenger teams were admitted to the second stage, where the typical team is formed by research experts, technology suppliers, system integrators, plus end users. Teams were required to benchmark use cases on standard robotic platforms empowered by the project consortium. After a mid-term evaluation, the teams have recently been admitted to showcasing the use case in a realistic environment. After an open judging process, 6 Challenge Finalists will be selected to run pilot experiments in a real environment at end-user sites to determine the final EuRoC Winner.

Talk 3:
The European Robotics League
Pedro U. Lima (Universidade de Lisboa) and Alan Winfield (University of the West of England, Bristol)
The European Robotics League (ERL) is an innovative concept for robot competitions. The ERL is composed of multiple Local Tournaments, held in different research labs across Europe, with certified test beds, and a few competitions as part of Major Tournaments, such as RoboCup. Teams participate in a minimum of 2 tournaments (Local and/or Major) per year and get scores based on their performances. A final end of year score is computed for each team, per Task Benchmark (TBM) and Functionality Benchmark (FBM), using the best two participation in tournaments, and teams are ranked based on their final score.
Currently, the ERL consists of three parallel challenges, that stem from predecessor competitions: ERL-Service Robots (RoCKIn@Home, inspired by RoboCup@Home), which focuses on the domain of service robotics for home application; ERL-Industrial Robots (RoCKIn@Work, inspired by RoboCup@Work), which focuses on the domain of industrial robotics, dealing with modern automation issues; and ERL-Emergency Robots (euRathlon), which is a civilian outdoor robotics competition, with a focus on realistic, multi-domain (land, underwater and aerial robots) emergency response scenarios.
In this talk I will cover the main outcomes of the ERL predecessor EU projects (Rockin and euRathlon) and their deplyment in the this new European endeavour.

World Robot Summit in 2020 Hosted by Japan - World Robot Challenge & World Robot Expo -
Tomomasa Sato (U. Tokyo and NEDO)
The presentation explains the "World Robot Summit"(WRS) that will be hosted by Japan in 2020. The WRS consists of robot competitions called the "World Robot Challenge" and exhibition of the latest robot technologies called the "World Robot Expo.” The presentation starts by explaining the "Declaration of Robot Revolution" that was announced by Prime Minister Shinzo Abe in 2014 and describes the successive activities by the Japanese Government which lead to the WRS. Then, the presentation touches upon the three fields (industrial robotics, service robotics, and disaster robotics) that will be encompassed in the World Robot Challenge and examples of competitions tasks. The presentation also explains the "Robot Expo" that will take place at the same time. The WRS aims to establish a long-lasting initiative that will be continued after 2020 and to accelerate the social implementation of robots around the world.

Talk 5:
MBZIRC competition: Challenges, Results and Social Impacts of the 2017 competition
Giuseppe Loianno (U. Penn)
The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) is taking place in Abu Dhabi and it aims to provide a technologically demanding set of challenges to the robotics community. It is composed of3 Challenges.
Challenges 1 and 3 relate to Unmanned Aerial Vehicles (UAVS) operations. They require a UAV to locate, track and land on top
of moving vehicle and a team of UAVs to collaborate in searching, locating, tracking, picking and placing a set of static and moving objects. Challenge 2 involves a manipulation task using a ground robot.
This talk will provide an overview of the requirements and challenges for this competition. In addition, I will focus on the benefits of international cooperation, resource sharing and competition between and within teams showing how these aspects clearly impact students and researchers growth. Finally, I will also present the strategies and results for the competition obtained by the University of Pennsylvania (UPenn), Czech Technical University (CTU) and University of Lincoln (UL) teams showing how UAVs are able to solve problems of autonomous navigation, dynamic interactions and aerial manipulations. These results demonstrate a level of autonomy not possible just a few years ago, with potential to benefit society.

Talk 6:
US FIRST Competition
Henrik I Christensen (UC San Diego)
The US FIRST competition was launched to inspire students to engage in Science, Technology, Engineering and Math (STEM) as part of their education. The world has a serious deficit of people educated on those topics. The FIRST competition was initially only for high school students. Today there are 4 different leagues from pre-school to high school. The competition by now involves more than 3,000 teams across 24 countries. The contest engages 75,000 students and more than 15,000 mentors. The contest was launched 1989 by Dean Kamen and Woodie Flowers, MIT. The teams are given 6 weeks to build a robot that solves a real world task such as playing basketball or conquering a castle. The teams have to use a standard set of components and there are strong safety, size and weight. Building a robot from scratch in 6 weeks is a major challenge. The robot has to have an option for both autonomous and tele-operated operation. The team has to provide the full logistics for participation from fund raising, over design to construction and deployments including logistics, promotion and cheering. It is almost like running a small company for the students. We will provide an overview of the FIRST competition and the key aspects of how it helps promote robotics and STEM based education

Talk 7:
Challenges for next generation of HRI and Socially Intelligent Robot competitions? Is Human-Robot Competition an Answer?
Amit Kumar Pandey (Softbank Robotics)
Presence of Robots in our day-to-day live and places is now a reality. Such robots are expected to behave in socially expected and accepted manner. This requires a kind of social intelligence in the robot. But, social intelligence is achieved through a set of various basic blocks of reasoning modules, including reasoning about other agents and the environment, Visuo-spatial Perspective Taking, Theory of Mind, Affordances, etc. The immediate concern is how a robot competition can take those into account. And the bigger concern of the future will be the need for benchmarking scenarios to assess the resultant intelligence, when all such reasoning blocks should be functioning together. There is a need of considering the human in the competition scenarios. This talk will present some real use cases for socially intelligent robots, outline some real end user need and feedback, present some of the basic building blocks of social intelligence of the robot, highlight challenges associated with their evaluation and present some of the potential solutions. It will conclude by presenting the idea of why not to put a human to compete with the robot, and evaluate the intelligence.

Talk 8:
ARGOS Challenge – Legged Robots for Inspection of Industrial Sites
Marco Hutter (ETH Zürich)
The ARGOS challenge is a competition initiated by Total SA targeting the development of robotic solutions for autonomous inspection of off-shore oil and gas sites. The participating robots need to be able to get around a typical industrial site with steps, stairs, gaps and different obstacles. Starting from a mission description provided by the organizers shortly before an actual run, a complete inspection mission must be executed whereby the robot finds and visually inspects elements like pressure meters, water level meters, valve handles, boiler temperature, gas leaks, etc. During the mission, different hazards can occur like alarms or obstacles, whereby the robot needs to execute an appropriate (predefined) action. Four international teams with tracked, flipper-based vehicles and one delegation using a legged robot are competing in this 2.5 years long project with two pre-competitions and one final completion in March 2017.
In my talk I will give an insight into the challenges of the ARGOS competition and the approaches taken by our team LIO (Legged Inspection of Oil industry). I will present strategies and tools for hard and software and talk about the benefits and drawback of such competition for a research lab.

Talk 9:
The Amazon Picking Challenge
Andrew Tinka (Amazon)
Every day, Amazon is able to quickly pick, pack and ship millions of items to customers from a network of fulfillment centers all over the globe. This wouldn't be possible without leveraging cutting-edge advances in technology. Commercially viable automated picking in unstructured environments still remains a difficult challenge. Amazon Robotics sponsors the Amazon Picking Challenge in order to spur the advancement of this technology and strengthen the ties between the industrial and academic robotic communities. This talk will discuss results from the first two challenges and some lessons learned in how to organize a robotic competition.

Talk 10:
RoboCup: a grand challenge toward 2050
Itsuki Noda (RoboCup Federation, and Artificial Intelligence Research Center, AIST)
RoboCup was proposed as a grand challgen of researches on robotics and artificial intelligence. The first RoboCup competition was held in 1997, in which year Deep Blue won against human chess champion, Garry Kasparov. RoboCup chose soccer as the next target of artificial intelligence after chess because soccer contains several new open research issues. As a grand challenge, RoboCup will provide several novel technology by attaching the issues. Actually, current RoboCup also include competitions of rescue, service and industry robots in addition to soccer robots. RoboCup also includes Junior event for education of children in order for sustainable research activities.

Talk 11:
An Ecosystem Approach to Performance Evaluations And Benchmarks
Adam Jacoff(NIST)  and Elena Messina (U. S. National Institute of Standards & Technology)
Competitions are a useful tool for measuring robotic system and component performance. Are there certain practices and approaches that may have greater impact? I will discuss some lessons-learned from competitions that may prove useful in selection of metrics and design of tests and benchmarks. Among those to be discussed are two fairly different competitions: RoboCup Rescue Robot and the ICRA Perception Challenge. In certain cases, conceiving of a competition within a greater ecosystem of innovation can yield greater advancements.