We validated PROMISE in terms of expressiveness and usability. The expressiveness is validated through experimentation, specifying complex missions found in literature and some others that we created. The missions we specified from literature are based on the definitions of the 2018's edition of RoboCup@Home (http://www.robocupathome.org/rules/2018_rulebook.pdf). Concretely, we replicated the restaurant management scenario, the dishwasher challenge, and the tour guide. Below, we provide some videos of the experimentation, where we specify some of the missions and execute them.
We evaluated the usability of our DSL by conducting two different user studies. The first study was a preliminary evaluation that triggered important refinement of the language and the tool. For the second study, we strove to understand what elements of PROMISE could be perceived as error-prone or make the participants less confident in their solutions. We learned that most issues that participants experienced during the first study were solved for the second one. Still, we learned that the tool implementation was not as user-friendly as we intended, and therefore we are constantly improving it. For more information regarding the user studies, check our publication here.
The first experimentation step was to simulate the scenario using ROS (https://www.ros.org) and Gazebo (http://gazebosim.org).
Note that the version of PROMISE in the video was preliminary.
After a succesful simulation of the scenario, we decided to test it using our Turtlebot2 in the offices of Chalmers and University of Gothenburg.
Note that the version of PROMISE in the video is slightly outdated.
Our colleagues and friends from PAL Robotics (http://pal-robotics.com) allowed us to test the scenario using one of their TIAGo robots (https://tiago.pal-robotics.com) in their facilities.
Note that the version of PROMISE in the video is slightly outdated.
We decided to design a mission of our own. The mission encompass two robots that must collaborate to achieve a goal. This mission was used as a running example for our SLE'19 publication.