Experiments and results
In this page I propose different scenarios, the setup of various experiments, and I will show the results in terms of accuracy or demonstrative videos.
Scenarios
- Scenario A consists of random events generated by the machine on a set of photos regarding a navigation session.
- Scenario B involves the recognition of human legs among objects with similar shapes.
- Scenario C is a multi-class classification, realized with three binary SVM models.
- Scenario D is a small object recognition experiment to show the power and the limits of the detection system.
- Scenario E regards a door handle detection
Results obtained in Gazebo simulations:
Two videos as a demonstration of multi-class recognition task, i.e. Scenario C:
- The Turtlebot, in order to escape from a blind alley, decides to move the balls instead of the cube thanks to its recognition capabilities;
- In the second, in its path, the robot meets a series of obstacles, a wall and then a human being, and reacts every time in the most appropriate way possible.
Finally, in Scenario D, our purpose is the small object detection. The cascade architecture is a great help when it comes to identify small objects among all the pixels of an input image: observe how the accuracy increases from a one to a four layers cascade.
One layer cascade confusion matrix
Four layers cascade confusion matrix
Results obtained in a real environment:
The stress test that I prepared to assign a score is based in scenario B:
- five different persons
- two different heights for each person (1 meter and 50 centimeters)
- four poses for each height (1: first pose near, 5: first pose far, 2: second pose near, 6: second pose far, and so on...
To evaluate the performance of the cascade in the case of scenario E, I prepared a stress test consisting on 60 images with the following characteristics:
- five doors
- 9 shots for each door (between brackets there is the legend for reading Sheet "ScenarioE")
- three frontal with varying distances (1 for short, 2 for medium, 3 for far)
- two not centered with medium distance (4 for right, 5 for left)
- two not centered from below with short distance (6 for right, 7 for left)
- two not centered from above with short distance (8 for right, 9 for left)
- different light conditions
- 15 images of walls that do not contain any handle
Scenario B: pose 1-5
pose 2-6
pose 3-7
pose 4-8
The objective of Scenario D is to detect a small ball:
Integration with Learning by Demonstration Module
After the separated development of the situation detection model in this thesis and the learning-by-demonstration framework in an external thesis, it is time to join the two parts together. Thanks to the overall system, a robot is able to apprehend when and how take a decision that was not predicted in the original plan. This decision, which depends on the information acquired through the camera, enables an opportunistic behavior that may consist in a maneuver, a talking option or other case-specific actions.
The architecture, after the training phase, of the overall system used in the following experiments works in this way: the classifiers recognize different situations, then they communicate their decision to the PNP (Petri Net Plan) framework through a specific ROS topic. Depending on the message received, PNP enables one of the policy generator and the robot undertake the correspondent action.