Results and Conclusion

Results

We were able to consistently reach our goal of playing a game of tag with two turtlebots. Take a look!

Analysis and Extension

We were able to accomplish our initial goal of having a game initiated by a human user through a visual signal and then having a game of tag with a pursuer and evader run to completion. This was our primary objective and the baseline we hoped to hit before the project deadline.

That said, we would have liked to experiment further with evasion algorithms, particularly when it comes to map boundaries, additional bots, and more complex behavior. With more time, we may have been able to research and implement a strategy that would allow the pursuer to pick out one evader that would then pursue a strategy that prevented it from going out-of-bounds, perhaps opting to confuse the pursuer into following a different bot instead. This would be a sort of "National Geographic" approach, since it mimics a hunted animal running back towards its herd to confuse an attacker. Our current "rotate-and-mimic" was a naive implementation that relied only on the position of the pursuer relative to the evader. This approach had the limitation that the evader was always moving even if it was close to a boundary.

We had an early implementation of boundary detection that did not make it to the demonstration. Our algorithm would print "Game Over" when an evader bot went out-of-bounds, where the boundary was the square marked by AR tags on the upper left and lower right of the grid. We would detect the bot's distance from the boundary using the norm of the translation vector of the twist generated by tf.lookup_transform and determining whether this distance was within a particular out-of-bounds threshold. However, this approach failed to meet the eye test -- although many times the game would stop due to the bot supposedly being outside the boundary according to the distance measurement, it would actually be well within bounds in the actual game. As a result of this issue, we also struggled with implementing the at-boundary evasion techniques we had in mind. Overall our thresholding approaches (used for boundary detection, tag detection, boundary evasion) were the least reliable components of our project implementation. With a little more time, research, and guidance, we think we would have hit this goal and gotten boundary evasion to work.

Similarly, a challenge we hoped to address in the allotted time was having obstacles within the game space, thereby limiting the navigable region available for attacker and pursuer. We were planning on marking obstacles as items with known AR markers, and having the evaders move around these while dodging the pursuer, while the pursuer would treat these blocks as boundaries to pin the evaders against and win. Based on some of the thresholding challenges described above, adding this component would have been possible if we had a robust boundary evasion strategy, since the evader could probably integrate some of that logic when getting around obstacles of certain sizes and the game would be able to recognize when the obstacle had been hit, thereby ending the game.

We were limited in terms of the space available to the bot, since we'd need to be within the shared lab area to run our code. Additionally, the tradeoff between camera placement and AR tracking effectiveness also impacted our results. If the camera was placed too far away it would allow a greater boundary but not be able to recognize tags far away. With a better wide-range camera, we would have been able to hit our goal here.

Lighting played a factor as well -- the stick detection and selection algorithms did slightly worse in the room with Sawyers and Baxters where the lighting was not as bright or direct. On top of this, due to some of the thresholding used to distinguish the stick from green background noise, if the original green stick was too far from the camera or at the wrong angle it would not be detected. To get around this issue, we ended up using a much larger green signal that the camera could see from afar. We pointed this stick directly at the bot we wanted to choose, meaning that the vector generated had to be dead-on. With more time, we would have experimented with Kalman filtering and edge detection to make the stick classifier more robust.

Conclusion

Despite all the challenges we faced, we had a fantastic time working on this project and getting all the different components to work together. The complexity of the task expanded upon a variety of the core fundamental topics discussed in the class, such as robotic manipulation, transform-based mathematics, and ROS fundamentals. Being able to use and build upon the skills we had learned was extremely rewarding. Additionally, since none of us had substantial CV experience, learning how to apply image filtering, PCA, and image transformation to be able to detect our green marker and where it was pointing was quite fulfilling as well.

We'd like to thank Professor Sastry for the opportunity to present our work in front of our peers! We'd also like to thank the TAs for their time, energy, and omniscience when answering our lab questions and helping us make progress!