Reconnaissance with Turtlebot

• The essence of this tricky problem is the difference in the fields of view of the two primary sensors in the robot - LIDAR and the Camera. The default exploration of the robot, ensures the environment is explored completely w.r.t the LIDAR, but the camera almost certainly wouldn't have seen everything LIDAR has seen.

• Our approach, ingeniously, is to explore the environment repeatedly without losing the accuracy in localization. We achieve this by exploring the environment with an auxiliary map, which is is generated by only those LIDAR points that were seen by the camera as well.

• How do we find which points in the LIDAR were seen by the camera? That is found by transforming the Lidar points to the Camera frame with the help of autonomous robust Extrinsic Calibration.

With this approach we managed to achieve immense performance improvements in terms of the number of tags that the robot saw. Initially, without a modification for the exploration, the robot only managed to observe 9/15 AprilTags, whereas with the modification, Leonardo managed to find 13/15 AprilTags and also estimate it's pose pretty accurately.