In order to find a shape that was consistently on the whiteboard rather than outside it, we needed to first define the boundaries of the whiteboard. We hard coded a range for the contour area of the whiteboard and while this limits the amount of places the whiteboard can be placed, we decided that due to time constraints hard coding the area and only using a specific table was a worthwhile substitute. When we did need to use a different table, there were a couple lines of code that gave the new area for the whiteboard which only took a couple minutes to run the code and change values to once again consistently find the whiteboard.
Now that our image processing had identified both the board and given us the pixel coordinates of the four corners of the board, Finding the shape was fairly straightforward. We choose to take the largest contour the algorithm could find that was within the boundaries of the board and we found that to be incredibly reliable. The only problems with that was that there were times where a light would shine a square shape onto the board which would be bigger than the shape we drew. We could have done additional image processing to reduce the light thresh hold the whiteboard to more robustly identify the shape but that is outside the scope of the course and we found that drawing our shapes larger or simply blocking the majority of the light would allow the algorithm to confidently find the shape.
Taking the points from the image processing, we transform them into points in the base frame and pass them into moveit. To better fill in the shape, we divided the points into two sets and had the robot draw a line between the two sets resulting in a filled shape. The points were divided based on their position relative to a calculated line of symmetry. We restricted the orientation of the last joint to be strictly facing down, improving both the accuracy of locating points as well as drawing the line itself. Theoretically, the motion planning should pick a predictable and safe path from one point to another as the points were not too spread out and while most of the time it did this, there were times the motion planning would not be wholly accurate. We could have gone around this by writing our own controller and motion planner but due to time constraints, we were unable to.