Conclusion
Conclusion
The first source of difficulty had to with the vision:
We first tried to just use a neural network on a slanted image of the board but that wasn't accurate enough so we opted to try and flatten the board through homography. However, to use homography, we needed to fnd the corners of the board which was its own challenge so we used blue tape to detect the corners of the board. We also had to include a bit of calibration in which we set a constant cropping to the image to remove as much noise as possible before detecting the corners. After that, it was merely a challenge of figuring out what colored balls were in what spots which required much training of the neural network.
The second source of difficulty had to do with the game and the AI:
It was a major time sink to design the game from scratch and there were many debates on certain design implementations. The code for the stronger AI was built early on while changes were still being made to the game design so the integration of the stronger AI into the system took longer than expected.
The third source of difficulty was the actuation:
The way we calculated the position of the points required a lot of calibration and would be prone to error if the table or board was ever moved or rotated. Additionally, we had to deal with the issue of the accuracy of the robot arm and tried to eliminate as many sources of error as possible. Despite that, due to the small size of the balls as well as the inaccuracy of the position of the ar-tag, there would be times where our actuation procedure would fail. We also had to do a careful balancing act of keeping the board close enough to the camera to be seen but not too far away from the arm that certain pieces couldn't be reached.
The fourth source of difficulty was the grippers:
Designing the grippers took a long time and required many reprints due to the very strict requirements of our project. We also broke a few grippers along the way during height calibrations.
The final source of difficulty was the full stack integration.
The integration took a while due to defining message types and building the packages. There were issues with package imports and figuring the startup procedure for everything to run smoothly.
There are many shortcomings to this project and many improvements we would have liked to make given more time. Some of the things we would work on are:
Improving the vision to make it more robust to lightning and position changes
Having the main game loop detect a human moving a piece on the board and responding accordingly
Integrating the stronger AI into the main game loop
Increasing the accuracy of locating the position of the pegs and making it more robust by making it less sensitive to orientation and position changes
Improving the path planning and actuation to make the whole motion of picking and placing a piece more smoother and quicker