Fortunately, the project was able to execute all of the tasks in our initial goals with pinpoint precision. Our robot was able to capture the input of the camera, locate all of the tiles on the board, locate all of the pieces on the board, plan its next move, reach down and grab the piece to relocate, and finally place the piece in a new tile.
After the robot's turn is complete, we are then able to move a piece on our side, and then resume the robot's operation. Doing so simulates a real game of chess against the Sawyer robot, as the robot takes in its next camera input and repeats the process mentioned above to make its move.
Due to our "read from the camera input only at the start of your turn" structure, the robot does not rely on previous board states to make its decisions. This means that the robot can play from any board state, and consequently be able to solve chess puzzles (as chess puzzles can start from any board state rather than the initial one)!
Although the video demonstrates a single move from the Sawyer arm, we hope that our showcase demo and our code proves that an entire game can played with our implementation.