Implementation

Hardware

Custom Grippers

Iteration I:

We designed this version of the gripper early as it was quick to print and seemed like it would accomplish our goals of gripping the pieces securely. It had a few major design flaws. The first was that there was no easy orientation to print the gripper in that would be strong and still have the gripping contact we wanted. This resulted in several weak spots falling on layer lines such as the end effector attachment point and the tuning fork-like prongs. These were easy to snap and although the end effector itself was effective, the overall design was too fragile and difficult to use for us to continue with the project.

Iteration II:

This was our second design and ultimately successful design. We took the STL of the gripper provided to us and manually converted it into a solid body based design as none of us pay for the more expensive automatic tools. This then allowed us to modify the default gripper adjusting its end effector to use a similar design to Iteration I. We adjusted the attachment to eliminate the weak points of the previous iteration but overall kept the same design as it proved effective at gripping the chess pieces. Another advantage of this design is it allowed us to keep the same gripper tip transform from the original grippers as we designed these such that their tip perfectly matched the original grippers, eliminating a potential step during later inverse kinematics testing.

Parts Used

What parts did you use to build your solution?

Software

If you would like to follow along with the codebase open, please refer to Additional Materials / GitHub Link.

Processing Camera Input (chess_tracking/services.py)

The USB camera constantly publishes to a topic called /camera_image_raw
A subscriber is created to take in the value from /camera_image_raw at every moment in time to update a msg variable
A service, when prompted, returns the msg variable (which is basically the value of /camera_image_raw at the time of prompt)

Processing the Board (full_stack/board_lib.py)

The service is now prompted, and the current value of /camera_image_raw is stored in the msg variable
Using the cv2 library, the msg is now converted into an Image class variable, called image
Corner Detection
- Using cv2.findChessBoardCorners, all the inner corners of the board are detected.
- The outer corners are then computed using the average distances between the inner corners
Piece Detection
- The image is converted to black and white and then a gaussian blur is applied for improved matching accuracy.
- Next it uses rotated versions of the template images (every 2 degrees from -16 to 16) to perform a best choice ranking piece match.
- This spawns multithreaded workers to compute each of these rotations using cv2.matchTemplate.
- We then save the best angle match and only save worse matches if they are beyond a distance threshold away from any better matches.
  - This is done to prevent matching the same piece multiple times.
- Doing this for each template type and then performing the same best match filter gets us our final piece mapping.
With the board corner coordinates and piece coordinates, we create a piece -> tile mapping and subsequently the FEN string.

Action Manager (full_stack/integration.py)

Using the piece -> tile mapping, generate an FEN string
Pass the FEN string into the Stockfish API through a POST request
Once request resolves, grab the response and extract tile to pick up piece, as well as tile to place piece
Prompt Sawyer robot to tuck, hover over a tile, go down and grab/release the piece on the appropriate tiles

Camera Transform

Initial Setup:
- Our C920 is mounted rigidly to the robot's gripper.
- An AR tag is placed on the table in view of the webcam.
Recording Transformations:
- Step 1: Record the transform from the robot's base to the gripper's base (T_base_to_gripper).
- Step 2: Record the transform from the AR tag to the webcam (T_AR_to_webcam).
Robot Alignment:
- Position the robot so the wrist camera can detect the AR tag.
Recalculate Gripper Transform:
- After alignment, record the transform from the robot's base to the gripper's base again (T_base_to_gripper).
Compute Transformations:
- Use the series of recorded transforms and the known rigid attachment of the webcam to the gripper to compute the transform from the robot’s base to the webcam (T_base_to_webcam).
Coordinate Mapping:
- Convert all pixel coordinates detected by the webcam into real-world coordinates relative to the webcam.
- Transform these coordinates into the robot base frame using T_base_to_webcam.
Robot Positioning:
- Use the transformed real-world coordinates to guide the robot to the desired location on the chessboard.

Full Stack

Here is a simple demonstration of the full game logic with examples on the side.

While game is playing:

Tuck and take a picture of the board
Get piece coordinates from picture (ex. Q at 124, 372)
Get board coordinates (ex. e7 lower left corner 100, 300)
Create piece -> board mapping (ex. Q is at e7)
Call stockfish API to get the next best move (ex. e7 to e3)
Go to pick up tile using IK and pick up piece (ex. go to e7 for Q)
Go to drop tile using IK and drop piece (ex. go to e3 and drop Q)

Page updated

Report abuse