System Implementation

End Effector - Gripper

The gripper was designed to be forgiving and to be lightweight. To accomplish the forgiveness of the gripper, the overall size of the gripper disk is slightly larger than the physical valves and switches. Therefore, in case of inaccurate localization, the gripper disks could still operate on different targets. Also to make the whole gripper assembly light-weight, laser-cutting and 3D printing were used. High friction foam was also added to the disk to ensure a tight grip on the valves. Initial testing showed that this setup is sufficient to turn the valves with little difficulty, as long as the pressure on the valves is sufficient.

Robotic Arm Subsystem

Due to the simple valves placements and simple operation movements, the arm was designed with 2 degrees of freedom. Two linear actuators (vertical and horizontal) were sufficient to approach the targets and also flip the breaker box switches. Testings show that the force exerted by vertical linear actuator was sufficient to flip switches.

Computer Vision and Object Detection

The chosen computer vision method was to use color filters and bounding boxes to find different valves. As the target devices and also their angle markers have special colors, color filtering would help to isolate the targeting devices and angler markers. Then bounding boxes would capture the contours of these isolated devices and angler markers, as shown below. Moreover, the bounding box could provide aditional informations like the center of the target in the image coordinate and also the size of the bounding box; the former would be used to fine-tune the arm position to better align with the target devices and the latter would be used to better filter out unwanted noise. The whole process was computationally fast and also easy to tune.

As the computer vision method would rely on different colors of the devices, lighting conditions were a major consideration during the testing and tuning process. The first step is to normalize the brightness histogram of each image frame. The lighting histogram was flattened, as shown below, so the brightness distribution was more consistent across different lighting conditions.

Also, due to the closeness of some colors like gray and blue, yellow and orange, and white and light green, the color filter ranges for different devices would need to be fine-tuned. The most efficient and controllable way to tune the color range was to use HSV color space, which helps to separate hue, saturation, and value (brightness). The separation of these three values helps to better control the color tuning process . With HSV color space, to differentiate gray to blue and to differentiate yellow, and orange, the lower HSV saturation range would need to be increased; to differentiate between white and light green, the higher HSV value (brightness) range would need to be decreased.

Although color filtering tuning reduced some false results with similar colors, further dimension-related tuning helped to pinpoint the target devices. The bounding box provides the width and length; the ratio of these two values and also the ratio of the physical width and length of the actual devices were used to determine if the identified contours were the target. As shown below, the angle marker dimension helped to accurately only identify the actual angle marker.

Page updated

Report abuse