Overview

To address the need for a method to measure the capsule reorientation as well as to facilitate further investigation and experimental data collection, the primary focus of the project shifted to creating the foundations of a comprehensive testing suite and workflow. The primary function of this testing suite is to streamline the creation, operation, and data collection of experiments dealing with the robotic handling of the spheremapping process.

Prior to this project, there were no pre-established software suits that could be used for the team's experimental use. As such, the programming and operation of the following suite was created during the project time frame.

Experimental Workflow

The experimental workflow as depicted above consisted primarily of three subcomponents. The first, RT Tool Box and Robot, involved the physical implementation of the experiment. This required the robot arm to be programmed to perform different testing scenarios of the current spheremapping process. After the programming was completed, they were controlled in conjunction with the camera systems, LabView and Pylon. Per each cycle of robotic programming and data collection, the resulting images and video were saved to a local computer for processing through either OpenCV or MATLAB.

Hardware

The final experimental setup shown above was housed in the micro-fabrication lab at the General Atomics, Energy Group facility. Two high resolution cameras, one with a higher resolution, the other with a higher frame rate, were fixed to capture images and video of the testing scenarios. To avoid losing the 5 etched testing capsules, blue tape was used to cover the optical table mounting holes and silicone sticky pads were placed to capture them in case they dropped.

Improvised polarizing sheet cut-outs were used to diffuse parts of an LED light, which were critical in making the fiducials visible for image processing. In the images below, the left one is of an etched capsule where the lighting has been untouched. The right image is of the same etched capsule using the diffused lighting method.

Software

To determine the orientation of the observed capsule, software was used for image and video processing. Traditional measuring tools were not suitable due to the scale of the target, its glass properties, and the nature of the measurement. The math required to extract pose estimation was first confirmed manually using sample images via MATLAB. The process involved the following steps:

1. Determine pixel coordinates of three points on the circle

2. Determine the equation of the circle for the shell in order to extract the radius information

3. Select & determine coordinates of the fiducial point(s)

4. Convert the fiducial coordinates to 3D coordinates

5. Solve for the rotation matrix, thereby characterizing the rotation of the capsule in Euler angles of yaw, pitch, and roll

After characterizing the rotation of the capsule using the method above, attention shifted to the development of an automated program capable of processing video footage and compiling rotation history in text or CSV (comma separated values) files. The framework for a regression estimation approach using OpenCV (an open source computer vision framework) in Python was established but remains a work in progress as the algorithms for robust tracking are highly involved. Depicted below is the testing implementation of the automated orientation tracking.

As mentioned, the described testing system and workflow are intended to be the foundations of a more comprehensive testing suite. Early testing of the system shows promise - delivering reasonable orientation and drift calculation results to the hundredths of a degree. On another note, the system has been designed to be modular to adapt to different conditions or different systems altogether.