Milestone 3

3.1 Implementation

Stereo Calibration

The above images are captured by the left and right stereo cameras respectively of a calibration test pattern displayed on a television. This pattern allows the team to measure camera intrinsics to account for distortion, as well as extrinsics to ascertain each camera's relative positioning in the real world.

Insect Positioning

Firstly, stereo video is captured at 1200x720 pixel resolution at 120fps

To determine the position of insects with minimal latency, the team utilized a GPU accelerated background subtraction model using a Mixture of Gaussians algorithm is. Next, a process to determine the centroid of contours is used, this should correspond to the center point of insects. Finally, by using extrinsics (rotation,  translation) and intrinsics (lens distortion, focal length) matrices generated from the stereo calibration step, along with finding correspondences between both cameras by evaluating candidate insect points along epipolar lines, the precise location of an insect can be determined.

Laser Calibration

To determine where the laser is pointing in world coordinates, the location of the laser dot has to be determined with respect to the cameras. Using openCV, the team aims to apply filters to video from cameras to locate the brightest spot. Next, a process to determine the centroid of contours is used, this should correspond to the center point of the laser dot.

3.2 Test

Prototype Fixture and Dual Op-Amp

Below is a demonstration of the utilization of the prototype galvo holder and op-amp for conversion of the analog control signal. The demonstration is done by manually sending coordinate values via serial to the arduino.

Insect Positioning

The video below is a demonstration of the MOG2 background subtraction algorithm, and contour finding used on a sample video of insects. The output still needs to be further processed to remove non-insect elements through further temporal filtering methods.

Laser Calibration

3.3 Teamwork

Our team is committed to innovation and excellence, meeting three times a week to accelerate our progress on developing the prototype. Our project encompasses distinct yet interconnected components—electrical, assembly, and programming. Here's how our team members contribute to our project's success:

Dorzhi Denisov: Dorzhi is at the forefront of our prototype's physical development, overseeing the construction of the prototype fixture as detailed in Section 3.1 (Implementation). His expertise extends to the intricate design of the Printed Circuit Board (PCB), incorporating a sophisticated dual Digital-to-Analog Converter (DAC), Arduino board, and inverter. This setup is crucial for controlling the laser galvanometer with a precise -5 to +5V signal. Furthermore, Dorzhi has developed a prototype breadboard equipped with a dual operational amplifier. This innovative design converts a 0.55-2.75V analog signal from an Arduino Due into the required -5 to +5V range, demonstrating his pivotal role in the electrical engineering aspect of our project.

Chiripol Sirikakan: Specializing in software development, Chiripol plays a critical role in processing and interpreting video input from stereo cameras. Using OpenCV, he is responsible for generating accurate 3D world coordinates of insects, a key feature of our prototype. His responsibilities also include the precise calibration of stereo cameras and laser targeting, ensuring the prototype's accuracy.