Vision-Guided Autonomous Drone Recall Using ArUco Marker Tracking

Project Type: Class Project, MEE 623 – Robot Vision Control
Date: March 2026

Input

The primary input to the computer vision pipeline is a 640 × 360 .mov conveyor-belt video containing fruits and vegetables.

• Video Frame

• Region of Interest (ROI)

• Classification Thresholds 

Control Pipeline

1. Video Acquisition and ROI Extraction

The video is loaded using OpenCV VideoCapture. Each frame is captured sequentially, and only the predefined conveyor-belt Region of Interest is extracted for further processing.

2. Color Space Conversion

The cropped BGR image is converted to the XYZ color space before background subtraction. After foreground extraction, the masked color image is converted to HSV for color analysis and dominant hue estimation.

3. Background Subtraction

A K-Nearest Neighbors (KNN) background subtraction algorithm is used to separate moving fruits and vegetables from the stationary conveyor background.The background subtractor generates a foreground mask containing the moving objects.

4. Shadow Removal and Mask Cleaning

Shadow pixels identified by the KNN algorithm are removed from the foreground mask. A 5 × 5 elliptical morphological kernel is then applied.

5. Object Segmentation and Contour Detection

The foreground mask is applied to the original color image to preserve only the moving object pixels. Contours are then extracted from the binary mask, where each contour represents a potential fruit or vegetable.

Objects touching the edge of the processing region are rejected to reduce incorrect classification of partially visible objects.

6. Geometric Feature Extraction

For each detected contour, the system calculates:

• Object position (x, y)

• Bounding box width and height

• Contour area

• Contour perimeter

• Circularity

Circularity = 4πA / P²

where A is the contour area and P is the contour perimeter. A circularity value close to 1 represents a more circular object, while elongated or irregular objects produce lower values. This geometric feature helps distinguish objects that may have similar colors but different shapes.

7. Dominant Hue Extraction

The detected object's HSV region is extracted using its bounding box and foreground mask. A 180-bin hue histogram is calculated using only the pixels belonging to the detected object.

Dominant Hue = argmax(Hue Histogram) × 2

The histogram bin containing the highest number of pixels is selected as the dominant hue. This value serves as the primary color feature for object classification.

8. Saturation Feature Extraction

The saturation channel is extracted from the HSV region of the detected object using its bounding box and foreground mask. A saturation histogram is calculated using only the pixels belonging to the detected object.

Dominant Saturation=argmax(Saturation Histogram)

The histogram bin containing the highest number of object pixels is selected as the dominant saturation value. This feature represents the color intensity or purity of the detected object and is used together with dominant hue to improve classification and distinguish objects with similar hue values but different color saturation levels.

9. Feature-Based Object Classification

The extracted dominant hue, dominant saturation, and circularity are compared against experimentally defined threshold ranges. Hue and saturation are used as color features to identify each fruit or vegetable, while circularity provides an additional geometric constraint for objects with similar color characteristics.

Hue + Saturation + Circularity → Object Classification

Output

The computer vision system generates a real-time annotated video stream containing the detection and classification results.

Thank you for reading! Please check out my other projects as well, and let me know if you have any suggestions for improvement.