Computer vision

Color blob webcam tracker

A Python OpenCV and Python Lego NXT implementation of an object tracking webcam that is mounted on a Lego pan-tilt device.

Using OpenCV 3.4.0 with an HSV color space transform for object color detection.

The program requires a connected webcam and an NXT pan-tilt mechanical setup and has three operating modes:

• 'm' key Mark: enter this more while holding a uniformly colored object in front of the webcam. High contrast colors seem to work best, with a uniform background. Use the mouse to mark a region over the object, which selects the color region to track as a blob.
• 's' key Show: this is a test mode that shows the tracked blob. Use this mode to test blob tracking on the display.
• 't' key Track: this mode activates the motors and sends power controls to them. The Track mode will be halted (red 'Track' symbol) if the blob is lost, or more than one blob is detected. Tracking will resume automatically when a single blob is re-detected.

Kernelized Correlation Filters (KCF) with pan-tilt object tracking

A Python OpenCV and Python Lego NXT implementation of an object tracking webcam that is mounted on a Lego pan-tilt device.

The program uses OpenCV 3.4.0 with a KCF (Kernelized Correlation Filters) tracker [https://www.learnopencv.com/object-tracking-using-opencv-cpp-python/].

The program requires a connected webcam and an NXT pan-tilt mechanical setup and has three operating modes:

• 'm' key Mark: enter this mode while holding an object in front of the webcam. Use the mouse to mark the region that you want to track.
• 's' key Show: this is a test mode that shows the tracked object. Use this mode to test tracking on the display.
• 't' key Track: this mode activates the motors and sends power controls to them. The Track mode will be halted (red rectangle) if the object is lost. Tracking will resume automatically when the is re-detected.

Mechanical setup is slightly different than the original setup. I added worm gears to the drive train that enabled the PID to run the motors at a higher power setting (RPM). This provides smoother and a more responsive pan-tilt movement.

Mechanical setup

A few options exist for a Pan-Tilt mechanical setup. There is a nice Differential Pan & Tilt: [https://www.youtube.com/watch?v=NSRRrAH-9cA] that I built first, but, being made out of Lego, it proved not to be robust enough. The backlash from the motors and the gears, especially the horizontal one, was excessive. The vertical axis was too 'wobbly' to hold a webcam.

I opted for this arrangement [https://www.youtube.com/watch?v=ei3JFqVvChU]. The tilt movement, when panning, was not an issue because the tilt PID took care of any error. The mechanical arrangement was much 'tighter' and backlash was minimal due to better gearing. There are many other arrangements to be found, but these were the most elegant ones to prototype quickly with Lego Mindstorms parts and NXT.

Software

Available on GitHub [https://github.com/eyalabraham/computer-vision]