Methods

There are three different tracking methods. The best method depends on the type of source video. Color is best for tracking 3 and 4 ball tricks. For five or more objects, optical flow or camshift are better choices.

Tracking with Colorspaces (link to Python Code - Github):

Tracking using colorspaces is the easiest and most accurate form of tracking, but it only works in high-resolution with slow-moving objects. The key to tracking with colorspaces, is to optimize the video. Each ball must have a unique color that is not found anywhere else in the frame. Excellent lighting is key. Use this Python script to find the specific HSV color ranges for your juggling props.

Tracking using Optical Flow (link to Python Code - Github):

Optical flow is the apparent movement of an object in a set of images. Optical flow works better at higher frame rates. 60fps or greater is required to track 5+ ball juggling. As frame rate increases, the distance the ball moves in successive frames decreases. The ball is easier to track if is moves a smaller distance between successive frames, so a higher frame rate makes tracking easier.

One disadvantage of optical flow, is that when the tracker gets lost. The user has to pause the tracker and manually reset it to the correct place.

A smarter tracker can be created by combining optical flow and colorspaces. This tracker can reset itself when it gets lost. Link to code.

Tracking using a Hybrid Approach

Here is a link to the hybrid tracker code. The best approach is to use both color and optical flow to track the balls. Color tracking fails for two reasons: when the ball travels quickly it becomes a blur and the colors are distorted, and sometimes the ball is obscured by the hand.

This hybrid tracker uses color to snap onto the ball's location, and if color tracking is not available, optical flow is used instead. This tracker works best at high frame rates.