Multiple-Kernel Tracking

The software implements multiple-kernel tracking. For more technical details, please refer to the following papers:

C. Chu, J. Hwang, H. Pai, and K. Lan, “Tracking human under occlusion based on adaptive multiple kernels with projected gradients,” IEEE Transactions on Multimedia, 2013.
C. Chu, J. Hwang, H. Pai, and K. Lan, “Robust video object tracking based on multiple kernels with projected gradients,” Proc. IEEE Conf. on ASSP, May, 2011.

Resources: The implementation uses OpenCV Version 2.1.0Please download the OpenCV library before trying the binary and the source code.


I. Change the directory of the video and other parameters in "tracking_config.txt". Change the basic parameters for quick experiment. For extensive experiments, check the paper or the source code for adjusting the additional parameters.

II. Execute.

III. Choose the codec for the result video like Fig. 1.
Fig. 1

IV. New window called "tracking" pops out. Change to this window and press "t".   ("t" stands for "tracking")

V. Initialization starts. Three types of initialization methods (selected in "tracking_config.txt"):

0: use mouse to select each kernel
This method allows you to mark the ROI for the model of each kernel individually. This usually gives better tracking result since the initial models include less noise.
  • New window called "image" pops out. This should be the first frame of your test video. (The system saves this image as "SelectROIFrame.jpeg" as reference)
  • Click on the image around the ROI for the model of the first kernel. A blue circle stands for each click and white lines connect two adjacent clicks like Fig. 2. Note that in order to make a valid ROI, the last click should be close to the first click, i.e., two blue circles should overlap (Fig. 3.)
  • After the selection of ROI is done, click "Enter". A new window called "ROI image" is shown. It creates a binary mask image named "KernelMask0.jpeg" that shows the mask for the model of  the first kernel. (Fig. 4)
  • Now the first kernel has been created. Go back to the "image" window (Fig. 4), you see those markers have been cleared, so now you can follow the similar steps to select the ROI for the second kernel.
  • After you have successfully created all the kernels, just press "Esc". The system starts tracking.

Fig. 2                                                                                                                                                            Fig. 3
Fig. 4

1: use coordinate
This method uses the centroid and size of the object for initialization.
  • Modify the parameter "BB" in tracking.config. 
  • The four entries correspond to [x coordinate of center      y coordinate of center    width    height]. 
  • In this way, the tracking will start right after the codec selection. No need to select ROI.

2: use mouse to select the whole object:
This method allows you to mark the ROI of the whole object, and the kernels will generated automatically based on the default layout.
  • New window called "image" pops out. This should be the first frame of your test video. 
  • Click on the image around the ROI of the whole object. The method to select the ROI is the same as above (mark the object - -> press "Enter" to complete the selection --> press "Esc" to start tracking). It will creates the models for all the kernels based on the whole ROI you select. 
  • The system starts tracking.

Below are some useful tips for ROI selection, and you may find them handy:
-    If you would like to use existing mask images instead of selecting ROI again, for instance, you want to use the same initial model for doing some repeating experiments, you can put those files under current directory. When you are asked to select ROI, instead of selecting ROIs, just press "Esc", and it will not create/overwrite binary mask images. The system then will use the existing mask images for initialization. Note that, if you want to do this to two kernels, you need to press "Esc" twice, since the initialization steps for different kernels are taken place separately.
-    During selecting ROI, if you mis-click on wrong places, just press "r" ("r" represents "restore"). All the markers will be cleared, and you can start over.
-    If you do not want to select ROI from the first frame of your video, press "p" ("p" stands for "proceed"). The system will proceed to next frame. Keep pressing "p" until the frame you want to use for initialization. Press "t" to start initialization.

VI. Tracking starts.

VII. After the tracking is complete, press "Esc" under the "tracking" window to quit. Now you can find the "result.avi", "tracklog.txt" and result images (optional) under the same folder.