Background Subtraction

Background Subtraction is one of the fundamental pre-processing steps in video processing. The goal is to distinguish between foreground and background for any given image and thus has numerous applications including security, privacy, surveillance and traffic monitoring to name a few. Unfortunately, no single algorithm exists that can handle various challenges associated with background subtraction such as illumination changes, dynamic background, camera jitter etc.

In this project, we are developing an universal and computationally efficient background subtraction algorithm to detect foreground objects that is robust in a wide range of operating conditions. The proposed algorithm first estimates a dense motion field between two consecutive frames, and obtains a motion-based foreground probability estimate for each pixel by comparing the motion field with its low-rank approximation. In parallel, color features are extracted by sliding a fixed-size neighborhood window over the entire image. Using the motion-based probability estimates, highly probable foreground and background color features are identified and used to learn foreground and background appearance models. These models then generate appearance-based probability estimate for each pixel. To overcome the inaccuracies in appearance modeling and background motion approximation, we incorporate an innovative Mega-pixel denoising process that uses color segmentation to smooth out the probability estimates. Finally, the denoised probability estimates are combined with the image gradient map to produce the output foreground mask under the Graph-Cut optimization framework. To cope with non-stationary dynamic scenes, the foreground and background appearance models are continuously updated with highly probable foreground and background color features. Comprehensive evaluation of proposed approach on publicly available test sequences show superiority of our system over other state-of-the-art algorithms.

Selected Publications:

• Sajid, H., S.-C. Cheung, and N. Jacobs. 2018. Motion and Appearance Based Background Subtraction For Freely Moving Cameras. Computer Vision and Image Understanding, Elsevier. (submitted)
• Sajid, H. and S.-C. Cheung. 2017. Universal Multimode Background Subtraction. IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3249-3260, July 2017.
• Sajid, H., S.-C. Cheung, and N. Jacobs. 2016. Appearance based Background Subtraction for PTZ Cameras. Signal Processing: Image Communication, Elsevier, vol. 47, September, pp. 417-425.