Privacy Aware Video Surveillance

Video Surveillance is a large application domain with many technical challenges. On the application side, our focus has been on how to incorporate privacy protection in surveillance without sacrificing its primary security functions. We have made novel contributions in moving object segmentation, camera placement, camera fusion, video obfuscation, and video data hiding for privacy data preservation.

Moving object segmentation refers to the techniques in identifying the pixels corresponding to moving objects in video sequences. Missed detections occur due to color confusion with background while false alarms can be caused by “uninteresting” moving background such as moving trees and rains. One of our early work provide a survey study and is one of the top cited work on this topic [1]. To increase the robustness of the segmentation algorithms, we have incorporated rapid illumination change adaptation [19], and developed novel fusion techniques in joint thermal-RGB segmentation [13,21] as well as combining object classifiers at different adaptation rates [3] and within a boosting framework [4].

Providing privacy protection to a large camera network needs to overcome a number of technical challenges. Before summarizing our technical contributions, the following video illustrates the technical challenges and highlights our solution:

Specifically, we have made contributions in five different areas. First, most applications require protection only for trusted individuals and a robust subject identification is thus required. We have developed optimal camera placement techniques to maximize the performance in triangulating each subject in the surveillance area under a binary integer programming framework [7,10,11,15,20]. Second, to obfuscate sensitive imaginary in the surveillance video, we propose an efficient object-based in-painting technique that completely removes the protected individuals from the scene while preserving the motion in the background [5][14][16]. Third, it is imperative that the obfuscation process is authenticated and reversible to maintain the integrity of the surveillance record. We are the first to propose the use of data hiding in preserving privacy data [2] and have developed a rate-distortion optimization framework in embedding privacy data in compressed video [8][9][17]. Fourth, we have constructed a privacy data management system in which the access chain of privacy information can only be authorized by the very individuals in the video [14][18][22][23][24]. Finally, we have also extended the privacy framework to other multimedia domains including life-log capture [6] and video conference [12].

Publications:

1. Cheung, S.-C. and C. Kamath. 2004. Robust techniques for background subtraction in urban traffic video. Proceedings of Electronic Imaging: Visual Communications and Image Processing 2004 (Part One), January 20-22 2004, San Jose, California. Bellingham, WA:SPIE. (5308):881-892.
2. Zhang, W., S.-C. Cheung, and M. Chen. 2005. Hiding privacy information in video surveillance system. In IEEE International Conference on Image Processing (ICIP 2005), September 11-14, Genova, Italy. Piscataway, NJ:IEEE: (3)868-871.
3. Cheung, S.-C. and C. Kamath. 2005. Robust Background Subtraction With Foreground Validation for Urban Traffic Video. In EURASIP Journal of Applied Signal Processing, New York, NY: Hindawi Publishing Co., Volume 14, pp. 1-11, August 2005.
4. Grossmann, E., A. Kale, C. Jaynes and S-C. Cheung, 2005. Offline generation of high-quality background subtraction data. British Machine Vision Conference 2005. Best Poster Award.
5. Cheung, S.-C., J. Zhao and M. V. Venkatesh. 2006. Efficient Object-based Video Inpainting. In IEEE International Conference on Image Processing (ICIP 2006), October 8-11, pages 705-708.
6. Chaudhari, J., S.-C. Cheung and M. V. Venkatesh. 2007. Privacy Protection for Lifelog Video. In IEEE Signal Processing Society SAFE 2007: Workshop on Signal Processing Applications for Public Security and Forensics (SAFE 2007), April 11-13, pages 1-5.
7. Zhao, J. and S.-C. Cheung. 2007. Multi-Camera Surveillance with Visual Tagging and Generic Camera Placement. In the Proceedings of the ACM/IEEE International Conference on Distributed Smart Camera (ICDSC 07), Sept. 25-28, 2007, p. 259-266.
8. Paruchuri, J. and S.-C. Cheung. 2008. Joint Optimization of Data Hiding and Video Compression. In IEEE International Symposium on Circuits and Systems (ISCAS 08), May 18-21, Seattle, WA, pp. 3021-3024.
9. Paruchuri, J., S.-C. Cheung and T. Nguyen. 2008. Managing Privacy Data In Pervasive Camera Networks. Proceedings of IEEE International Conference on Image Processing (ICIP 08), October 12-15, 2008, pp. 1676-1679. (Invited Paper)
10. Zhao, J., S.-C. Cheung and T. Nguyen. 2008. Optimal Camera Network Configurations for Visual Tagging. In IEEE Journal on Selected Topics in Signal Processing, Volume 2, Number 4, August, 2008, pp. 464-479.
11. Zhao, J. and S.-C. Cheung. 2009. Optimal Visual Sensor Planning. Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS 09), May 22-24, 2009, pp. 165-168.
12. M., V. Venkatesh, J. Zhao, L. Profitt and S.-C. Cheung. 2009. Audio-visual Privacy Protection for Video Conference. Proceedings of the IEEE International Conference on Multimedia Expo (ICME 09), June 28 - July 2, 2009, pp. 1574-1575.
13. Zhao, J., and S.-C. Cheung. 2009. Human Segmentation by Fusing Visible-light and Thermal Imaginary. Proceedings of the Ninth IEEE International Workshop on Video Surveillance at IEEE International Conference on Computer Vision (ICCV 2009), Sept. 27 – Oct. 4, 2009, pp. 1185-1192.
14. Venkatesh, M. V., S.-C. Cheung, J. Paruchuri, J. Zhao and T. Nguyen. 2009. Protecting and Managing Privacy Information In Video Surveillance Systems. In Protecting Privacy in Video Surveillance, edited by Andrew Senior, Springer, 2009.
15. Zhao, J., S.-C. Cheung and T. Nguyen. 2009. Camera Network Configuration and its application in Privacy-protected Video Surveillance. In Multi-Camera Networks: Concepts and Applications, edited by H. Aghajan and A. Cavallaro, Elsevier Science and Technology Book Group, 2009.
16. M., V. Venkatesh, S.-C. Cheung and J. Zhao. 2009. Efficient Object-Based Video Inpainting. In Pattern Recognition Letter, Special Issue on Video-based Object and Event Analysis, Volume 30, Issue 2, January 2009, pp. 168-179.
17. Paruchuri, J., S.-C. Cheung and M. Hail. 2009. Video data-hiding for managing privacy information in surveillance systems. EURASIP Journal on Information Security, Volume 2009, Article 236139.
18. S. Yee, Y. Lou, J. Zhao and S.-C. Cheung. 2009. Anonymous Biometric Access Control. EURASIP Journal on Information Security, Volume 2009, Article 865259.
19. Paruchuri, J., E. Sathiyamoorthy, S.-C. Cheung and C.-H. Chen. 2011. Spatially Adaptive Illumination Modeling for Background Subtraction. International Conference on Computer Vision (ICCV) - 11th IEEE Workshop on Visual Surveillance (VS 2011), Nov. 6-13, 2011, pp. 1745-1752.
20. Zhao, J., D. Hawkes, R. Yoshida, and S.-C. Cheung. 2011. Approximate techniques in solving optimal camera placement problems. International Conference on Computer Vision (ICCV) - 11th IEEE Workshop on Visual Surveillance (VS 2011), Nov. 6-13, 2011, pp. 1705-1712.
21. Zhao, J. and Sen-ching, S.C., 2014. Human segmentation by geometrically fusing visible-light and thermal imageries. Multimedia tools and applications, 73(1), pp.61-89.
22. Paruchuri, J., Y. Luo and S.-C. Cheung, 2012. Preserving and Managing Privacy Information in Video Surveillance Systems. In Effective Surveillance for Homeland Security: Balancing Technology and Social Issues, edited by F. Flammini, R. Setola & G. Franceschetti, CRC Press/Taylor & Francis, pp. 87-109.
23. Luo, Y., and S.-C. Cheung. 2013. Privacy Information Management for Video Surveillance. In SPIE conference on Defense, Security + Sensing, Biometric and Surveillance Technology for Human and Activity Identification X, April 29 – May 2, 2013, doi:10.1117/12.2015999.
24. Luo, Y., S.-C. Cheung, T. Pignata, R. Lazzeretti, and M. Barni. 2018. Anonymous Subject Identification and Privacy Information Management in Video Surveillance. International Journal of Information Security, Springer. https://doi.org/10.1007/s10207-017-0380-2.