- Video Analysis and Processing for Multiple Distributed Wearable Devices

Advanced technologies have recently empowered new smart tiny devices and visual sensors for a wide range of applications including visual surveillance and mobile augmented reality. These advanced devices are able to address the emerging application challenges by cooperatively using multiple visual sensors. They can also enrich visual information by capturing multi-view images and efficiently interact with the environment information within heterogeneous networks in real-time. However, these smart tiny cameras have limited resources for such time-critical applications. In particular, multi-view object tracking and recognition are challenging in a distributed sensing scenario of cooperatively performing tiny cameras. Gathering and processing information can be further enhanced by utilizing correlations among the multi-view images without communications among them.

This project aims at developing novel algorithms and methods by focusing on designing a novel adaptive signal processing framework for multiple distributed visual data sources targeted to visual understanding and mobile augmented reality applications. The project results in developing algorithms and theoretical results, in particular, two of the highlighted applications, rate-efficient multiview object recognition and video separation from compressive measurements as follows:

1. Rate-efficient multiview object recognition: In support of applications involving multiview sources in distributed object recognition using lightweight cameras, we propose a new method for the distributed coding of sparse sources as visual descriptor histograms extracted from multiview images. The problem is challenging due to the computational and energy constraints at each camera as well as the limitations regarding inter camera communication. Our approach addresses these challenges by exploiting the sparsity of the visual descriptor histograms as well as their intra- and inter-camera correlations. Our method couples distributed source coding of the sparse sources with a new joint recovery algorithm that incorporates multiple side information signals, where prior knowledge (low quality) of all the sparse sources is initially sent to exploit their correlations. Experimental evaluation using the histograms of shift-invariant feature transform (SIFT) descriptors extracted from multiview images shows that our method leads to bit-rate saving of up to 43% compared to the state-of-the-art distributed compressed sensing method with the independent encoding of the sources.

2. Video separation from compressive measurements: This work considers online robust principal component analysis (RPCA) in time-varying decomposition problems such as video foreground-background separation. We propose a compressive online RPCA algorithm that decomposes recursively a sequence of data vectors (e.g., frames) into sparse and low-rank components. Different from conventional batch RPCA, which processes all the data directly, our approach considers a small set of measurements taken per data vector (frame). Moreover, our algorithm can incorporate multiple prior information from previous decomposed vectors via proposing an n -ℓ 1 minimization method. At each time instance, the algorithm recovers the sparse vector by solving the n -ℓ 1 minimization problem-which promotes not only the sparsity of the vector but also its correlation with multiple previously recovered sparse vectors-and, subsequently, updates the low-rank component using incremental singular value decomposition. We also establish theoretical bounds on the number of measurements required to guarantee successful compressive separation under the assumptions of static or slowly changing low-rank components. We evaluate the proposed algorithm using numerical experiments and online video foreground-background separation experiments. The experimental results show that the proposed method outperforms the existing methods.

("Compressive Online Robust Principal Component Analysis Via n-l1 Minimization," IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4314-4329, Sep. 2018.)

References

1. Huynh Van Luong, Nikos Deligiannis, Jürgen Seiler, Søren Forchhammer, and André Kaup, "Sparse Signal Recovery with Multiple Prior Information: Algorithm and Measurement Bounds", Signal Processing, vol. 152, pp. 417-428, Nov. 2018. (Code)
2. Huynh Van Luong, Nikos Deligiannis, Jürgen Seiler, Søren Forchhammer, and André Kaup, "Compressive Online Robust Principal Component Analysis Via n-l1 Minimization," IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4314-4329, Sep. 2018. (Codes: Matlab, C++)
3. Huynh Van Luong, Nikos Deligiannis, S. Forchhammer, and A. Kaup, "Compressive online decomposition of dynamic signals via n-l1 minimization with clustered priors" in IEEE Statistical Signal Processing Workshop, Freiburg, Germany, Jun. 2018.
4. Huynh Van Luong, Nikos Deligiannis, S. Forchhammer, and A. Kaup, "Online decomposition of compressive streaming data using n-l1 cluster-weighted minimization," in Data Compression Conference 2018, Snowbird, Utah, Mar. 2018.
5. S. Prativadibhayankaram, Huynh Van Luong, Thanh-Ha Le, and André Kaup, "Compressive Online Video Background-Foreground Separation Using Multiple Prior Information and Optical Flow", Journal of Imaging, Special Issue on Detection of Moving Objects, vol. 4, no. 7, 90, Jul. 2018. (C++ Code)
6. Huynh Van Luong, N. Deligiannis, J. Seiler, S. Forchhammer, and A. Kaup, "Compressive Online Robust Principal Component Analysis with Multiple Prior Information," in IEEE Global Conference on Signal and Information Processing (GlobalSIP 2017), e-print in arXiv, Montreal, Canada, Nov. 2017. (CORPCA Code)
7. S. Prativadibhayankaram, Huynh Van Luong, Thanh-Ha Le, and André Kaup, "Compressive Online Robust Principal Component Analysis with Optical Flow for Video Foreground-Background Separation," in International Symposium on Information and Communication Technology (SoICT 2017), e-print in arXiv, Nha Trang, Vietnam, Dec. 2017.
8. Huynh Van Luong, N. Deligiannis, S. Forchhammer, and A. Kaup, "Distributed Coding of Multiview Sparse Sources with Joint Recovery," in Picture Coding Symposium 2016 (PCS 2016), Nuremberg, Germany, Dec. 2016.
9. Huynh Van Luong, J. Seiler, A. Kaup, and S. Forchhammer, "Sparse Signal Reconstruction with Multiple Side Information using Adaptive Weights for Multiview Sources," in IEEE International Conference on Image Processing 2016 (ICIP 2016), Phoenix, Arizona, USA, Sep. 2016. (RAMSIA Code)
10. Huynh Van Luong, J. Seiler, A. Kaup, and S. Forchhammer "A Reconstruction Algorithm with Multiple Side Information for Distributed Compression of Sparse Sources," in Data Compression Conference 2016 (DCC 2016), Snowbird, Utah, Apr. 2016.

Copyright notice: Copyright of the above material belongs to the author. The material may not be copied or re-used without permission from the author.