ICIP 2013 Tutorial

Image Filtering 2.0: Efficient Edge-Aware Filtering and Their Applications

A Tutorial at IEEE Int. Conf. on Image Processing (ICIP) 2013

Jiangbo Lu (ADSC), Dongbo Min (ADSC), Minh N. Do (UIUC)

Description: The last decade has witnessed the exciting development of efficient edge-aware filtering (EAF) techniques, stemming from different theories and principles. Thanks to their strong power in adaptively dealing with various visual signals as well as significant computational and implementation advantages, this kind of modern image filtering techniques have found a great variety of applications in image/video processing, computer vision and computer graphics. These applications range broadly from addressing the classical image denoising task to recently tackling fully connected high-order conditional random field (CRFs) for accurate semantic image segmentation and labeling. In these applications, the EAF methods have been employed either to allow for local data adaptivity or to efficiently integrate global supports. In this tutorial, we will first present various state-of-the-art nonlinear EAF techniques, while revealing theoretical connections, new insights and generalization. Especially, we will focus on fast filtering approaches, including using the bilateral grid, color-line model, multipoint aggregation, domain transform, and recursive data propagation. Representative applications of these filtering techniques will be illustrated and discussed, including edge-preserving smoothing, color image denoising, stereo matching, depth map enhancement, stylization, detail enhancement, and image segmentation/matting. We will also cover recent research works that efficiently deal with a fully-connected or high-order Conditional Random Field (CRF) model using the efficient EAF algorithms (e.g. based on mean field approximation or quadratic programming).

After spending a large part of the tutorial on efficient filtering and applications, we will discuss a serious computational challenge faced by most cost volume filtering-based approaches (and also MRF-based approaches), i.e., the curse of the huge discrete space in labeling problems (e.g. dense stereo and optical flow estimation). Along this direction, we will present some recent works to tackle such research challenges e.g. using a randomized hypothesis sampling and propagation approach (e.g. PatchMatch) or importance sampling on the label space, and then introduce several issues that should be addressed for developing more powerful computational tools.

Finally, we will conclude this tutorial by suggesting some intriguing problems and exciting future directions that may drive this adaptive filtering paradigm further into a cost-effective, general-purpose tool for many applications, going beyond the recent attempts of high-order CRFs inference or the randomized search approaches.

Contact

Dr. Dongbo Min: dongbo (at) adsc (dot) com (dot) sg, dbmin99 (at) gmail (dot) com

Dr. Jiangbo Lu: jiangbo (dot) lu (at) adsc (dot) com (dot) sg

Course Material

Part 1: Introduction (ppt, rar)
Part 2: Efficient edge-aware filter techniques (ppt, rar)
Part 3: Image processing and graphics applications (ppt, rar)
Part 4: Dense discrete labeling problems for vision tasks (ppt, rar)

If you can't download the 'ppt' files due to some weird problems of 'Dropbox' (automatically converting 'ppt' files to 'pdf' format), please download the 'rar' files.

Code

Cross-based Local Multipoint Filtering (CLMF, CVPR 2012): C++ code (readme.txt)
Weighted Mode Filtering (WMF, TIP 2012): C++ code (readme.txt)

(Note: More codes will be available, once they are ready.)

Bibliography

NEW: K. Wang, L. Lin, J. Lu, C. Li, and K. Shi, ““PISA: Pixelwise Image Saliency by Aggregating Complimentary Appearance Contrast Measures with Edge-Preserving Coherence,” IEEE Trans. on Image Processing, 2015 (PDF).
NEW: D. Min, S. Choi, J. Lu, B. Ham, K. Sohn, and M. N. Do, “Fast Global Image Smoothing Based on Weighted Least Squares,” IEEE Trans. on Image Processing, 2014 (Project page).
NEW: H. Yang, W.-Y. Lin, and J. Lu, “Daisy Filter Flow: A Generalized Discrete Approach to Dense Correspondences,” in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, Jun. 2014. (Project page)
NEW: D. Vu, B. Chidester, H. Yang, M. N. Do, and J. Lu, “Efficient Hybrid Tree-Based Stereo Matching with Applications to Post-Capture Image Refocusing,” IEEE Trans. on Image Processing (TIP), vol. 23, no. 8, pp. 3428-3442, Aug. 2014. (PDF)
J. Lu, H. Yang, D. Min, and M. N. Do, “PatchMatch Filter: Efficient Edge-Aware Filtering Meets Randomized Search for Fast Correspondence Field Estimation,” in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Portland, Oregon, Jun. 2013.
J. Lu, K. Shi, D. Min, L. Lin, and M. N. Do, "Cross-based local multipoint filtering," in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, June 2012.
D. Min, J. Lu, and M. N. Do, "Joint Histogram Based Cost Aggregation for Stereo Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, Oct. 2013.
K. Shi, K. Wang, J. Lu, and L. Lin, “PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors,” in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Portland, Oregon, Jun. 2013.
D. Min, J. Lu, and M. N. Do, "Depth video enhancement based on weighted mode filtering," IEEE Trans. Image Processing (TIP), vol. 21, no. 3, pp. 1176-1190, March 2012.
K. Zhang, J. Lu, Q. Yang, G. Lafruit, R. Lauwereins, and L. Van Gool, "Real-time and accurate stereo: A scalable approach with bitwise fast voting on CUDA," IEEE Trans. Circuits and System for Video Technology, vol. 21, no. 7, pp. 867-878, July 2011.
K. Zhang, J. Lu, and G. Lafruit, "Cross-based local stereo matching using orthogonal integral images," IEEE Trans. Circuits and System for Video Technology, vol. 19, no. 7, pp. 1073-1079, July 2009.
D. Min, J. Lu, V. Nguyen, and M. N. Do, "Weighted mode filtering and its applications to depth video enhancement and coding," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan, Mar. 2012.
D. Min, J. Lu, and M. N. Do, "A revisit to cost aggregation in stereo matching: How far can we reduce its computational redundancy?" IEEE Int. Conf. Computer Vision (ICCV), Barcelona, Spain, Nov. 2011.
J. Lu, D. Min, R. S. Pahwa, and M. N. Do, "A revisit to MRF-based depth map super-resolution and enhancement," in Proc. IEEE. Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, May 2011, pp. 985-988.
V.-A. Nguyen, D. Min, and M. N. Do, "Efficient Techniques for Depth Video Compression Using Weighted Mode Filtering," IEEE Trans. on Circuits and Systems for Video Technology (TCSVT), 2013.
K. He, J. Sun, and X. Tang, “Guided image filtering,” in European Conference on Computer Vision (ECCV), 2010, pp. 1–14.
S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” in European Conference on Computer Vision (ECCV), 2006.
F. Porikli, “Constant time O(1) bilateral filtering,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008.
Q. Yang, K.-H. Tan, and N. Ahuja, “Real-time O(1) bilateral filtering,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009, pp. 557–564.
A. Adams, N. Gelfand, J. Dolson, and M. Levoy, “Gaussian KD trees for fast high-dimensional filtering,” in ACM SIGGRAPH, 2009, pp. 21:1–21:12.
A. Adams, J. Baek, and M. A. Davis, “Fast high-dimensional filtering using the permutohedral lattice,” Computer Graphics Forum, vol. 29, no. 2, pp. 753–762, 2010.
F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic-range images,” in ACM SIGGRAPH, 2002.
C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz, “Fast cost-volume filtering for visual correspondence and beyond,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011, pp. 3017–3024.
L. De-Maeztu, S. Mattoccia, A. Villanueva, and R. Cabeza, “Linear stereo matching,” in Proc. IEEE Int Computer Vision (ICCV) Conf, 2011, pp. 1708–1715.
Y. Ding, J. Xiao, and J. Yu, “Importance filtering for image retargeting,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011, pp. 89–96.
P. Bauszat, M. Eisemann, and M. Magnor, “Guided image filtering for interactive high-quality global illumination,” Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering (EGSR)), vol. 30, no. 4, pp. 1361–1368, Jun. 2011.
J. Kopf, M. Cohen, D. Lischinski, and M. Uyttendaele, “Joint bilateral upsampling,” in ACM SIGGRAPH, 2007.
K.-J. Yoon and I. S. Kweon, “Adaptive support-weight approach for correspondence search,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 650–656, 2006.
B. Weiss, “Fast median and bilateral filtering,” in ACM SIGGRAPH, vol. 25, no. 3, Jul. 2006, pp. 519–526.
J. Chen, S. Paris, and F. Durand, “Real-time edge-aware image processing with the bilateral grid,” in ACM SIGGRAPH, vol. 26, no. 3, 2007.
E. S. L. Gastal and M. M. Oliveira, “Adaptive manifolds for real-time high-dimensional filtering,” in ACM SIGGRAPH, 2012.
R. Fattal, “Edge-avoiding wavelets and their applications,” in ACM SIGGRAPH, 2009.
E. S. L. Gastal and M. M. Oliveira, “Domain transform for edge-aware image and video processing,” in ACMSIGGRAPH, vol. 30, no. 4, 2011, pp. 69:1–69:12.
V. Katkovnik, A. Foi, K. Egiazarian, and J. Astola, “From local kernel to nonlocal multiple-model image denoising,” International Journal of Computer Vision, pp. 1–32, 2010.
K. He, J. Sun, and X. Tang, “Fast matting using large kernel matting Laplacian matrices,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2010, pp. 2165–2172.
Philipp Krähenbühl and Vladlen Koltun,“Efficient inference in fully connected CRFs with Gaussian edge potentials,” in Proc. of NIPS, 2011.
Yimeng Zhang and Tsuhan Chen, “Efﬁcient inference for fully-connected CRFs with stationarity,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2012.
Vibhav Vineet, Jonathan Warrell, and Philip H.S. Torr, “Filter-based mean-field inference for random fields with higher-order terms and product label-spaces,” in European Conference on Computer Vision (ECCV), 2012.
P. Milanfar, “A tour of modern image filtering,” in IEEE Signal Proc. Magazine, vol. 30, no. 1, Jan. 2013.

Page updated

Google Sites

Report abuse