Salient Region Detection via High-Dimensional Color Transform

Jiwhan Kim, Dongyoon Han, Yu-Wing Tai, and Junmo Kim

  • Abstract

     In this paper, we introduce a novel technique to automatically detect salient regions of an image via high-dimensional color transform. Our main idea is to represent a saliency map of an image as a linear combination of high-dimensional color space where salient regions and backgrounds can be distinctively separated. This is based on an observation that salient regions often have distinctive colors compared to the background in human perception, but human perception is often complicated and highly nonlinear. By mapping a low dimensional RGB color to a feature vector in a high-dimensional color space, we show that we can linearly separate the salient regions from the background by finding an optimal linear combination of color coefficients in the high-dimensional color space. Our high dimensional color space incorporates multiple color representations including RGB, CIELab, HSV and with gamma corrections to enrich its representative power. Our experimental results on three benchmark datasets show that our technique is effective, and it is computationally efficient in comparison to previous state-of-the-art techniques.


  • Paper

     [1] Jiwhan Kim, Dongyoon Han, Yu-Wing Tai, and Junmo Kim, "Salient Region Detection via High-Dimensional Color Transform and Local Spatial Support", IEEE Transactions on Image Processing, Vol. 25, No. 1, pp. 9-23, Jan. 2016.

     [2] Jiwhan Kim, Dongyoon Han, Yu-Wing Tai, and Junmo Kim, "Salient Region Detection via High-Dimensional Color Transform", CVPR, 2014. [PDF]


  • Code
      [Code] (4.0MB, zip file, Matlab version) (2015.11.11 updated)
      Note : Our code requires the VLfeat library, which can be downloaded at : http://www.vlfeat.org/.


  • Results
Figure 1. Visual comparisons of our results and results from previous methods.  Each image denotes (a) test image, (b) ground truth, (c) our approach.  (d) DRFI [1], (e) GMR [2], (f) HS[3], (g) SF[4], (h) LR[5], (i) RC[6], (j) HC[6], (k) LC[7].

 

 
  • FAQ
     1. Which dataset did you use for training?
     We used the same training set and test set as the paper of Jiang et. al [1]. 
     2. The computational time is more longer than that is described in the paper.
     (We recently updated the source code, which is more faster than the previous version.)
      3. From the Table 1, why global contrast term has 9 dimensions? according to the definition, it's just a scalar.
     We calculated the global contrast from RGB(3), CIELab(3) and HSV(3), these 9 color channels, therefore it has 9 dimensions. 
      4. The dimension of the high-dimensional color is  l = 4*11=44, but how is the 'Dim' in Table 2 related to the 11 different color channel representations?
     There are RGB(3), CIELab(3), Hue(1), Saturation(1), and Gradient of RGB(3), all 11 different color transform space.  And since we use four gamma values for each color space, there exist 11*4 dimensions.
      5. According to the Table 2, it seems that the high-dimensional color space has 44 dimensions, but why the Figure 5 has only three?
     The Figure 5 is just simple visual examples, which shows that a saliency map can be represented by using a linear combination of multiple color channels.
      6. What are the failure cases?
     If identical colors appear in both foreground and background, our work fails. This assumption is explicitly mentioned in our introduction. Of course, if the initialization of color seed estimation is very wrong, our work will also fail.

 

  • References
     [1] H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, " Salient object detection: A discriminative regional feature integration approach", CVPR, 2013.
     [2] C. Yang, L. Zhang, H. Lu, X. Ruan, and M.-H. Yang, "Saliency detection via graph-based manifold ranking", CVPR, 2013.
     [3] Q. Yan, L. Xu, J. Shi, and J. Jia, "Hierarchical saliency detection", CVPR, 2013.
     [4] F. Perazzi, P. Krahenbuhl, Y. Pritch, and A. Hornung, "Saliency filters: Contrast based filtering for salient object detection", CVPR, 2012.
     [5] X. Shen, and Y. Wu, "A unified approach to salient object detection via low rank matrix recovery", CVPR, 2012.
     [6] M. Cheng, G. Zhang, M. Mitra, X. Huang, and S. Hu, "Global contrast based salient region detection", CVPR, 2011.
     [7] Y. Zhai, and M. Shah, "Visual attention detection in video sequences using spatiotemporal cues", ACM Multimedia, 2006.