Mrinmoy Ghorai Homepage - Subspace Clustering and HOSVD in Image Inpainting

Subspace Clustering and HOSVD in Image Inpainting

Mrinmoy Ghorai* Sekhar Mandal ** Bhabatosh Chanda*

* Indian Statistical Institute , Kolkata ** Indian Institute of Engineering Science and Technology, Shibpur

Abstract

This work presents a Markov random field (MRF) based image inpainting algorithm, which performs patch selection based on groups of similar patches and determines optimal patch assignment through joint patch refinement of selected top similar (candidate) patches. In patch selection, a novel group-based strategy is introduced to search the candidate patches in relevant source region specified by some pre-defined groups. This overcomes the problem of searching patches in the whole source region or improper sub-regions of the input image. Further, we propose an efficient patch refinement scheme to process each candidate patch for capturing underlying pattern. This eliminates random variation and unwanted artifacts as well. Finally, this pattern is incorporated as a weighted data term in the objective function of the MRF model to improve the optimal patch assignment. Here, we use subspace clustering and higher order singular value decomposition (HOSVD) for group formation and patch refinement, respectively. Experimental results on a large number of natural images and comparisons with well known existing methods demonstrate the efficacy and superiority of the proposed method.

Overview

Group Formation and Patch Selection

Steps:

A few groups are constructed by merging a some subgroups, which are, in turn, the collection of source patches provided by the boundary patches.
The candidate patches are selected from a proper search space obtained from some selected groups of patches.

HOSVD-based Patch Refinement

Selected candidate patches are filtered jointly to capture the core pattern among them which helps to find an optimal patch for the corresponding target patch.
Joint filtering is realized through higher order singular value decomposition (HOSVD).
We propose a new thresholding technique, namely soft thresholding, that nullifies a few larger coefficients representing the artifacts, and hence, preserves the texture clarity and image sharpness.

Comparison with State-of-the-Art Methods

Some Failure Cases

Publication

Mrinmoy Ghorai, Sekhar Mandal and Bhabatosh Chanda, A Group-based Image Inpainting using Patch Refinement in MRF Framework, IEEE Transactions on Image Processing, vol. 27, no. 2, pp. 556-567, 2018.
Mrinmoy Ghorai and Bhabatosh Chanda, An Image Inpainting Algorithm Using Higher Order Singular Value Decomposition, IEEE International Conference on Pattern Recognition (ICPR), 2014: 2867-2872.
Mrinmoy Ghorai, Sekhar Mandal and Bhabatosh Chanda, A Two-Step Image Inpainting Algorithm Using Tensor SVD, Asian Conference on Computer Vision (ACCV) Workshops (2) 2014, 63-77