Convex Optimization Approaches to Image Fusion

Post date: Mar 21, 2011 9:50:0 AM

Image fusion is an imaging technique to visualize information from multiple images in one single image, which is widely used in remote sensing, medical imaging etc. For example, in medical imaging, while both the Magnetic Resonance (MR) and Computed Tomography (CT) imagery provide standard diagnostic tools other than fluoroscopy and ultrasound techniques, it is well-known that a CT scan will adequately highlight the bone structure details while soft tissue information is not clearly visible; on the other hand, a T2 weighted MR scan produces significantly better details for images of these tissues. In this respect, it is highly desirable to have a combined view of CT and MR images, which illustrates significant details both from both CT and MR inputs and assists clinical diagnoses. Pictures below show one example combining two sources of CT (left) and MRI (center) to one image (right).

We study two variational approaches to image fusion which are closely related to the modern TV-L2 and TV-L1 image approximation methods. We investigate their convex optimization models under the perspective of primal and dual and propose the associated new image decomposition models. In addition, we consider the TV-L1 based image fusion approach and study the problem of fusing two discrete-constrained images f1(x) in L_1 and f2(x) in L_2, where L_1 and L_2 are the sets of linearly-ordered discrete values. We prove that the TV-L1 based image fusion actually gives rise to an exact convex relaxation to the corresponding non-convex image fusion, given the discrete-valued constraint which is the combination of L_1 and L_2. This extends the results for the global optimization of the

discrete-constrained TV-L1 image approximation by Chan and Yuan et al to the case of image fusion. The proposed dual models also lead to new fast and reliable global optimization algorithms in numerics, based on modern convex optimization techniques.

In contrast, for fusing two medical images which often contain a large number of discrete gray-values, it is definitely impossible to apply graph-cuts for the global energy minimization! Moreover, such convex relaxation approaches also avoid metrification errors when applying graph-cuts.

Related works:

• • Jing Yuan, Juan Shi, Xue-Cheng Tai, Yuri Boykov, A Study on Convex Optimization Approaches to Image Fusion, International Conference on Scale Space and Variational Methods in Computer Vision (SSVM) 2011