Layered images refer to images of the same scene taken at different angles and altitudes. In order to utilize all the information of layer images to the greatest extend, they need somehow to be fused, and the segmentation and registration can be seen as the first job to achieve this.

I proposed a methodology by using a modified multi-phase joint segmentation and registration method, which again falls in the realm of level set method, to segment and register co-existed objects in the layered images. Below show some results:

a rough alignment is carried out by affine transforming the triangles on the left and right images, and manually depict the object of interest

examples of the contour evolution

the segmentation result of one image

Furthermore, optical flow method was used along the object edges, to track the camera movement, and therefore providing as an prediction of next frame contour initialization, in the layered video scenario.

layered video scenario, and the optical flow along the object edges.

Publication:

• P. Chen, H. Krim, and O. L. Mendoza, ''Multi-phase Joint Segmentation-Registration and Object Tracking for Layered Images,'' Image Processing, IEEE Transaction on, vol. 19, issue 7, pp. 1706 – 1719, 2010. [pdf]
• P. Chen, H. Krim, and O. L. Mendoza, ''Exploring approaches to layered image registration,'' Intelligent Computing: Theory and Applications VI 6961(1), pp. 69610J, SPIE, 2008. [pdf]