Panoramic View from Video

Mosaic-based Video Scene Representation

This page contains my past work of mosaic-based object analysis in the project "Object Highlighting for Mobile Video" at Thomson Corporate Research.

Summary:

Here a mosaic-based visual object analysis framework is presented. A mosaic is created by geometrically aligning a set of images and stiching them together. By warping a sequence of images onto a single reference mosaic image, we not only obtain an overview of the content across the whole sequence but also reduce the spatio-temporal redundancy in the original sequence of images. The mosaic technique can be applied for scene stabilization, change detection, video compression, video indexing and enhancement. There are two kinds of mosaics, static and dynamic mosaics, suitable for different needs and scenarios. To construct a mosaic, some basic modules are motion estimation, image alignment, sequence integration (blending) and residual analysis. We employ KLT-based good feature tracking in the robust LMedS framework to get an initial estimate of dominant (affine) motion between consecutive frames, then a direct method (pixel-based) with M-estimator can refine the estimated camera motion parameters. When the current frame is warped into the mosaic, local alignment is performed to alleviate the accumulated registration errors. Temporal median filtering is applied to blend the images of the sequence together. Eventually the background mosaic would be projected back to each frame to detect the foregrounds and track them in a scene.

Demo Video: Mosaic generation with 'Stephan' videodata.