Self Similarity based image search


This site presents an implementation method for matching images based on their self-similarity patterns. The work presented here is based upon the approach presented within the article "Matching Local Self-Similarities across Images and Videos" By Eli Shechtman and Michal Irani.
You can find the aforementioned article and more information here.

What does self similarity mean?


Self-similarity is the property held by those parts of an image which resemble themselves in comparison to other parts of the picture. Resemblance may be defined by texture, color and any combination of both. This property of self-similarity enables human visual perception to distinguish some parts of the image from other parts,  which in turn facilitates the recognition of shapes, objects, scenes and more.


General Approach


There are different approaches as to how the task of image comparison may be best accomplished. Most approaches assume that images share a common visual property such as pixels, colors, and edges between them. This assumption might not always hold true, as illustrated by the following figures:

 
   


It is self-evident that there is no obvious property shared between the above figures regarding color and texture. However, the similarity between the "Star of David" symbols creates a clear correlation between the two images. The similarity of these figures is manifested by the internal local patterns of each figure.  These patterns are repeated in a similar relative geometric layout in both images to form the shape of the symbol.

Although the color-texture patterns generating these self-similarities are not shared by those images, they do share internal layouts of self-similarities, and therefore can be matched according to them.

About our implementation


We implemented the described approach as a final year project of our first degree in computer science at the Academic College of Tel-Aviv Yaffo. In this project we have created a program that tries to find a query image (template) within other images. The implementation includes a fully functional set of functions written in C++ and Open CV framework, and also includes a MATLAB integration kit using MEX files.

To watch some of our test results you can navigate to the Results Index Page.

The project was supervised and directed by Dr. Tal Hassner.

For more details you can contact us at:

Yoad Snapir - YoadSn@gmail.com
Rachely Esman - RachelyEsman@gmail.com


September 2008

Comments