Content-based Image Retrieval using semantic graphs

What is content-based image retrieval?

In short, it is search by images. A typical Google search by images returns visually similar images. For example when a user query this image

Google returns the following result

Problems of traditional image retrieval

Google search result looks great! Isn't it? The result images are indeed visually similar to the query image. However, the concept of "visually similar" is ambiguous. Although these images matches the query image very well, it may not be the intention of the end user. End user may try to find:

• Pictures of a specific species of the fish
• Pictures of coral
• Pictures of fish tanks
• Pictures of ocean

Do we know the intention of a query?

Can we know the intention of the end user? Not likely! Without knowing the actual intention of the end user, we can group the result images into groups according to their semantic meanings. For example, in the proposed graph-based content-based image retrieval system, we return the following result

Notice that many pictures in the result is not visually similar to the query image. However, the semantic meaning of the results matches the query image very well. End user could browse the required category more efficiently than the Google search.

Demo system

A live demo system, click here.

Note, the database used in the demo system contains disturbing and inappropriate content. Use it with care! The system is purely for research purpose, no other usage is allowed.