This site hosts image similarity data which can be used to evaluate and to improve image similarity metrics.

The data are used in the following CVPR'2014 paper.
"Learning Fine-grained Image Similarity with Deep Ranking",
by Jiang Wang, Yang Song, Thomas Leung, Charles Rosenberg, Jingbin Wang, James Philbin, Bo Chen, Ying Wu