clothing_retrieval

Deep Bi-directional Cross-triplet Embedding for Cross-Domain Clothing Retrieval

Shuhui Jiang¹ , Yue Wu ¹ and Yun Fu^1,2

¹Department of Electrical and Computer Engineering, Northeastern University, Boston, MA 02115, USA

²College of Computer and Information Science, Northeastern University, Boston, MA 02115, USA

{shjiang,yunfu}@ece.neu.edu

abstract

In this paper, we address two practical problems when shopping online: 1) What will I look like when wearing this clothing on the street? 2) How to find the exact same or similar clothing that other people are wearing on the street or in a movie? In this paper, we jointly solve these two problems with one bi-directional shop-to-street street-to-shop clothing retrieval framework. There are three main challenges of cross-domain clothing retrieval task. First is to learn the discrepancy (e.g., background, pose, illumination) between street domain and shop domain clothing. Second, both intra-domain and cross-domain similarity need to be considered during feature embedding. Third, there is large bias between the number of matched and non-matched street and shop pairs. To solve these challenges, in this paper, we propose a deep bi-directional cross-triplet embedding algorithm by extending the start-of-the-art triplet embedding into cross-domain retrieval scenario. Extensive experiments demonstrate the effectiveness of the proposed algorithm.

Framework

Shop-to-Street Retrieval Results

Street-to-Shop Retrieval Results