Program

Zoe Papakipos has been a Research Engineer at FAIR Paris since 2019. She has a Bachelors degree in Computer Science from Brown University, where she did research with Michael Littman on RL for self-driving cars. She currently works on data augmentations and copy detection and her research interests include adversarial robustness and AI fairness.

Abstract: While advances in automated fact-checking are critical in the fight against the spread of misinformation in social media, we argue that more attention is needed in the domain of unfalsifiable claims. In this talk, we outline some promising directions for identifying the prevailing narratives in shared content (image & text) and explore how the associated learned representations can be used to identify misinformation campaigns and sources of polarization.

Abstract: We tackled copy detection by training CNNs with contrastive learning. Training with large memory-bank and hard data augmentation enables the CNNs to obtain more discriminative representation. Our proposed negative embedding subtraction further enhances the copy detection accuracy by a large margin.

GitHub code: https://github.com/lyakaap/fbisc

Arxiv paper: http://arxiv.org/abs/2112.04323

Slides here.

Abstract: This talk presents an efficient way to produce feature rich, high-dimensionality embedding spaces from real life images. The features produced are designed to be independent of augmentations used in real-life cases that appear on social media. Our approach uses convolutional neural networks (CNN) to produce an embedding space. An ArcFace head was used to train the model using automatically produced augmentations. Additionally, we present a way to make an ensemble of different embed dings containing the same semantic information, a way to normalize the resulting embedding using an external dataset, and a novel way to perform quick training of these models with a high number of classes in the ArcFace head. Using this approach we achieved the 2nd place in the 2021 Facebook AI Image Similarity Challenge: Descriptor Track.

GitHub code: https://github.com/socom20/facebook-image-similarity-challenge-2021

Arxiv paper: https://arxiv.org/abs/2112.03415

Slides here.

Sanjay Addicam is a Sr. Principal Engineer and CTO of an Intel group focused on Retail, Banking, Hospitality and education. He focuses on Deploying Autonomous Deep learning solutions on the edge and accomplish training on the edge. Sanjay is a Kaggle competition master with a best rank of 23.

Abstract: Image copy detection is of great importance in real-life social media. In this paper, a bag of tricks and a strong baseline are proposed for image copy detection. Unsupervised pre-training substitutes the commonly-used supervised one. Beyond that, we design a descriptor stretching strategy to stabilize the scores of different queries. Experiments demonstrate that the proposed method is effective. The proposed baseline ranks third out of 526 participants on the Facebook AI Image Similarity Challenge: Descriptor Track.

GitHub code: https://github.com/WangWenhao0716/ISC-Track2-Submission

Arxiv paper: https://arxiv.org/abs/2111.08004

Slides here.

Wenhao Wang is a Research Intern in Baidu Research co-supervised by Yifan Sun and Yi Yang. His research interest is deep metric learning and computer vision. Prior to Baidu, he was a Remote Research Intern in Inception Institute of Artificial Intelligence from 2020 to 2021. He gained his bachelor’s degree in 2021, and will join in ReLER Lab to pursue a Ph.D. degree in the near future.

Yifan Sun is currently a Senior Expert at Baidu Research. His research interests focus on deep metric learning, long-tailed visual recognition, domain adaptation and generalization. He has publications on many top-tier conferences/journals such as CVPR, ICCV, ECCV, NIPS and TPAMI. His papers have received over 3000 citations and some of his researches have been applied into realistic AI business.

Weipu Zhang is an undergraduate student majoring in EECS as well as a research intern in Baidu Research. He had learned and practiced traditional algorithms and data structure for about 3 years. Currently he is interested in data science and machine learning, especially in computer vision and game agent.

Abstract: This presentation demonstrates the 2nd place solution to the Facebook AI Image Similarity Challenge : Matching Track on DrivenData. The solution is based on self-supervised learning, and Vision Transformer(ViT). The main breakthrough comes from concatenating the query and reference image to form as one image and asking ViT to directly predict from the image if the query image used a reference image. With this result, we can see that ViT's global attention is strong enough to learn relations between patches, even when patches include two different images, query and reference.

GitHub code: https://github.com/seungkee/2nd-place-solution-to-Facebook-Image-Similarity-Matching-Track

Arxiv paper: https://arxiv.org/abs/2111.09113

Slides here.

Abstract: As a basic task of computer vision, image similarity retrieval is facing the challenge of large-scale data and image copy attacks. This paper presents our 3rd place solution to the matching track of Image Similarity Challenge (ISC) 2021 organized by Facebook AI. We propose a multi-branch retrieval method of combining global descriptors and local descriptors to cover all attack cases. Specifically, we attempt many strategies to optimize global descriptors, including abundant data augmentations, self-supervised learning with a single Transformer model, overlay detection preprocessing. Moreover, we introduce the robust SIFT feature and GPU Faiss for local retrieval which makes up for the shortcomings of the global retrieval. Finally, KNN-matching algorithm is used to judge the match and merge scores. We show some ablation experiments of our method, which reveals the complementary advantages of global and local features.

GitHub code: https://github.com/sun-xl/ISC2021

Arxiv paper: https://arxiv.org/abs/2112.02373

Slides here.

Xinlong Sun currently works for Tencent in Shenzhen, China. His recent research interests include image retrieval and video retrieval.

Abstract: This talk introduces our approach, named Feature cOmpatible Self-Supervised Learning (FOSSL), to this image similarity challenge. This challenge aims to learn good representations for images in order to perform image retrieval on original images and manipulated images. The key to this challenge is to extract informative image features from unlabelled data, which are edited by hand-crafted PS or automated transformations. In the proposed FOSSL, we adopt the recent popular and effective self-supervised learning techniques and use a multi-task self-supervised learning framework to learn representation models. The multi-task self-supervised learning framework in FOSSL contains a self-distillation branch and an image-restoring branch, which is able to learn invariant features from various transformations. Moreover, in FOSSL we apply ensemble on features obtained from different models to enhance the performance. Specifically, we design an embedding compatible technique to solve the feature compatibility issue resulted from multiple models. The embedding compatible technique in FOSSL "fixes" the reference image features, therefore, query image features from different models are comparable and we fuse the query image features to enhance image representations. With the above-mentioned two methods, the proposed FOSSL is able to extract good representations from images and achieve the top performance in this challenge.

Slides here.