PsyNet:

Self-supervised Approach to Object Localization Using Point Symmetric Transformation

People

Kyungjune Baek*, Min Hyun Lee*, Hyunjung Shim (* indicates an equal contribution)

Abstract

  Existing co-localization techniques significantly lose performance over weakly or fully supervised methods in accuracy and inference time. In this paper, we overcome common drawbacks of co-localization techniques by utilizing self-supervised learning approach. The major technical contributions of the proposed method are two-fold. 1) We devise a new geometric transformation, namely point symmetric transformation and utilize its parameters as an artificial label for self-supervised learning. This new transformation can also play the role of region-drop based regularization. 2) We suggest a heat map extraction method for computing the heat map from the network trained by self-supervision, namely class-agnostic activation mapping. It is done by computing the spatial attention map. Based on extensive evaluations, we observe that the proposed method records new state-of-the-art performance in three fine-grained datasets for unsupervised object localization. Moreover, we show that the idea of the proposed method can be adopted in a modified manner to solve the weakly supervised object localization task. As a result, we outperform the current state-of-the-art technique in weakly supervised object localization by a significant gap.

Proposed structure and Contributions

1) On three benchmarks, we achieve substantial improvements over existing state-of-the-arts in unsupervised object localization. More surprisingly, PsyNet outperforms the state-of-the-art weakly supervised techniques, in terms of the GT-known localization accuracy. 

2) To the best of our knowledge, we are the first to introduce self-supervised learning approach in the problem of unsupervised object localization. Beyond simple adaptation, we introduce PST, a transformation for addressing the common issue in object localization. 

3) We show that our class agnostic activation mapping (CAAM) is effective to generate heat map from the CNN trained with self-supervised learning. 

4) PsyNet with a simple modification is also an effective solution for weakly supervised object localization. As a result, we achieve the new state-of-the-art performance in weakly supervised object localization.

Localization Results

The green and red box are the ground truth and predicted bounding box, respectively. For each triplet, result from the original, DDT (Wei et al. 2019) and the proposed method are listed from left to right.

Acknowledgements 

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2019R1A2C2006123), MSIT(Ministry of Science and ICT), Korea, under the "ICT Consilience Creative Program'' (IITP-2019-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Planning & Evaluation), and ICT R&D program of MSIP/IITP. [R7124-16-0004, Development of Intelligent Interaction Technology Based on Context Awareness and Human Intention Understanding]

Publication

PsyNet: Self-supervised Approach to Object Localization Using Point Symmetric Transformation

Kyungjune Baek*, Min Hyun Lee*, Hyunjung Shim (* indicates an equal contribution)

Association for the Advancement of Artificial Intelligence (AAAI), 2020