Facebook AI Self-Supervision Challenge

Overview

The Facebook AI self-supervision learning challenge (FASSL) aims to benchmark self-supervised visual representations on a diverse set of tasks and datasets using a standardized transfer learning setup.

In this first iteration, we base our challenge on the following subset of the tasks defined in Goyal et al., 2019

  1. CLS-Places205: Image Classification on the Places205 dataset. Training a linear classifier on fixed feature representations.
  2. CLS-VOC07: Image Classification on the VOC2007 dataset. Training a linear SVM on fixed feature representations.
  3. DET-VOC07: Object Detection on the VOC2007 dataset. Training only the ROI heads of a Fast R-CNN model with precomputed proposals.
  4. Lowshot-VOC07: Image Classification on the VOC2007 dataset using a small subset of the training data. Training a linear SVM on fixed feature representations.

We will announce the results at the ICCV Workshop on Extreme Vision at Seoul, Korea on October 27, 2019.

Winners

The challenge has concluded and the leaderboard can be viewed at https://evalai.cloudcv.org/web/challenges/challenge-page/405/leaderboard.

We are excited to announce the winners of the challenge!

  1. Winner of CLS-Places205: Team MMLab-SelfSup
  2. Winner of CLS-VOC07: Team MMLab-SelfSup
  3. Winner of DET-VOC07: Team MMLab-SelfSup
  4. Winner of Lowshot-VOC07: Team MMLab-SelfSup

Honorable mention to Team Leapfrog for being a runner up in three tracks.

The members of these teams are

1. Team MMLab-SelfSup - Xiaohang Zhan, Jiahao Xie, Ziwei Liu, Yew Soon Ong, and Chen Change Loy [slides] [report]

2. Team Leapfrog - Runmin Doing, Yi Yao, Yi Zhao [slides] [report]

These two teams will present their approaches at the ICCV Workshop on Extreme Vision Modeling on October 27th at 9:30am in Seoul, Korea.

Dates

Challenge submission deadline: October 15, 2019

Results announced: October 20, 2019

This challenge is now over. See the results and winners.

Participation (Starter Code)


Teams should base their submission using the codebase from https://github.com/facebookresearch/fair_self_supervision_benchmark/tree/master/iccv_challenge

We encourage the teams to use the hyper-parameters and transfer learning setup from the same codebase and our paper.

Challenge Details

Please read the detailed instructions on the challenge here.

Our challenge is hosted on Eval AI at https://evalai.cloudcv.org/web/challenges/challenge-page/405/overview

Contact information

You can reach us at ssl-challenge-iccv19@fb.com with any questions or concerns about the challenge.

Competition Rules

This challenge is meant to evaluate self-supervised representations only. This means that the representations are trained without any human/semantic labels. For example, pre-training on ImageNet with labels is NOT a self-supervised method. Similary, pre-training on images from the web (like Flickr, Instagram etc.) using the tags as labels is NOT a self-supervised method. However, pre-training on ImageNet images only (without labels) or Flickr images only (without tags, GPS or other metadata info etc.) to learn a representation counts as a self-supervised method.

If you are not sure whether your method counts as a self-supervised one, please reach out to us.


NO PURCHASE NECESSARY TO ENTER/WIN. A PURCHASE WILL NOT INCREASE YOUR CHANCES OF WINNING. Submission period ends October 15, 2019 at 11:59:59 pm ET. Finalists must submit Abstract by October 23, 2019 ant 11:59:59 pm ET. Open to legal residents of the Territory, 18+ & age of majority. "Territory" means any country, state, or province where the laws of the US or local law do not prohibit participating or receiving a prize in the Challenge and excludes Cuba, Iran, North Korea, Sudan, Syria and any other jurisdiction or area designated by the United States Treasury's Office of Foreign Assets Control. Void outside the Territory and where prohibited by law. Participation subject to Official Rules. See Official Rules for entry requirements, judging criteria and full details. No cash prize will be awarded. See Rules for industry recognition opportunities for winners. Sponsor: Facebook, Inc., 1 Hacker Way, Menlo Park, CA 94025 USA.