Facebook AI Self-Supervision Challenge

Challenge Details

The challenge is hosted on EvalAI at https://evalai.cloudcv.org/web/challenges/challenge-page/405/overview. All submissions must be uploaded there.

Phase 1: CLS-Places205 - Image Classification on the Places205 dataset.

In this challenge we evaluate fixed feature representations of a self-supervised model. We ask the participants to train a linear classifier using SGD on fixed feature representations using the hyperparameters defined in our codebase. We ask you to submit the results for your best performing layer (e.g. res4 layer in a ResNet-50 or conv4 layer in a AlexNet).

Data

We use the image classification task on the Places205 dataset. The linear classifiers are trained on the train split and we ask the participants to submit the predictions on the val split.

You can find example predictions here. The results are the predictions of a single classifier on the best performing layer.

Phase 2: CLS-VOC07 - Image Classification on the VOC2007 dataset.

In this challenge we evaluate fixed feature representations of a self-supervised model. We ask the participants to train a linear SVM on fixed feature representations using the hyperparameters defined in our codebase. We ask you to submit the results for your best performing layer (e.g. res4 layer in a ResNet-50 or conv4 layer in a AlexNet).

Data

We use the image classification task on the VOC2007 dataset. The SVM classifiers are trained on the trainval split and we ask the participants to submit the predictions on the test split.

You can find example predictions here. The results are the predictions of a single classifier on the best performing layer.

Phase 3: DET-VOC07 - Object Detection on the VOC2007 dataset.

In this challenge we evaluate fixed ConvNet backbone of a self-supervised model. We ask the participants to train a Fast R-CNN model model with a frozen backbone (training only ROI heads) using fixed bounding box proposals. The step-wise instructions are provided in our codebase.

Data

We use the object detection task on the VOC2007 dataset. The setup for training the Fast R-CNN model can be found here (including config files, bounding box proposals).

You can find example predictions here.

Phase 4: Lowshot-VOC07 - Low-shot Image Classification on the VOC2007 dataset.

In this challenge we evaluate fixed feature representations of a self-supervised model. We ask the participants to train a linear SVM classifier fixed feature representations using the hyperparameters defined in our codebase. We ask you to submit the results for your best performing layer (e.g. res4 layer in a ResNet-50 or conv4 layer in a AlexNet).

Data

We use the image classification task on the VOC2007 dataset. The SVM classifiers are trained on the low-shot splits and we ask the participants to submit the predictions on the full test split.

You can find example predictions here. The results are the predictions of a single classifier on the best performing layer.

Note - The example predictions are provided for only 2 independent samples and low-shot values. For the submission, participants must provide it for 5 independent samples and all low-shot values [1, 2, 4, 8, 16, 32, 64, 96].

FAQs

Q: Can I train a separate model for each track? Potentially on separate data and using a separate method?

A: Participants can take part in any or all of the phases below. However, if you take part in multiple phases, we do ask that you keep your self-supervised constant across phases. Please do not train a separate self-supervised model for each track below.