Challenge

Overview

3D reconstruction is mostly evaluated qualitatively. For this challenge, however, we invite participants to optimize their models for a difficult quantitative task, the IQ test task (IQTT).

Dataset

For this challenge we provide the IQTT dataset, which is designed to be similar to mental rotation IQ test questions. Each element in the dataset consists of 4 images: reference object, answer 1-3, and one of the answers is the reference object, but randomly rotated. For every question, participants have to use their model to pick the rotated model out of the 3 answers.

Link to dataset

Link do data loader

Task

The dataset encourages semi-supervised or unsupervised 3D reconstruction because it contains a large corpus of unlabeled data and only a small set of sampled where the correct answer is known.

Link to development package: TBD

Submissions

The submissions will be evaluated automatically by sending us a json file with the answers for an unlabeled test set. The results will be published in the form of a leaderboard.

Challenge on EvalAI

Awards

The top submission will receive a prize from our sponsors.

Award: TBD

Important Dates

• Beginning of the competition (release of development and test data): 4th March 2019
• Scoreboard opening for testing: 20th March 2019
• Challenge submission finish: 1st May 2019
• Winners announcement: 15th May 2019