A NeurIPS 2021 workshop on

ImageNet: past, present, and future


Since its release in 2009, ImageNet has played an instrumental role in the development of deep learning architectures for computer vision, enabling neural networks to greatly outperform hand-crafted visual representations. ImageNet also quickly became the go-to benchmark for model architectures and training techniques which eventually reach far beyond image classification. Today’s models are getting close to “solving” the benchmark. Models trained on ImageNet have been used as strong initialization for numerous downstream tasks. The ImageNet dataset has even been used for tasks going way beyond its initial purpose of the training of classification models. It has been leveraged and reinvented for tasks such as few-shot learning, self-supervised learning and semi-supervised learning. Interesting re-creation of the ImageNet benchmark enables the evaluation of novel challenges like robustness, bias, or concept generalization. More accurate labels have been provided. About 10 years later, ImageNet symbolizes a decade of staggering advances in computer vision, deep learning, and artificial intelligence.

We believe now is a good time to discuss what’s next: Did we solve ImageNet? If not, what is there left to do? What are the main lessons learnt thanks to this benchmark? What should the next generation of ImageNet-like benchmarks encompass? Is language supervision a promising alternative? How can we reflect on the diverse requirements for good datasets and models, such as fairness, privacy, security, generalization, scale, and efficiency?

Our workshop is featured in the December 2021 issue of the Computer Vision News.

← Click to check out!

Schedule & contents

The workshop has taken place on Monday 13 December - Tuesday 14 December 2021, depending on your time zone.

The recordings are available below.

ImageNet Workshop 2021

How to attend the workshop (closed)

You can attend the live event through the NeurIPS virtual site for our workshop. Access to the page requires registration at NeurIPS. You will find links to the Zoom and Gathertown sessions on the virtual site.

How to actively participate in the workshop (closed)

We highly encourage the active participation of the attendees. Attendees can participate in various ways.

Invited talks, spotlight talks and panel sessions

Invited speakers will answer the selected questions during the panel sessions.

Poster sessions

  • Join the Gathertown during the poster sessions and interact with the authors, speakers, other attendees, and organisers.

After the workshop

  • We plan to publish the recorded events from the workshop.

The NeurIPS Code of Conduct applies to our workshop.

Important dates (closed)

Submission system open: 26 August 2021

Deadline for tracks #1 and #2: 18 September 2021 25 September 2021, 23:59 AOE (final; no extension)

Deadline for track #3: 5 December 2021 (Please submit to imagenetworkshoppc@gmail.com)

Author notification: 15 October 2021 22 October 2021

Workshop date: 13 December 2021

Feel free to join the mailing-list to be notified of updates, or simply check back later.

Call for papers (closed)

We call for research work on and around ImageNet to be presented at the workshop, see “Scope and topics” below for details. There are three submission tracks. All accepted papers, including the ones for the main track, are non-archival; that is, there are no formally published proceedings. Nonetheless, we will make all accepted papers available on OpenReview.

1. Main track

We welcome original research papers in the main track. The submissions must be at most 5 pages in length, excluding the references and appendix, and use our modified NeurIPS 2021 style file (see below). Reviewers are not obliged to read the appendix. Please submit a single PDF file that includes the main paper, references, and any appendix. The review process is double-blind, so please ensure that the submission is appropriately anonymised.

2. Extended abstract track

We call for extended abstract submissions that present open problems, novel-yet-not-thoroughly-tested ideas, and any perspective on the future of ImageNet and related research. The submissions must be at most 3 pages in length, excluding the references, and use our modified NeurIPS 2021 style file (see below). The review process is double-blind, so please ensure that the submission is appropriately anonymised.

3. Published work track

To encourage lively discussion around ImageNet, we allow the presentation of previously published papers, if they are highly relevant to the topic. Please note that it is not possible to submit papers accepted at the NeurIPS’21 main conference, as per the NeurIPS guidelines this year. There is no formal style and page limit on this track; authors may submit a published paper as-is. The review process is single-blind. Authors of the accepted papers will be able to present their work during the workshop.

Prizes for best papers

We will award the best papers for the main track (track 1) with a cash prize of 1000 USD. Stay tuned for further updates on the prizes!

Reviewer registration for authors

Given the expected size of the workshop, we will likely be short of reviewers. Authors will be given an option to register a subset of authors as reviewers in the OpenReview submission form. We strongly encourage at least one author for each paper to be registered as a reviewer. The paper reviewing will take place between late September and early October. We aim to assign at most two papers per reviewer.

Links and resources

Submission link: Closed. Track 3 is still open; please submit to imagenetworkshoppc@gmail.com.

Template files:

Scope and topics

All three tracks welcome any submission on the topics below. The list is non-exhaustive; if unsure, please contact the organizers at imagenet21@googlegroups.com.

ImageNet as a benchmark

  • How is ImageNet serving the community as a benchmark?

  • Did we solve the ImageNet classification task?

  • How can we spot and remedy the remaining errors?

  • What are the remaining challenges not captured by ImageNet?

  • How can we measure the research progress on robustness, bias, and other out-of-domain generalization issues using ImageNet?

  • How to fix errors in ImageNet labels and compare results across different labels?

ImageNet as a pre-training dataset

  • Is ImageNet still the best dataset for pre-training?

  • What could a better pre-training dataset look like? What are good properties?

  • Zero- and few-shot learning on or using ImageNet

  • Unsupervised and self-supervised learning

  • Semi-supervised learning

  • Transfer learning

Explorative and innovative ideas even without state-of-the-art (SOTA) results

  • We encourage submissions that are groundbreaking but are difficult to get published at conferences because they do not get SOTA performances.

Beyond ImageNet

  • Recipes for building a large-scale dataset

  • Socially responsible benchmarks and datasets: fairness, privacy, security, robustness, scalability, efficiency, and environmental friendliness

  • Efficiency and quality aspects of annotations and crowdsourcing

  • Going beyond classification labels for supervising visual models - e.g. language descriptions for images

  • Post-ImageNet benchmarks and datasets

Feel free to join the mailing-list to be notified of updates, or simply check back later.

Accepted papers

Best paper award (1000 USD sponsored by Naver)



Published Work Track


Matthias Bethge
Univ. of Tübingen

Emily Denton
Google Research

Vittorio Ferrari
Google Research

Alex Hanna
Google Research



UC Berkeley

Alex Kolesnikov
Google Brain


Yixuan Li

UW Madison

Rebecca Roelofs
Google Brain

Olga Russakovsky
Princeton University

Shibani Santurkar

Dawn Song
UC Berkeley

Kaiyu Yang Princeton University

Ross Wightman
Independent Researcher


You can contact the organizing team via imagenet21@googlegroups.com, and subscribe to the mailing-list to receive organizational updates.

Zeynep Akata
Univ. of Tübingen

Lucas Beyer
Google Brain

Sanghyuk Chun
Naver AI Lab

A. Sophia Koepke
Univ. of Tübingen

Diane Larlus
Naver Labs Europe

Seong Joon Oh
Naver AI Lab

Rafael Rezende
Naver Labs Europe

Sangdoo Yun
Naver AI Lab

Xiaohua Zhai
Google Brain