The goal of the I Can’t Believe It’s Not Better workshop series is to promote “slow science” that pushes back against “leaderboard-ism”, and provides a forum to share surprising or negative results. In 2023 we propose to apply this same approach to the timely topic of foundation models. 

The hype around ChatGPT, Stable Diffusion and SegmentAnything might suggest that all the interesting problems have been solved and artificial general intelligence is just around the corner. In this workshop we cooly reflect on this optimism, inviting submissions on failure modes of foundation models, i.e. unexpected negative results. In addition we invite contributions that will help us understand when we should expect foundation models to disrupt existing sub-fields of ML and when these powerful methods will remain complementary to another sub-field of machine learning.

We invite submissions on the following topics: 

• Failure modes of current foundation models (safety, explainability, methodological limitations, etc.)

• Failure modes of applying foundation models, embeddings or other massive scale deep learning models.

• Development of machine learning methodologies that benefit from foundation models, but necessitate other techniques.

• Meta machine learning research and reflections on the impact of foundation models on the broader field of machine learning.

• Negative scientific findings in a more general sense. In keeping with previous workshops we will accept findings on methodologies or tools that gave surprising negative results without foundation models. Such submissions are encouraged especially with discussion on the relevance of findings in the present climate where foundation models are changing the field. 

Technical submissions may center on machine learning, deep learning or deep learning adjacent fields (causal DL, meta-learning, generative modelling, adversarial examples, probabilistic reasoning, etc) as well as domain specific applications. 

Papers will be assessed on:

• Clarity of writing

• Rigor and transparency in the scientific methodologies employed

• Novelty and significance of insights

• Quality of discussion of limitations

• Reproducibility of results

Selected papers will be optionally included in a special issue of PMLR.  Alternatively, some authors may prefer their paper to be in the non-archival track which is to share preliminary findings that will later go to full review at another venue.

Formatting Instructions & Guidelines

Submissions should use the workshop LaTeX style files and should be anonymous (by using \usepackage{neurips_2023}).

Submissions should be 4-6 pages long (excluding references), and will be evaluated using the following criteria:

• Clarity of writing

• Rigor and transparency in the scientific methodologies employed

• Novelty and significance of insights

• Quality of discussion of limitations

• Reproducibility of results

Authors may include unlimited appendices but reviewers will not be required to take them into account. Where relevant, it is encouraged to include the checklist from the LaTeX template and a broader impact statement but these are neither required nor included in the page limit.

We welcome first time authors to submit to this workshop. The workshop will be run in-person with the possibility to attend virtually; papers may be submitted by both in-person and virtual attendees, although the latter will be limited to a 5 minute video posted on the workshop website.  

Reviewers will nominate papers for spotlight and contributed talks, and two awards: the "Entropic Award," for most surprising negative result, and the "Didactic Award,"' for most well-explained and pedagogical paper. Reviewers will also nominate papers with exemplary scientific rigour and insightful findings for publication in a special edition of PMLR.

Papers should be submitted here.