Workshop on Shortcomings in Vision and Language (SiVL)

NAACL 2019, Minneapolis, USA

6 June, 2019

Welcome to the second edition of the workshop on Shortcomings in Vision and Language (SiVL)!

Previous SiVL workshops: ECCV2018, Munich

The primary purpose of this workshop is to bring together researchers at the intersection of vision and language to discuss shortcomings of modern approaches, tasks, datasets, and evaluation metrics for problems including image and video captioning, visual question answering, visual dialog, activity recognition, image retrieval and referring expressions. By highlighting common shortcomings in these domains, the workshop aims to facilitate discussion of novel research directions and to steer the community towards high-level challenges affecting the vision and language community broadly.

SiVL will strictly adhere to the ACL Anti-Harassment Policy.

Visa info

Visa information is available at (last item in the list to the right):

Visa support letter for NAACL-2019

We hope to see you in Minneapolis!


  • The winners of the best paper award sponsored by Google are Gabriel Grand and Yonatan Belinkov for their paper Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects
  • Building where the workshop will be held: Hyatt Exhibit Hall on the Main Level
  • The poster board measurements are: 8ft wide and 4ft high.
  • The program is online.
  • List of accepted papers and abstracts now available!
  • The workshop will be on Thursday, June 6, 2019 - right after the NAACL-2019 main conference.

Invited Speakers

Angeliki Lazaridou

Senior Research ScientistDeepMind

Margaret Mitchell

Senior Research Scientist Google AI

Yoav Artzi

Assistant ProfessorCornell University

Call for Abstracts and Papers

We call for abstracts and papers exploring shortcomings in current vision and language models covering topics including but not limited to:

  • Analysis of current vision and language models
  • Analysis of current tasks and datasets
  • Novel evaluation metrics
  • Novel language and vision tasks
  • Other pertinent work about shortcomings of vision and language

Papers must describe substantial, original, completed and unpublished work. Paper submissions must be anonymous and will receive at least two peer-reviews. They may consist of up to eight (8) pages of content, plus unlimited pages for references and must be prepared as specified in the NAACL guidelines ( A separate file with supplementary material consisting of additional text (.pdf) or multimedia (.zip) can also be uploaded to aid exposition of the ideas presented in the main paper. Camera-ready versions of accepted papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account. All paper submissions will be presented in the Poster Session and as Spotlight talks. Paper submissions will be published in the ACL Anthology.

Abstracts can describe work in progress, work under review, accepted to be published elsewhere or already published work. They may consist of up to two (2) page of content, plus additional pages for references. No supplementary materials are allowed for abstracts. Abstract submissions are subject to single-blind review to evaluate relevance to the workshop topics. Accepted abstracts will be posted online on the workshop website and will be presented only at the poster session. Abstracts are not published in the ACL Anthology.

All submissions are made via softconf page for SiVL.

[New!] Free Grammarly Premium accounts to authors

Diversity and Inclusion committee for NAACL 2019 has decided to send out access codes for Grammarly Premium accounts to authors of NAACL-2019 workshops. The aim of this step was to help authors from non-English speaking backgrounds improve their submissions in order to encourage participation. Please fill out the following link to request access codes:

Important Dates

February 27, 2019: March 6, 2019: Abstract and paper due date (Submission Link)

March 27, 2019: Notification of acceptance

April 5, 2019: Camera-ready abstract and papers due (firm deadline)

June 6, 2019: Workshop date

All deadlines are : 11:59PM UTC –12h


Raffaella Bernardi

Assistant Professor, University of Trento

Raquel Fernandez

Associate Professor, University of Amsterdam

Spandana Gella

Applied Scientist, Amazon AI

Kushal Kafle

PhD Candidate, Rochester Institute of Technology

Christopher Kanan

Assistant Professor, RIT
Senior AI Scientist, PAIGE

Stefan Lee

Research Scientist, Georgia Tech.

Moin Nabi

Senior Research Scientist, SAP SE

Program Committee

  • Aaditya Prakash, , Brandeis University
  • Abhisek Das, Georgia Tech
  • Aishwarya Agrawal, Georgia Tech
  • Alane Suhr, Cornell University
  • Albert Gatt, University of Malta
  • Anna Rohrbach, University of California, Berkeley
  • Aurelie Herbelot, University of Trento
  • Brandon Birmingham, University of Malta
  • Carina Silberer, Universitat Pompeu Fabra
  • Damien Teney, The University of Adelaide
  • David Schlangen, Bielefeld University
  • Desmond Elliott, University of Copenhagen
  • Douwe Kiela, Facebook AI Research
  • Elia Bruni, University of Amsterdam
  • Emiel van Miltenburg, Tilburg University
  • Enver Sangineto, University of Trento

  • Iacer Calixto, University of Amsterdam
  • Ivan Vulic, University of Cambridge
  • Jasper Uijlings, Google AI
  • Justin Johnson, Stanford University
  • Katerina Pastra, Cognitive Systems Research Institute
  • Lisa Anne Hendricks, UC Berkeley
  • Lucia Specia, University of Sheffield
  • Malihe Alikhani, Rutgers University
  • Nazli Ikizler-Cinbis, Hacettepe University
  • Peratham Wiriyathammabhum, University of Maryland, College Park
  • Pranava Madhyastha, University of Sheffield
  • Ramakanth Pasunuru, UNC Chapel Hill
  • Ranjay Krishna, Stanford University
  • Sandro Pezzelle, University of Amsterdam
  • Sina Zarriess, Bielefeld University
  • Volkan Cirik, Carnegie Mellon University
  • Yash Goyal, Georgia Tech

For further information, please, contact: