LSMDC 2016‎ > ‎



Prior to accessing the data you should perform these steps (if not done previously):

Movie description / All challenges

  • Details about the dataset and download procedure are given in the README.txt (here and below: use username/password provided by MPII-MD).
  • Download script for the movie description challenge data [link]

Movie annotation and retrieval

      Training data:
  • The video clips and original annotations can be downloaded using the script in the Movie description (above). This data could be used for training joint visual-language models for challenge tracks on movie multiple-choice test and movie retrieval.
  • Details about the para-phrases data are given in the README_PP.txt
  • Download script for para-phrases data[link]
      Challenge data:
  • Movie Retrieval track: this challenge track will be evaluated  based on random 1000 test clips data [link]
  • Movie Multiple-Choice Test track:
    1. Details about the multiple-choice test are given in the README_MC.txt
    • Download script for multiple-choice test data [link]
    • This track will be evaluated on 10,053 test clips (LSMDC16_multiple_choice_test_randomized.csv) which is provided in above line download script.

Movie fill-in-the-blank

  • The video clips can be downloaded using the script in the Movie description above.
  • Details about the annotation format and download procedure are given in the README_FIB.txt
  • Download script for the movie fill-in-the-blank challenge data [link]

If you have any problems using the data or find any issues, please, contact "arohrbach at mpi­", "torabi.atousa at" and "tegan.maharaj at".


If you intend to publish results that use the data and resources provided by this challenge, please include the following references:

Movie description dataset paper:

author = {
Rohrbach, Anna and Torabi, Atousa and Rohrbach, Marcus and Tandon, Niket and  Pal, Chris and Larochelle, Hugo and Courville, Aaron and Schiele, Bernt},
title = {
Movie Description},
journal={International Journal of Computer Vision},
year = {2017},
url = {}}

Movie annotation and retrieval paper:

author = {Torabi, Atousa and Tandon, Niket and Sigal,Leon}, 
title = {Learning Language-Visual Embedding for Movie Understanding with Natural-Language}, 
journal = {arXiv:1609.08124},
year = {2016}, 
url = {}}

Movie Fill-in-the-Blank paper:

  title={A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering.},
  author={Maharaj, Tegan and Ballas, Nicolas and Rohrbach, Anna and Courville, Aaron C and Pal, Christopher Joseph},
  booktitle={Computer Vision and Pattern Recognition (CVPR)},