Movie Description

Introduction

Automatically describing open-domain videos using rich natural sentences is among the most challenging tasks of computer vision, natural language processing and machine learning. To stimulate research on this topic, we propose the Large Scale Movie Description and Understanding Challenge (LSMDC), which features a unified version of the recently published large-scale movie datasets (M-VAD and MPII-MD). These datasets have been built using Audio Descriptions (AD) / Descriptive Video Service (DVS) resources for the visually impaired, which are transcribed and aligned to the video. The task in this challenge is, given a clip from a movie, to generate a single sentence that describes this clip.

Challenge description

The proposed dataset, LSMDC16, contains short (4-5 seconds) movie clips with associated sentence descriptions. All the sentences have been manually aligned to the video. We provide a training, validation, public test and blind test set. For the blind test set we do not share the sentence references. The evaluation is done both on public and blind test set, while the challenge winner is determined on the blind test set. Similar to last year we will perform automatic and human evaluation of all submissions. We will soon announce the details of the evaluation protocol.

Submission server

You can submit here.

Download

Data can be downloaded here.

Citations

If you intend to publish results that use the data and resources provided by this challenge, please include the following reference: