Labeling Interface
In causal cloze tasks, the participants are asked to write an event description given a cause or outcome event. We specify the expectation of responses (such as using complete sentences, avoiding ambiguous third-person pronouns, etc.) in the instruction. We design a small comprehension quiz with 7 multiple choice questions and 2 chances to submit to ensure the participants understand the instructions correctly.
In the CEG annotation tasks, the participants are asked to label the correctness and causal relation of two event descriptions. We give 4 examples with detailed explanations to help them understand the rationale of the task. We also give an illustration of object colors and shapes for reference. Bounding boxes are added to the videos to accelerate the process of locating objects of interest.
The participants are first asked to select if the event descriptions are correct. If both correct, they are asked to label the level of causal relations between the descriptions. For each event pair, we provide bounding boxes for objects involved in the events for better annotation efficiency.