Task Description

Main Task

Following the experience of the previous TempEval Evaluation Exercises, the EVENTI task will consist of 4 subtasks:

• TASK A: Determine the extent and the normalization of temporal expressions (i.e. timex) in a text according to the TimeML TIMEX3 tag. Empty TIMEX3 tags, as specified in the TimeML Annotation Guidelines, will be taken into account as well.
• TASK B: Determine the extent of the events in a text according to the TimeML EVENT tag definition adapted to Italian. In addition, determine the values of the feature CLASS.
• TASK C: Determine temporal relations from raw text. This task involves performing Task A and Task B and subsequently identifying pairs of elements (event\event and event\timex pairs in the same sentence) which stand in a temporal relation (TLINK) and classifying temporal relations among them. As in TempEval-3, all temporal relation values are used.
• TASK D: Determine the temporal relations given two gold items. In particular, given the pair of elements (event\event or event\timex pairs) that have a temporal link, classify the temporal relation between them with the same values as in task C.

Pilot Task: Temporal Processing of Historical Texts

Participants will be asked to run their systems on a corpus of about 5,000 tokens following the same sub-tasks defined for the main task.

No training data will be provided, while the test set will consist of newspaper articles published in “Il Trentino” by Alcide De Gasperi in 1914, the year of the outbreak of World War 1. This topic is particularly relevant in 2014, the 100th anniversary of the Great War.

Participants will be required to process the texts with the same system used for the main task. The results obtained will be used to analyze how well systems built for contemporary languages perform on historical texts. In addition, participants can use any freely available training data for building a new system or for adapting an already existing one.

Use of additional resources

Participants in both tasks may use any additional resources (lexica, Wikipedia, knowledge basis, etc.) provided that a description is included in the final report.