Temporal relation classification
About
Temporal Relation Classification, one of the most important temporal information extraction tasks, involves classifying a given event-event pair or event-time pair as one of a set of predefined temporal relations such as Before, After, Overlap, etc.
This page is the access site of two different rulesets developed for the task of Temporal Relation Classification in newswire data and in medical data, respectively. Our work on enhancing the state-of-the-art in temporal relation classification is extended as we move from the newswire dataset to the medical dataset. The ruleset developed on newswire data, presented in Section 1, was employed only within a classification experimental setting. Whereas, the ruleset presented in Section 2, developed on clinical data, was employed in two different experimental settings: 1) classification of human-annotated entity pair links, and 2) classification of automatically linked entity pairs. Along with making accessible the rulesets used in classification, the pruning heuristics employed in the identification system are also provided at the page linked to in Section 2.
- Temporal Relation Classification in News Articles
Our approach to the classification task is described in the following paper.
Jennifer D’Souza and Vincent Ng. 2013. Classifying Temporal Relations with Rich Linguistic Knowledge. In Proceedings of NAACL-HLT. pp. 918-927.
Download
The download link below contains our manually desgined rules to facilitate classification of all 14 temporal relation types annotated in the TimeBank(v1.2) corpus.
Download rules employed in our hybrid system.
- Temporal Relation Identification and Classification in Medical Data
Our systems for the relation identification and classification task are described in the following papers. The second paper differs from the first in that it offers a more comprehensive and detailed description of the system, as well as includes a new category of features to further improve the classification task, namely, semantic medical relations.
Jennifer D'Souza and Vincent Ng. 2013. Temporal Relation Identification and Classification in Clinical Notes. In Proceedings of the Fourth ACM Conference on Bioinformatics, Computational Biology and Biomedicine. pp. 392-401.
Jennifer D’Souza and Vincent Ng. 2014. Knowledge-rich temporal relation identification and classification in clinical notes. Database (2014) Vol. 2014: article ID bau0109; doi:10.1093/database/bau109.
One of the challenges encountered in trying to improve task performance was that of missing annotations for temporal relations in the manually annotated datasets. The presence of missing relations presents problems to both the training and evaluation of temporal relation extraction systems: not only do they cause a temporal relation extraction system to be unfairly evaluated, but they also cause systems to be trained on instances with incorrect temporal relation labels. In order to alleviate this problem, we manually annotated certain types of missing links that were not automatically recoverable. In the following paper, we describe the guidelines used in our annotation task.
Jennifer D’Souza and Vincent Ng. 2014. Annotating Inter-Sentence Temporal Relations in Clinical Notes. In Proceedings of the Ninth Language Resources and Evaluation Conference, pp. 2758-2765.
Download
The download link below provides access to the pruning heuristics employed in the identifier, and the ruleset employed in the classifier. The rules were designed to facilitate classification of all 12 temporal relation types annotated in the comprehensive version of i2b2 Temporal Relations Challenge corpus (see here for more information on corpus access).
Download heuristics and rules employed in our hybrid system.
Below are the datasets with our extended annotations for missing temporal relations in the original corpus. The annotations are provided in terms of the offsets of the words in the text.
Funding Statement
This work was supported in part by NSF Grants IIS-1147644 and IIS-1219142. Any opinions, findings, or conclusions expressed above are those of the authors and do not necessarily reflect the views or official policies of NSF.