Invited Speakers
Jackie Chi Kit Cheung is an associate professor at McGill University's School of Computer Science where he co-directs the Reasoning and Learning Lab, and a Canada CIFAR AI Chair at the Mila Quebec AI Institute. His research focuses on natural language generation tasks such as automatic summarization, and on integrating diverse sources of knowledge into NLP systems for pragmatic and common-sense reasoning. He is motivated in particular by how the structure of the world can be reflected in the structure of language processing systems. Dr. Cheung received a best paper award at ACL 2018. He is a consulting researcher at Microsoft Research.
Keeping Track of Entities Over Time, Minds, and Knowledge Sources
Transformer-based language models form the basis of many state-of-the-art NLP systems across a wide range of tasks, including tasks related to discourse and coherence modelling. Yet there is now ample evidence that these systems fail to systematically generalize to rare events, and make mistakes in reasoning and logical inferences. I argue that many of these issues stem from the fact that the training tasks of language modelling and finetuning are not sufficient for keeping track of entity and world states over a discourse, or for comparing or updating world states based on newly ingested text. I'll discuss my lab's work on creating probes, datasets, and evaluations that demonstrate these specific problems. I'll then present work that highlights the promise of explicitly modelling entities in different contexts, from integrating external knowledge for the prediction of rare events, to adjudicating between conflicting world states to solve theory-of-mind questions.
Vera Demberg obtained her PhD degree from the University of Edinburgh in 2010. She then moved to Saarland University, where she first held a research group leader position within the Cluster of Excellence on Multimodal Computing and Interaction. Since 2016, she is a professor for computer science and computational linguistics at Saarland University. Vera Demberg was recently awarded an ERC Starting Grant "Individualized Interaction in Discourse" (2021-2026).
Inter-annotator agreement in discourse annotation -- the role of domain knowledge and individual differences
Disagreements between annotators are a well-known issue in discourse annotation, especially for implicit relations. Our experimental results show that many of the disagreements are due to actual ambiguities in interpretation of the materials. I will talk about experimental studies at our lab, which try to identify the underlying reasons that give rise to the interpretation differences, and will relate them to differences in cognitive processing strategies during comprehension as well as linguistic experience.
We furthermore observed that disgreements in discourse interpretation can sometimes be due to a lack of background knowledge. In a recent study, we recruited specialists in economics and specialists in biology, and asked them to interpret coherence relations from biomedical as well as economics texts. Our study reveals that biologists interpreted coherence relations in biomedical texts more accurately than economists. Our analysis also provides insight into how background knowledge facilitates discourse interpretation.
Finally, I will address the question how we can learn from these findings for computational models of discourse relation classification and will present our recent results on injecting biomedical knowledge into a discourse parser for biomedical texts.