Research

Current Research Interests

• • Evaluating Large Language Models

  • Machine Translation evaluation

  • Question answering and explanations

Previous Research Topics

• • NLP for Climate Change

  • Multilingual dialogue dataset construction

  • Uncertainty detection

  • Linguistic modality

  • Temporality

  • Entailment Graphs

  • Open Information Extraction (OIE)

  • Discourse in Machine Translation

Collaborations

• Joakim Nivre, Olof Görnerup, Gabriele Messori, Clare Flynn, Ni Li, and Wim Thiery: Detecting extreme climate events and their impacts from text (2023-2024)

• Jussi Karlgren, Aarne Talman, Joakim Nivre, Luise Dürlich, Evangelia Gogoulou, Magnus Sahlgren: ELOQUENT lab at CLEF 2024, on evaluating Large Language Models (2023- )

• Nikita Moghe, Chantal Amrhein, Arnisa Fazla: Challenge sets for Machine Translation metrics (2022-2024)

• Nikita Moghe, Evgeniia Razumovskaia, Ivan Vulic, Alexandra Birch: Multilingual dialogue dataset construction (2021-2022)

• Sander Bijl de Vroe, Javad Hosseini, Miloš Stanojević, Mark Johnson, Mark Steedman: Temporality and modality for Entailment Graphs (2020-2022)

• Nick McKenna, Sander Bijl de Vroe, Javad Hosseini, Mark Johnson, Mark Steedman: Multivalent Entailment Graphs (2020-2021)

• Bonnie Webber, Christian Hardmeier, Sharid Loáiciga, Jörg Tiedemann, Ekaterina Lapshinova-Koltunski, Aaron Smith: Various projects on pronoun translation and evaluation (2012-2018)

• Christian Hardmeier, Sharid Loáiciga, Ekaterina Lapshinova-Koltunski: Shared Task Evaluation at WMT (2018)

• Christian Hardmeier, Preslav Nakov, Sara Stymne, Jörg Tiedemann, Yannick Versley, Mauro Cettolo, Bonnie Webber, Andrei Popescu-Belis: Shared Task on Pronoun Translation at WMT (2016)

PhD Students (co-supervised)

• Nick Ferguson, CDT in NLP student at the University of Edinburgh. Nick is working on explainability for the FRANK QA system.

CDT Individual Project in Advanced NLP (co-supervised; primary supervisor and marker)

• Investigating the use of Paraphrase Generation for Question Reformulation in the FRANK QA System (Nick Ferguson, 2021/22)

Masters Projects (co-supervised)

• • Sarcasm Detection on Twitter: Contextual features and Collection methods (Rory Higgins, 2017)

  • Contextual Hate Speech Detection on Twitter (Jason Taylor, 2017)

  • Identifying Social Media Users Across Platforms and Highlighting Privacy Concerns (Timo Mulder, 2017)

  • Text-based Geolocation Inference on Twitter and its Implications on Privacy (Alexander Caughey, 2017)

  • Statistical Machine Translation of Russian with Full Morphological Generalization (Jekaterina Silivask, 2016)

  • Automatically Resolving 2nd Person Pronouns in Public Presentations (Pingping Dai, 2014) 

  • Text Simplification Using SMT and Automated Word Sense Disambiguation (Stefanie Hegle, 2013) 

  • Evaluation of SMT Generated Simplified Text (Anni Zhang, 2013) 

Shared Tasks

• • 2023: Co-organiser of the ELOQUENT lab at CLEF 2024

  • 2022: Participated in the Challenge Sets subtask of the WMT 2022 Metrics Shared Task

  • 2018: Organised the evaluation of pronoun translation for English-German translation at WMT 2018

  • 2016: Organised the shared task on cross-lingual pronoun prediction at WMT 2016

  • 2015: Participated in the DiscoMT shared task on pronoun translation