We hold regular online or hybrid research seminars (as invited talks or reading groups). Our seminars are supported by IDSAI (Institute of Data Science and Artificial Intelligence) at the University of Exeter and NLP group from the University of Sheffield.
All times below are in London / UK time zone. We update seminar information through Google Groups.
25 Sep 2025, 3-4pm: Dr Nafise Sadat Moosavi at the University of Sheffield
Title: TBA
9 Oct 2025, time TBA: Dr Emmanuele Chersoni at Hong Kong Polytechnic University
Title: TBA
25 Oct 2025, 3-4pm: Prof Chenghua Lin at the University of Manchester
Title: TBA
Previous seminars:
4 Sep 2025, 1-2pm: Dr Edoardo Ponti at the University of Edinburgh
Title: A Blueprint for Adaptive and Tokenizer-free Foundation Models
Abstract: Foundation models (FMs) process information as a sequence of internal representations; however, the length of this sequence is fixed and entirely determined by tokenization. This essentially decouples representation granularity from information content, which exacerbates the deployment costs of FMs and narrows their “horizons” in long sequences. What if, instead, we could free FMs from tokenizers by modelling bytes directly, while making them faster than current tokenizer-bound FMs? I argue that a recipe to achieve this goal already exists. In particular, I helped prototype how to: 1) dynamically pool representations in internal layers, progressively learning abstractions from raw data; 2) compress the KV cache of Transformers during generation without loss of performance; 3) predict multiple bytes per time step in an efficient yet expressive way; 4) retrofit existing tokenizer-bound FMs into byte-level FMs through cross-tokenizer distillation. By blending these ingredients, we may soon witness the emergence of efficient byte-level FMs.
Link to video recording
21 Aug 2025, 4-5pm: Dr Sai Ashish Somayajula at Oracle Cloud Infrastructure & University of California (UC), San Diego
Title: Trustworthy Optimization of Pre-Trained Models for Healthcare: Generalizability, Adaptability, and Security
Abstract: Pre-trained language models have opened new possibilities in healthcare, showing promise in mining scientific literature, analyzing large-scale clinical data, identifying patterns in emerging diseases, and automating workflows, positioning themselves as intelligent research assistants. However, general-purpose models, typically trained on web-scale corpora, often lack the clinical grounding necessary for reliable deployment in high-stakes domains like healthcare. To be effective, they must be adapted to meet domain-specific requirements. My PhD thesis addresses three core challenges in leveraging pre-trained models for healthcare: (i) the scarcity of labeled data for fine-tuning, (ii) the evolving nature of healthcare data, and (iii) the need to ensure transparency and traceability of AI-generated content. In this talk, I will focus on the third challenge: enabling traceability of content generated by large language models. I will begin with an overview of prior watermarking approaches and then present our proposed solution. We introduce a watermarking algorithm applied at inference time that perturbs the model’s logits to bias generation toward a subset of vocabulary tokens determined by a secret key. To ensure that watermarking does not compromise generation quality, we propose a multi-objective optimization (MOO) framework that employs lightweight networks to produce token-specific watermarking logits and splitting ratios, specifying how many tokens to bias and by how much. This approach effectively balances watermark detectability with semantic coherence. Experimental results show that our method significantly improves detectability and robustness against removal attacks while preserving the semantics of the generated text, outperforming existing watermarking techniques.
Link to video recording
18 July 2025, 3-4pm: Prof Massimo Poesio at Queen Mary University of London and the University of Utrecht
Title: Clarify your references in Minecraft
Location: Harrison 203, Streatham Campus, University of Exeter and Microsoft Teams
Link to video recording
26 June 2025, 3-4pm: Dr Tanja Samardžić at the University of Zurich
Title: Understanding text tokenisation across diverse languages
Abstract: Large language models (LLMs) are commonly said to be trained on raw text, but, in fact, they are trained on tokenised text, segmented into small, subword units (tokens). For example, a popular tokenizer splits the word 'plausible' into 'pl aus ible'. Somewhat surprisingly, general data compression algorithms such as BPE turned out more efficient than linguistic subword analyses when used as tokenizers in the context of training LLMs. These algorithms seem to find recurrent patterns that are well exploited by neural networks despite being linguistically incorrect. In this talk, I will show that, despite the perceived linguistic inadequacy, structural features of languages can be extracted from text data by tracking the text compression steps in subword tokenisation. I will present several studies showing that exploiting these features more directly allows us improve cross-lingual transfer and multilingual fairness of pre-trained LLMs.
Link to video recording
12 June 2025, 3-4pm: Prof Yulan He at King's College London
Title: LLMs Need a Bayesian Meta-Reasoning Framework for More Robust and Generalisable Reasoning
Abstract: While Large Language Models (LLMs) excel at many reasoning tasks, they still struggle with robustness, cross-task generalisation, and efficient scaling. Current training approaches, such as next-token prediction, reinforcement learning, and prompt optimisation, offer performance improvements but often lack adaptability across diverse tasks. In this talk, I will advocate for a transformative shift in how LLMs approach reasoning by introducing a Bayesian meta-reasoning framework that equips them with self-awareness, monitoring, evaluation, regulation, and meta-reflection. This framework aims to enhance LLMs' ability to refine reasoning strategies and generalise more effectively. I will discuss key challenges in current approaches and outline future directions for developing more adaptable and generalisable LLMs.
Link to video recording
16 Jan 2025, 3-4pm: Prof Roberto Navigli at Sapienza University of Rome
Title: What's Behind Text? The Long, Challenging Path Towards a Unified Language-Independent Representation of Meaning
Abstract: In the era of Large Language Models (LLMs), the pursuit of a unified, language-independent representation of meaning remains both essential and complex. This talk revisits the rationale for advancing semantic understanding beyond the capabilities of LLMs and highlights the development of a large-scale multilingual inter-task resource like MOSAICo and the design of innovative methods that bridge word- and sentence-level meanings across languages. I will also explore how building a robust, multilingual framework for interpreting meaning with greater precision and depth enhances the quality and reliability of system outputs, including text generated by LLMs.
Link to video recording
12 Dec 2024, 4-5pm: Dr Vered Shwartz at the University of British Columbia
Title: Navigating Cultural Adaptation of LLMs: Knowledge, Context, and Consistency
Abstract: Despite their amazing success, large language models and vision and language models suffer from several limitations. This talk focuses on one of these limitations: the models’ narrow Western, North American, or even US-centric lens, as a result of training on web text and images primarily from US-based users. As a result, users from diverse cultures that are interacting with these tools may feel misunderstood and experience them as less useful. Worse still, when such models are used in applications that make decisions about people’s lives, lack of cultural awareness may lead to models perpetuating stereotypes and reinforcing societal inequalities. In this talk, I will present a line of work from our lab aimed at quantifying and mitigating this bias.
Link to video recording (Passcode: G12jVD+#)
24 Oct 2024, 3-4pm: Rosni Vasu at the University of Zurich
Title: SciHyp: A Fine-grained Dataset Describing Hypotheses and Their Components from Scientific Articles
Abstract: Scientific discovery involves understanding and structuring hypotheses, a challenging task due to the complexity of scientific texts. This talk presents SciHyp, a novel dataset containing RDF descriptions of 689 hypothesis sentences from 479 computer science articles. These hypotheses include relation-finding and comparative types. The dataset was created using a multi-step annotation pipeline with expert annotation, Language Models (LMs) like BERT and Sci-BERT, and crowd-based refinement. Our pipeline effectively identified non-hypothesis sentences with a 96.1% consensus rate between the LMs and crowd annotations, demonstrating its effectiveness in identifying relevant sentences that contain hypotheses. We also used GPT-4 to extract hypothesis components. SciHyp aims to benefit the scientific community by providing a structured dataset for model training and evaluation. The talk concludes with a glimpse into an ongoing project making use of the SciHyp pipeline for scientific hypotheses generation.
Link to video recording (Passcode: Ru7p.EQ?)
10 Oct 2024, 3-4pm: Dr Rodrigo Souza Wilkens at the University of Exeter and previously at Université catholique de Louvain
Title: Assessing Language Proficiency: What Are We Really Measuring with NLP?
Abstract: The use of computational models in assessing language proficiency has grown significantly, yet questions remain about which aspects of language these models can effectively encode. This presentation assesses the capabilities of Large Language Models (LLMs) in the context of language proficiency assessment in the French language, focusing on their application to Automatic Readability Assessment and Automated Essay Scoring. The presentation will address how effectively LLMs capture linguistic skills (e.g., lexical and syntactic knowledge) and assess the potential of combining traditional linguistic features with transformer-based models to enhance the model representation. The presentation will conclude by discussing the alignment between linguistic theory and computational approaches and highlighting both current achievements and ongoing challenges.
Link to slides and video recording (Passcode: 8.@1GgYp)
17 May 2024, 2-3pm: Dr Filip Miletić at the University of Stuttgart.
Title: Modeling the compositionality of noun compounds: Challenges in models and data
Abstract: The meaning of multiword expressions such as noun compounds is characterized by varying degrees of compositionality, i.e. it may be similar to the sum of the component parts (e.g. climate change) or unrelated to it (e.g. silver bullet). This is one key feature that makes computational modeling of compound semantics remarkably challenging. In the first part of the talk, I will present recent work on probing pretrained language models for noun compound semantics, which highlights strong variability in performance with respect to properties of model architectures as well as preprocessing decisions. In the second part of the talk, I will discuss research aiming to model the evolution of compound meanings over time, comparing modeling approaches of various complexity while working on diachronic corpora as a notoriously sparse type of data.
Link to slides and video recording (Passcode: c4PWH?sK)
25 April 2024, 2-3pm: Dr Hang Dong at the University of Exeter.
Title: Integrating Language Models and Knowledge Graphs for NLP in Healthcare
Abstract: Neural Language Models and Knowledge Graphs are two distinct ways in AI with a broadly similar aim to organise data and information for their retrieval. They can be integrated to provide more explainable and trustworthy AI applications. I will present previous research on classifying and linking texts to Knowledge Graphs (including ontologies) in healthcare, including the applications of automated clinical coding, rare disease identification, and discovering new medical concepts to enrich a Knowledge Graph. Finally, I will present a research agenda on integrating Large Language Models and Knowledge Graphs to process heterogeneous data to support healthcare professionals (e.g., in dementia).
Link to slides and video recording (Passcode: h&0m7QcE)
12 April 2024, 2-3pm: Dr Marten van Schijndel at Cornell University.
Title: Next Best Bytes Belie Robust Representations
Abstract: In language applications, vectors from neural network language models are commonly used to encode semantic information. While these proxies work well for natural language processing tasks, it remains an open question how human-like the resulting representations are. First, I will argue that language model training objectives produce language representations that differ in key ways from human linguistic representations. I will then discuss multiple studies from my group that identify weaknesses in standard tokenization and analysis methods which produce mismatches with human linguistic response patterns, after which I will suggest alternatives to better model human responses.
Link to video recording (Passcode: ZVBx%T$7)
2 April 2024, 3-4pm: Prof Lilian Cristine Hübne at Pontifical Catholic University of Rio Grande do Sul - PUCRS - Brazil.
Title: Language and cognitive assessment in (a)typical adulthood and aging using computational tools
Abstract: Studies have demonstrated the influence of aging on language and general cognitive processing. Variables such as educational level and literacy habits over the course of a lifetime can be associated with cognitive-linguistic decline, maintenance, or improvement in healthy older adults and also appear to play a role in the onset of cognitive decline and dementia syndromes. This presentation will discuss studies that have been developed in language and cognitive assessment using computational tools and how the integration of these fields can contribute to the diagnosis of cognitive disorders as well as to the differentiation of groups by socioeconomic and cultural characteristics in adulthood and aging.
Link to slides and video recording (Passcode: #A6#qRWG)