In conjunction with EMNLP 2025 and CODI-CRAC 2025 workshop
November 9, 2025
📢 News
[09-12] Notifications for paper acceptance have been sent out! Congrats to all authors! Looking forward to seeing you at CODI-CRAC (W11 at EMNLP)!
[07-16] Test Data released, including some surprise datasets! All at: https://github.com/disrpt/sharedtask2025
[06-18] Training Data released, including some new datasets: https://github.com/disrpt/sharedtask2025
[06-10] Check our FAQ page for updated shared task rules (esp. 5,6,7)
[05-29] Parameter count limitation: Closed-track participants must ensure that the total number of parameters in their system is below 4 billion.
[05-16] Sample Data released! Go and check: https://github.com/disrpt/sharedtask2025
Please check our FAQ page for more information about participation, evaluation etc.!
With the rise of LLMs, study of coherence relations in frameworks such as RST (Mann & Thompson 1988), SDRT (Asher & Lascarides 2003) and PDTB (Miltsakaki et al. 2004), has entered a new level in which models can help advance discourse parsing (Kurfalı 2022; Nishida and Matsumoto 2022; Huber et al. 2022; Li et al. 2023, 2024a,b; Metheniti et al. 2024; Maekawa et al. 2024), produce new data (Huber & Carenini 2022; Liu et al. 2024), leverage discourse modeling for downstreadm tasks (Xing et al. 2022; Li et al. 2023; Cimino et al. 2024) and understand errors in parser predictions (Liu et al. 2023; Pastor & Oostdijk 2024). At the same time, evaluation of results in discourse parsing remains challenging (see Morey et al. 2017, Braud et al. 2024), and progress in harmonizing discourse treebanking frameworks for joint approaches has been slow. The DISRPT shared tasks on discourse unit segmentation, connective identification and discourse relation classification across formalisms (see below) aim to promote convergence of resources and a joint evaluation of discourse parsing approaches, following three past campaigns (Zeldes et al. 2019; Zeldes et al. 2021; Braud et al. 2023).
More frameworks, with datasets from: RST / eRST, SDRT, PDTB, ISO 24617, and discourse dependencies.
New corpora and new languages, some of them kept a surprise!
A unified set of labels for the discourse relation prediction task, to make evaluation across datasets easier. More details coming soon!
A new constraint: only one multilingual model should be submitted per task, with a limited parameter count (<=4B)!
Two tracks:
Closed track: Parameter-count limited (total params <= 4B), openly reproducible models will be evaluated by the DISRPT team and ranked.
Open track: We also welcome descriptions of systems based on large / closed models, but these will not participate in the final rankings as we cannot evaluate them.
Tasks 1 and 2 were initially proposed in 2019: you can consult the results here and the associated papers here.
Task 3 was first introduced in 2021: you can view the last results for all tasks here and the associated papers here.
The last edition was held in 2023: you can view the last results for all tasks here and the associated papers here.
The DISRPT 2019 workshop introduced the first iteration of a cross-formalism shared task on discourse unit segmentation. Since all major discourse parsing frameworks imply a segmentation of texts into segments, learning segmentations for and from diverse resources is a promising area for converging methods and insights. We provide training, development, and test datasets from all available languages and treebanks in the RST, eRST, SDRT and PDTB formalisms, using a uniform format. Because different corpora, languages and frameworks use different guidelines for segmentation, the shared task is meant to promote design of flexible methods for dealing with various guidelines, and help to push forward the discussion of standards for discourse units. For datasets which have treebanks, we will evaluate in two different scenarios: with and without gold syntax, or otherwise using provided automatic parses for comparison.
Since 2019, DISRPT has included a shared task on cross-lingual connective identification, using data annotated within the PDTB framework. We repeat the task in 2025 with updated datasets and connective data from the eRST framework, aiming to identify the location of discourse connectives indicating the presence of coherence relations.
We also continue the past iterations of a cross-formalism shared task on discourse relation classification introduced in 2021. Data is converted from several distinct, but overlapping frameworks: Rhetorical Structure Theory (RST, Mann & Thompson 1988) and its enhanced version (eRST, Zeldes et al. 2024), Penn Discourse Treebank (PDTB, Miltsatsaki et al. 2004), Segmented Discourse Representation Theory (SDRT, Asher & Lascarides 2003), relations following ISO 24617 (ISO 2016), and Discourse Dependencies. The goal of the shared task is to bring together diverse strands of research on discourse relation identification, which are sometimes siloed separately due to differences between underlying data structures and assumptions of different frameworks. In order to enable approaches benefiting from multiple datasets created using distinct points of view, the task aims to find a common denominator in representing all available datasets, for the widest possible range of languages.
Data for the shared task will be soon released via GitHub together with format documentation and tools. A sample of the data is already available on: https://github.com/disrpt/sharedtask2025.
May 16 2025 – Sample data release
June 17 2025 – Training / dev data release
July 16 2025 – Test data release
August 4 2025 – System and paper submissions due
September 12 2025 – Notification of acceptance
September 19 2025 – Camera ready paper due
November 9 2025 – CODI-CRAC workshop at EMNLP
📧 Contact
Google group for participants, please join us on: disrpt2025_participants@googlegroups.com to receive e-mail updates whenever new data is made available for the shared task.
Discord group for participants, please join us on: https://discord.gg/3f7JuTYs
To contact the organisers, you can also send an email to disrpt_chairs@googlegroups.com.
Asher, Nicholas, and Alex Lascarides (2003) Logics of Conversation. Cambridge: Cambridge University Press.
Cimino, Gaetano, Chuyuan Li, Giuseppe Carenini, and Vincenzo Deufemia (2024) Coherence-based Dialogue Discourse Structure Extraction using Open-Source Large Language Models. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 297–316, Kyoto, Japan. Association for Computational Linguistics.
Huber, P., Xing, L., & Carenini, G. (2022) Predicting above-sentence discourse structure using distant supervision from topic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 10, pp. 10794-10802).
ISO (2016) ISO 24617-8:2016 Language resource management — Semantic annotation framework (SemAF)Part 8: Semantic relations in discourse, core annotation schema (DR-core).
Kurfalı, M. (2022) Contributions to Shallow Discourse Parsing: To English and beyond (Doctoral dissertation, Department of Linguistics, Stockholm University).
Liu, Yang Janet, Tatsuya Aoyama, Wesley Scivetti, Yilun Zhu, Shabnam Behzad, Lauren Elizabeth Levine, Jessica Lin, Devika Tiwari, and Amir Zeldes (2024) GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domains. In: Proceedings of EMNLP 2024. Miami, FL, 12287–12303.
Mann, William C., and Sandra A. Thompson (1988) Rhetorical Structure Theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse 8(3), 243–281.
Li, Chuyuan, Patrick Huber, Wen Xiao, Maxime Amblard, Chloe Braud, and Giuseppe Carenini (2023a) Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2562–2579, Dubrovnik, Croatia. Association for Computational Linguistics.
Li, Chuyuan, Yuwei Yin, and Giuseppe Carenini (2024a) Dialogue Discourse Parsing as Generation: A Sequence-to-Sequence LLM-based Approach. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 1–14, Kyoto, Japan. Association for Computational Linguistics.
Li, Chuyuan, Chloé Braud, Maxime Amblard, and Giuseppe Carenini (2024b) Discourse Relation Prediction and Discourse Parsing in Dialogues with Minimal Supervision. In Proceedings of the 5th Workshop on Computational Approaches to Discourse (CODI 2024), pages 161–176, St. Julians, Malta. Association for Computational Linguistics.
Aru Maekawa, Tsutomu Hirao, Hidetaka Kamigaito, and Manabu Okumura (2024) Can we obtain significant success in RST discourse parsing by using Large Language Models?. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2803–2815, St. Julian’s, Malta. Association for Computational Linguistics.
Eleni Metheniti, Philippe Muller, Chloé Braud, and Margarita Hernández Casas (2024) Zero-shot Learning for Multilingual Discourse Relation Classification. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17858–17876, Torino, Italia. ELRA and ICCL.
Miltsakaki, Eleni, Rashmi Prasad, Aravind K. Joshi & Bonnie L. Webber (2004) The Penn Discourse Treebank. In Proceedings of LREC 2004. Lisbon, Portugal.
Morey, Mathieu, Philippe Muller & Nicholas Asher (2017) How Much Progress have we Made on RST Discourse Parsing? A Replication Study of Recent Results on the RST-DT. In: Proceedings of EMNLP 2017. Copenhagen, Denmark, 1319–1324.
Nishida, N., & Matsumoto, Y. (2022) Out-of-Domain Discourse Dependency Parsing via Bootstrapping: An Empirical Analysis on Its Effectiveness and Limitation. Transactions of the Association for Computational Linguistics, 10, 127-144.
Xing, Linzi, Patrick Huber, and Giuseppe Carenini (2022) Improving Topic Segmentation by Injecting Discourse Dependencies. In Proceedings of the 3rd Workshop on Computational Approaches to Discourse, pages 7–18, Gyeongju, Republic of Korea and Online. International Conference on Computational Linguistics.
Yu, N., Zhang, M., Fu, G., & Zhang, M. (2022) RST Discourse Parsing with Second-Stage EDU-Level Pre-training. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 4269-4280).
Zeldes, Amir, Debopam Das, Erick Maziero Galani, Juliano Desiderato Antonio, and Mikel Iruskieta (2019) Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019. Association for Computational Linguistics, Minneapolis, MN, edition.
Zeldes, Amir, Tatsuya Aoyama, Yang Janet Liu, Siyao Peng, Debopam Das, and Luke Gessler (2025) eRST: A Signaled Graph Theory of Discourse Relations and Organization. Computational Linguistics 51(1), 23–72.
Zeldes, Amir, Yang Janet Liu, Mikel Iruskieta, Philippe Muller, Chloé Braud, and Sonia Badene (2021) Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2021). Association for Computational Linguistics, Punta Cana, Dominican Republic, edition.