Denis Paperno

My research focuses on how linguistic meaning can be computed, a question that extends naturally to understanding what modern computational models have learned and how they process information internally. I bring extensive experience analyzing linguistic phenomena across diverse languages, combined with developing rigorous evaluation methods for computational models including state-of-the-art LLMs.

I ask questions like:

• How do computational models compose the meaning of text?

• What internal representations underlie model behavior, and how can we verify this causally?

• What would it take to ground interpretability methods in precise theoretical frameworks?

Publications: 2026

• Jim Maar, Denis Paperno, Callum McDougall, and Neel Nanda. What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering. Accepted to ICLR 2026

Publications: 2025

• Abzianidze, Lasha, Lisa Bylinina, and Denis Paperno. Semantics and Deep Learning. Elements in Semantics. 

• Daniil Ignatev, Denis Paperno, and Massimo Poesio. Hypernetworks for Perspectivist Adaptation. In Proceedings of the The 4th Workshop on Perspectivist Approaches to NLP, 111-122

• Yingjin Song, Yupei Du, Denis Paperno, and Albert Gatt. Burn After Reading: Do Multimodal Large Language Models Truly Capture Order of Events in Image Sequences?. In Findings of the Association for Computational Linguistics: ACL 2025, 24316-24342

• Cristina Reguera-Gomez, Denis Paperno, and Maaike de Boer. Empathy vs Neutrality: Designing and Evaluating a Natural Chatbot for the Healthcare Domain. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), pp. 508-517

• Daniil Ignatev, Denis Paperno, and Massimo Poesio. Annotator disagreement in RST annotation schemes. In Society for Computation in Linguistics 8(1): 7

• Pablo Mosteiro and Damian Blasi, and Denis Paperno. West Germanic noun-noun compounds and the morphology-syntax trade-off. In Proceedings of the The 22nd SIGMORPHON workshop on Computational Morphology, Phonology, and Phonetics, pp. 15-22

Publications: 2024

• McNally, Louise, Olivier Bonami, and Denis Paperno. It’s time for a complete theory of partial predictability in language. Theoretical Linguistics 50, no. 1-2 (2024): 77-91. 

• Song, Yingjin, Denis Paperno, and Albert Gatt. Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning. In Proceedings of the 17th International Natural Language Generation Conference, pp. 384-401. 2024. 

Publications: 2023

• Timothee Mickus, Elaine Zosa, and Denis Paperno. Grounded and well-rounded: A Methodological Approach to the Study of Cross-modal and Cross-lingual Grounding. In Findings of EMNLP 2023. [Poster]

• Claudia Tagliaferri, Sofia Axioti, Albert Gatt and Denis Paperno. The Scenario Refiner: Grounding subjects in images at the morphological level. LIMO workshop (Linguistic Insights from and for Multimodal Language Processing @KONVENS 2023).

• Timothee Mickus, Eduardo Calò, Léo Jacqmin, Denis Paperno and Mathieu Constant. "Mann" Is To "Donna” As「国王」is To «Reine»: Adapting The Analogy Task For Multilingual and Contextual Embeddings. Proceedings of the 12th Joint Conference on Lexical and Computational Semantics: 270-283. [Poster]

• Aleksey Tikhonov, Lisa Bylinina and Denis Paperno. Leverage Points In Modality Shifts: Comparing Language-Only and Multimodal Word Representations. Proceedings of the 12th Joint Conference on Lexical and Computational Semantics: 11–17.

Publications: 2022

• • Denis Paperno. On learning an interpreted language with recurrent models. Computational Linguistics. Computational Linguistics 48 (2): 471–482. 

  • Timothee Mickus, Denis Paperno, and Mathieu Constant. How to Dissect a Muppet: The Structure of Transformer Embedding Spaces. Transactions of the Association for Computational Linguistics (2022) 10: 981–996.  

  • Timothee Mickus, Kees van Deemter, Mathieu Constant, and Denis Paperno. Semeval-2022 Task 1: CODWOE -- Comparing Dictionaries and Word Embeddings. 

  • Ryzhova, D. and Paperno, D., 2022. Constructing a typological questionnaire with distributional semantic models. In E. Rakhilina and T. Reznikova (eds.), The Typology of Physical Qualities. John Benjamins, pp. 309-328.

Publications: 2021

• • Timothee Mickus, Mathieu Constant, and Denis Paperno. A Game Interface to Study Semantic Grounding in Text-Based Models. In Proceedings of the 3rd IEEE Conference on Games.

  • Timothee Mickus, Mathieu Constant, and Denis Paperno. About Neural Networks and Writing Definitions. Dictionaries: Journal of the Dictionary Society of North America, Volume 42, Issue 2, 2021, pp. 95-117

Publications: 2020

• • Mickus, T., Bernard, T. and Paperno, D., 2020, December. What Meaning-Form Correlation Has to Compose With: A Study of MFC on Artificial and Natural Language. In Proceedings of the 28th International Conference on Computational Linguistics (pp. 3737-3749). 

  • Timothee Mickus, Denis Paperno, Mathieu Constant, and Kees van Deemter. What do you mean, BERT? Assessing BERT as a Distributional Semantics Model. Proceedings of the Society for Computation in Linguistics: Vol. 3 , Article 34.

  • Nikiforova, S., Deoskar, T., Paperno, D. and Winter, Y., 2020, December. Geo-Aware Image Caption Generation. In Proceedings of the 28th International Conference on Computational Linguistics (pp. 3143-3156). 

  • (in French) Mickus, T., Constant, M. and Paperno, D., 2020. Génération automatique de définitions pour le français (Definition Modeling in French). In Actes de la 6e conférence conjointe Journées d'Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Volume 2: Traitement Automatique des Langues Naturelles (pp. 66-80). 

Publications: 2019

• • Timothee Mickus, Denis Paperno, Mathieu Constant. Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling. Proceedings of The First NLPL Workshop on Deep Learning for Natural Language Processing, Turku, pp. 1-11.

  • Lin Li, Kees van Deemter, Denis Paperno, Jingyu Fan. Choosing between Long and Short Word Forms in Mandarin. Proceedings of the 12th International Conference on Natural Language Generation, pp. 34–39.

  • Denis Paperno and Daria Ryzhova. Automatic construction of lexical typological questionnaires. In Language Documentation and Conservation: Special issue on Methodological Tools for Linguistic Description and Typology, ed. by Aimée Lahaussois and Marine Vuillermet, pp. 45-61. Associated data: 100 questionnaires described in the paper.

  • Mickus, Timothee; Bonami, Olivier; and Paperno, Denis (2019). Distributional Effects of Gender Contrasts Across Categories, Proceedings of the Society for Computation in Linguistics: Vol. 2 , Article 19

Publications: 2018

• • Denis Paperno. Limitations in learning an interpreted language with recurrent models. More complete preprint version. Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 384–386.

  • Anupama Chingacham and Denis Paperno. Generalizing Representations of Lexical Semantic Relations. Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018), Torino, Italy, December 10-12, 2018.

  • Olivier Bonami and Denis Paperno. A characterisation of the inflection-derivation opposition in a distributional vector space. Preprint version. Lingue e Linguaggio VII.2 (2018), pp. 173-196.

• • Alicia Krebs, Alessandro Lenci and Denis Paperno. SemEval-2018 Task 10: Capturing Discriminative Attributes. Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018). Pp. 732–740.

  • Ryzhova D.A., Меlnik A.A., Yershov I.A., Panteleeva I. M., Paperno D.A., Singh Ya., Sobolev M. Automatic data collection in lexical typology. Proceedings of Computational Linguistics and Intellectual Technologies (Dialog) conference 2018.

Publications: 2017

• • Denis Paperno and Edward Keenan. Handbook of Quantifiers in Natural Language: Volume 2. Studies in Linguistics and Philosophy 97. Springer, 2017.

  • (in Russian) Denis Paperno and Anna Maloletniaia. Ben iazyk [The Beng language]. Languages of the World: Mande Languages. Edited by Valentin F. Vydrin, Yulia V. Mazurova, Andrej A. Kibrik, Elena B. Markus. St. Petersburg: Nestor-Historia, 2017. Pp. 1000-1031. 

  • (In French) Claire Gardent and Denis Paperno. Linguistique informatique et linguistique de terrain [Computational linguistics and field linguistics]. La lettre de l'INSHS. Juillet 2017. Pp. 28-31.

Publications: 2016

• • D. Paperno and M. Baroni. When the whole is less than the sum of its parts. Computational Linguistics 42(2): 345-350.

  • G. Kruszewski, D. Paperno, R. Bernardi and M. Baroni. There is no logical negation here, but there are alternatives. Computational Linguistics 42(4) (special issue on Formal Distributional Semantics): 637-660. The data set described in this article.

  • Alicia Krebs and Denis Paperno. 2016. When Hyperparameters Help: Beneficial Parameter Combinations in Distributional Semantic Models. Proceedings of *SEM 2016 (Fourth Joint Conference on Lexical and Computational Semantics), East Stroudsburg PA: ACL.

  • D. Ryzhova, M. Kyuseva, and D. Paperno. 2016. Typology of adjectives benchmark for compositional distributional models. Proceedings of the 10th Language Resources and Evaluation Conference, 2016: 1253-1257.

  • D. Paperno, G. Kruszewski, A. Lazaridou, Q. Pham, R. Bernardi, S. Pezzelle, M. Baroni, G. Boleda and R. Fernandez. 2016. The LAMBADA dataset: Word prediction requiring a broad discourse context. Proceedings of ACL 2016 (54th Annual Meeting of the Association for Computational Linguistics), East Stroudsburg PA: ACL. Details of LAMBADA creation can be found under Supplementary Material. Also: Download the dataset!

• • Panchenko A., Ustalov D., Arefyev N., Paperno D. Konstantinova N., Loukachevitch N. and Biemann C. Human and Machine Judgements about Russian Semantic Relatedness. In Proceedings of the 5th Conference on Analysis of Images, Social Networks and Texts (AIST'2016). Communications in Computer and Information Science (CCIS). Springler-Verlag Berlin Heidelberg

  • Alicia Krebs and Denis Paperno. 2016. Capturing Discriminative Attributes in a Distributional Space: Task Proposal. Presented at RepEval 2016: The First Workshop on Evaluating Vector Space Representations for NLP. Berlin, Germany.

Publications: 2015 

• G. Kruszewski, D. Paperno and M. Baroni. 2015. Deriving Boolean structures from distributional vectors. Transactions of the Association for Computational Linguistics 3: 375-388.

• Panchenko A., Loukachevitch N. V., Ustalov D., Paperno D., Meyer C. M., Konstantinova N. RUSSE: The First International Workshop on Russian Semantic Similarity. In Proceedings of International Conference on Computational Linguistics Dialogue 2015

• S. Ritter, C. Long, D. Paperno, M. Baroni, M. Botvinick and A. Goldberg. 2015. Leveraging preposition ambiguity to assess compositional distributional models of semantics. Proceedings of *SEM 2015 (Fourth Joint Conference on Lexical and Computational Semantics), East Stroudsburg PA: ACL, 199-204. 

• R. Bernardi, G. Boleda, R. Fernandez and D. Paperno. 2015. Distributional semantics in use. Proceedings of LSD 2015 (First Workshop on Linking Computational Models of Lexical, Sentential and Discourse-level Semantics), East Stroudsburg PA: ACL, 95-101. 

Publications: 2014

• D. Paperno, N. Pham and M. Baroni. 2014. A practical and linguistically-motivated approach to compositional distributional semantics. Proceedings of ACL 2014 (52nd Annual Meeting of the Association for Computational Linguistics), East Stroudsburg PA: ACL, 90-99. Code for the PLF model.

• D. Paperno, M. Marelli, K. Tentori and M. Baroni. 2014. Corpus-based estimates of word association predict biases in judgment of word co-occurrence likelihood. Cognitive Psychology 74: 66-83.

• Grammatical description of Beng, a Mande language spoken in Cote d'Ivoire. Appeared in Mandenkan 51, 2014.

• An Alternative semantics for Negative Conjunction in Russian. To appear. Proceedings of FASL 23, Michigan Slavic Publications, 2014.

• M.Hurlimann, R. Bernardi and D. Paperno. Nominal Coercion in Space. Proceedings of CLIC-it 2014.

Major earlier work (see this page for more papers)

• • PhD Thesis: Semantics and Syntax of Non-Standard Coordination. UCLA, 2012.

  • Conjunction is parallel computation. Proceedings of SALT 22: 403–423, 2012.

  • The Handbook of Quantifiers in Natural Language (co-edited with Ed Keenan) explores typological variation of quantifier structures across languages. The first volume appeared in 2012, the second one in 2017.