Anaphora

Anaphora is the linguistic device of referring back to previously introduced entities with expressions such as pronouns, as in

1. The city council refused the women a permit because they feared violence

Anaphora resolution has been one of my main areas of research in the past twenty years or so; the handbook that I co-edited ( Poesio, Stuckardt and Versley, 2016) summarizes the state of the field. Over the years I have worked on the linguistics of anaphora (e.g., the problem of telescoping); on anaphora resolution (or coreference) both from a scientific and a technological perspective; on the creation of anaphorically annotated resources such as ARRAU and GNOME; and on the use of anaphora resolution in applications such as summarization and information extraction.

Anaphoric Annotation

One of the main stumbling blocks for research in semantics and discourse has been the lack of semantically annotated resources of a size and quality comparable to that of the Penn Treebank. As a result, in order to support the research on anaphora discussed below (and in particular our work on salience, bridging references, reference to abstract objects, and disagreements on anaphora) I have become involved in a number of projects concerned with the creation of such resources: the MATE project in which general-purpose anaphoric annotation guidelines were designed, the GNOME project focused on annotation to support research on salience and bridging, the ARRAU project on abstract object anaphora and disagreements on anaphora; the Phrase Detectives project; and the ongoing DALI project. See the page on annotation for more details.

Disagreements in Anaphora Resolution

An issue that became apparent from the very first data collections I was involved with (the TRAINS corpus, the Vieira-Poesio corpus) and then became the focus of several subsequent projects is that of disagreements in the interpretation of anaphoric expressions (Poesio and Reyle, 2001; Poesio, Reyle, and Stevenson, 2007; Poesio et al, 2006). There are many anaphoric expressions on whose interpretation people disagree without finding them ambiguous. An example are pronouns like IT in the following example,

2. Can you kindly hook up engine E3 to the boxcar at Elmira and send IT to Corning as soon as possible please?

These examples led us to formulate the Justified Sloppiness Hypothesis to explain the felicitousness of such cases. In subsequent work we identified a number of other cases of 'felicitous ambiguity', and carried out numerous empirical investigations of the problem, in particular in the ARRAU project, the Phrase Detectives Game-With-A-Purpose for anaphoric annotation (Poesio et al, 2013), and now in the DALI project (see the page on ambiguity for more discussion.)

The Role of Salience in Anaphora Resolution

With regards to anaphora resolution, I have been especially interested in the effect of salience on anaphora resolution (Poesio et al, 2004; Poesio et al, 2006; Karamanis et al, 2009), the use of lexical and commonsense knowledge in the interpretation of so-called bridging references (Poesio et al, 1997; Poesio and Vieira, 1998; Poesio, Schulte im Walde and Brew, 1998; Vieira and Poesio, 2000; Poesio et al, 2002; Poesio et al, 2004) see also next Section) and the treatment of non-anaphoric definites (see Section on the Linguistics of Anaphora).

My work with Janet Hitzeman, Rosemary Stevenson, and other collaborators (in particular Hua Cheng, Renate Henschel, and Nikiforos Karamanis) on the role of salience in anaphoric interpretation started with a careful analysis of the claims of theories of the local focus such as Sidner's theory (Sidner, 1979) or Centering theory (Grosz, Joshi and Weinstein, 1995; Walker, 1998) at the light of empirical evidence. According to such theories, at each stage in the interpretation of a discourse, certain entities are more 'important' or `salient' than others, and this affects the interpretation of anaphoric expressions including pronouns and demonstratives. Consider for instance the contrast pointed out by Gundel between 3a and 3b. Most theories of the local focus predict the apple to be `more salient' than the napkin; this would explain why it is the preferred interpretation of pronoun it in 3a. By contrast, demonstratives are generally used to refer to entities that are not in the local focus; hence the preferred interpretation of that in 3b is the napkin.

3.a Put the apple on the napkin and then move it to the side.

3.b Put the apple on the napkin and then move that to the side.

Our studies involved a combination of traditional behavioral methods from psycholinguistics, of corpus analysis, and of computational modelling. Some of the questions we studied using standard psychological techniques include the effect of rhetorical structure on pronoun interpretation, and especially the interaction between animacy and thematic roles in determining the salience of a discourse entity (Pearson, Stevenson and Poesio, 2001). This work has been continued in recent years by my current student Kevin Glover, who, in his PhD (Glover, 2015) proposes a novel model of animacy and for estimating the animacy of noun phrases from corpora, which he then deploys in a variety of applications ranging from predicting pronominalization to clinical applications. In parallel with this behavioral work, we used annotated corpora (in particular, the GNOME corpus - see below) to evaluate how the claims of local focus theories such as Centering are affected by different ways of setting their `parameters'--e.g., whether position in the sentence or grammatical function are used to rank discourse entities, how frequently the local focus is updated (after every clause or every sentence), etc. (Poesio et al, 2004).

These analyses were subsequently extended in several directions. In collaboration with Natalia Modjeska I used the computational models of the local focus to investigate the claims mentioned above about the relation between the local focus and the use of demonstrativce NPs in English (Poesio et al, 2004). In collaboration with Barbara Di Eugenio and several students including Amrita Patel I tested several models of the global focus proposed in the literature on Centering (Poesio et al, 2006). Finally, in collaboration with Janet Hitzeman, I looked at the relation between local focus and global focus studying the interpretation of so-called long distance pronouns (Hitzeman and Poesio, 1998).

Salience and Noun Phrase Generation in GNOME

The work with Janet Hitzeman and Rosemary Stevenson on psychologically plausible models of anaphora resolution had obvious applications in the area of natural language generation - the development of systems capable of producing natural language that find easy to process. This observation led to a joint project on language generation in which Janet, Rosemary and I collaborated with Donia Scott (then at the University of Brighton) and Barbara di Eugenio from the University of Illinois at Chicago. The GNOME project, which ran from 1998 to 2000, was an EPSRC-funded project whose goal was to apply results from psychological research and from corpus analysis to develop and evaluate algorithms for generating nominal expressions.

In GNOME we developed both hand-crafted and statistical models of the processes involved in discourse entity realization, including statistical models of the choice of NP type (Poesio and Henschel, 1999 ; Poesio et al, 1999 ; Poesio, 2000 ) and NP modification ( Cheng, Poesio, Henschel, and Mellish, 2001 ). We were particularly interested on pronominalization ( Henschel, Cheng and Poesio, 2000 ).

More recently, I collaborated with Nikiforos Karamanis, Chris Mellish and Jon Oberlander on developing statistical models of other types of generation, including text structuring ( Karamanis et al 2009).

Lexical and Commonsense Knowledge in Anaphora Resolution

Definite descriptions like the handles are a particularly interesting class of anaphoric expressions in that lexical and commonsense knowledge is often required to interpret them, unlike the case of pronouns. For instance, in the following example from the GNOME corpus (reported by (Poesio et al, 2004)), `the handles' is an associative reference to the `egg vases' in 4a.

4a. These `egg vases' are of exceptional quality.

4b. Basketwork bases support egg-shaped bodies, and bundles of straw form the handles

Hence, studying how such anaphoric references are interpreted may lead to insights in the interface between grammar and commonsense knowledge. Together with my former student Renata Vieira we first of all produced a classification of bridging references, and tested whether lexical sources such as WordNet provided sufficient information, with mediocre results ( Poesio, Vieira, and Teufel, 1997; Vieira & Poesio, 2000 ). In subsequent work, we started investigating whether the necessary lexical and commonsense knowledge could be acquired in a psychologically plausible way from corpora, using so-called `semantic space' or `distributional' models (Poesio, Schulte im Walde, and Brew, 1998; Poesio, Ishikawa, Schulte in Walde, and Vieira, 2002; Poesio et al, 2004). This work led me to initiate a new line of research in such models for the acquisition of lexical and commonsense knowledge from corpora.

In recent years, I have started investigating the use of Wikipedia to resolve encyclopedic definite descriptions such as the composer in Bach ... the composer, in collaboration with Olga Uryupina (Uryupina et al, 2012).

Summarization and Other Applications of Anaphora Resolution

From a technological perspective, I've been especially interested in the use of anaphora resolution for summarization. In collaboration with Josef Steinberger and Mijail Kabadjov, I managed to show that using anaphora resolution can result in significant improvements in the quality of summaries as measured in terms of scores such as ROUGE (Steinberger et al, 2007).

More recently, I started investigating the use of anaphora resolution to summarize online forums, in the SENSEI project.

BART and Other Anaphora Resolution Systems

Over the years I co-led the development of a number of anaphora resolution systems: first the Vieira-Poesio definite description resolver, then two of the first off-the-shelf toolkits for anaphora resolution, GUITAR (Poesio and Kabadjov, 2004) and more recently BART (Versley et al, 2008; Broscheit et al, 2010; Uryupina et al, 2011, 2012).

The Linguistics of Anaphora

The aspects of the linguistics of anaphora that I investigated include the phenomena of telescoping (Poesio and Zucchi, 1990); of weak definites (Poesio, 1994); and more in general whether familiarity is really the foundation of definiteness (Poesio, 1999, 2001).

`Weak definites' are definite descriptions that seem to have `lost' their presuppositional status ( Poesio, 1994 ). This initial work was based on the assumption that the defining characteristic of definites is familiarity, as suggested, e.g., by Heim (1982). More recently, I have been led to reconsider this assumption, primarily because of my work with Renata Vieira on analyzing the uses of definite descriptions in corpora. We found that more than half of definite descriptions are not `familiar' in an obvious sense. As a result, I began considering theories based on the competing hypothesis, that what characterizes definites is uniqueness. Specifically, I concentrated on Loebner's (1987) theory. Using the GNOME corpus, we compared the familiarity-based account with Loebner's; preliminary results of this work are discussed in (Poesio, 2001 )

I also developed a theory of definite description interpretation, which made use of research on the formal semantics of discourse and on defeasible reasoning ( Poesio, 1992; Poesio, 1993b; Poesio, 1994b; Poesio and Rieser, 2011 ).

Projects (in inverse chronological order)

  • DALI, funded by the ERC (2016-2021). This project is concerned with the role of disagreements in anaphoric interpretation.
  • SENSEI , funded by the EU (2013-2016). This project was concerned with the use of discourse to summarize spoken and online conversations such as those in online forums.
  • LiveMemories, funded by the Provincia di Trento (Italy) (2008-2011). The objective of this project was to develop information extraction techniques to support access to collectively created digital archives. As part of this project we carried out much research on anaphora resolution in Italian (Iida and Poesio, 2011), supported by the creation of the LiveMemories Anaphora corpus (Rodriguez et al, 2010), and more in general on multilingual anaphora resolution (Sikdar et al, 2013)
  • AnaWiki, funded by EPSRC (2007-2009). This project was concerned with developing games with a purpose to collect data about anaphoric reference. The main outcome of the project is the Phrase Detectives game-with-a-purpose, discussed here.
  • ELERFED. The aim of this Johns Hopkins Workshop project, funded by DARPA and NSF, was to develop new approaches to entity disambiguation. One of its outcomes was the BART toolkit.
  • ARRAU, funded by EPSRC (2004-2007). Its aim was to study 'difficult' cases of anaphoric reference. One of its outcomes was the ARRAU corpus.
  • Cases of Unresolved Underspecification, funded by the Royal Society (1998-2000). This project, a collaboration with Uwe Reyle from University of Stuttgart, was focused on the study of the cases of anaphoric underspecification that then became one of the foci of the ARRAU project.
  • GNOME, funded by EPSRC (1998-2000). This project, a collaboration between the Universities of Edinburgh, Brighton, and Durham, was concerned with the generation of anaphoric expressions. Its outcomes include much research on salience, and the GNOME corpus.

Courses and talks on the subject

Main publications

  • Massimo Poesio, Roland Stuckardt, and Yannick Versley, 2016. Anaphora Resolution: Algorithms, Resources, and Applications. Springer.
  • Steinberger, Josef, Mijail Kabadjov and Massimo Poesio, 2016. Coreference applications to summarization. In M. Poesio, R. Stuckardt, and Y. Versley (eds.), Anaphora Resolution: Algorithms, Resources and Applications. Springer. Chapter 15. (pdf)
  • Versley, Yannick, Massimo Poesio and Simone Ponzetto, 2016. Using Lexical and Commonsense Knowledge for Anaphora Resolution. In M. Poesio, R. Stuckardt, and Y. Versley (eds.), Anaphora Resolution: Algorithms, Resources and Applications. Springer. Chapter 14. (pdf)
  • Uryupina, Olga, Mijail Kabadjov and Massimo Poesio, 2016. Detecting non-reference and non-anaphoricity. In M. Poesio, R. Stuckardt, and Y. Versley (eds.), Anaphora Resolution: Algorithms, Resources and Applications. Springer. Chapter 13. (pdf)
  • Poesio, Massimo, Sameer Pradhan, Marta Recasens, Kepa Rodriguez, and Yannick Versley, 2016. Annotated Corpora and Annotation Tools. In M. Poesio, R. Stuckardt, and Y. Versley (eds.), Anaphora Resolution: Algorithms, Resources and Applications. Springer. Chapter 4. (pdf)
  • Poesio, Massimo, Roland Stuckardt, Yannick Versley and Renata Vieira, 2016. Early Approaches to Anaphora Resolution: Theoretically Inspired and Heuristic-Based. In M. Poesio, R. Stuckardt, and Y. Versley (eds.), Anaphora Resolution: Algorithms, Resources and Applications. Springer. Chapter 3. (pdf)
  • Poesio, Massimo, 2016. Linguistic and Cognitive Evidence About Anaphora. In M. Poesio, R. Stuckardt, and Y. Versley (eds.), Anaphora Resolution: Algorithms, Resources and Applications. Springer. Chapter 2. (pdf)
  • Massimo Poesio, Jon Chamberlain, Udo Kruschwitz, Livio Robaldo and Luca Ducceschi, 2013. Phrase Detectives: Utilizing Collective Intelligence for Internet-Scale Language Resource Creation. ACM Transactions on Intelligent Interactive Systems, 3(1). (pdf)
  • Sikdar, U., A. Ekbal, S. Saha, O. Uryupina and M. Poesio, 2013. Adapting a State-of-the- art Anaphora Resolution System for Resource-poor Language. Proc. of IJCNLP, Tokyo, October. (pdf)
  • Uryupina, O., A. Moschitti and M. Poesio, 2012. BART goes multilingual: The UniTN / Essex submission to the CoNLL-2012 Shared Task. Proc. of CONLL Shared Task, Jeju, Korea. (pdf)
  • Uryupina, O. and M. Poesio, 2012. Domain-specific vs. Uniform Modeling for Coreference Resolution. In Proc. of LREC, Istanbul.
  • Uryupina, Olga, Massimo Poesio, Claudio Giuliano, and Kateryna Tymoshenko, 2012. Disambiguation and Filtering Methods in Using Web Knowledge for Coreference Resolution. In C. Boonthum- Denecke , P. M. McCarthy, T. Lamkin (eds.), Cross-Disciplinary Advances in Applied Natural Language Processing: Issues and Approaches. Hershey, IGI Global, pp. 185- 201.
  • Massimo Poesio and Hannes Rieser, 2011. An Incremental Model of Anaphora and Reference Resolution Based on Resource Situations. Dialogue and Discourse, 2(1), 235-277. (pdf)
  • Uryupina, O., S. Saha, A. Ekbal and M. Poesio, 2011. Multi-metric optimization for coreference: The Trento / IITP / Essex Submission to the CONLL Shared Task 2011. In Proc. of CONLL, Portland, OR. (pdf)
  • Iida, R. and M. Poesio. 2011. A Cross-Lingual ILP Solution to Zero Anaphora Resolution. In Proc. of ACL 2011, Portland, OR. (pdf)
  • Marta Recasens, Lluís Màrquez, Emili Sapena, M. Antònia Martí, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. SemEval-2010 Task 1: Coreference Resolution in Multiple Languages. In Proc. of SEMEVAL 2010. (pdf)
  • Kepa Joseba Rodríguez, Francesca Delogu, Yannick Versley, Egon W. Stemle and Massimo Poesio, 2010. Anaphoric Annotation of Wikipedia and Blogs in the Live Memories Corpus. In Proc. of LREC.
  • Nikiforos Karamanis, Massimo Poesio, Chris Mellish and Jon Oberlander, 2009. Evaluating Centering for Information Ordering using Corpora. Computational Linguistics, v. 35, n. 1. (pdf )
  • Yannick Versley, Simone Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang and Alessandro Moschitti, 2008. BART: A Modular toolkit for coreference resolution. In Proc. of LREC, Marrakesh. (pdf)
  • Josef Steinberger, Massimo Poesio, Mijail Kabadjov and Karel Jezek 2007. Two uses of anaphora resolution in summarization. Information Processing and Management, v. 43, n. 6, 1663-1680. Special Issue on Summarization (Donna Harman, ed.). (pdf of preliminary version)
  • Massimo Poesio, Uwe Reyle, and Rosemary Stevenson, 2007. Justified Sloppiness in Anaphoric Reference. In H. Bunt and R. Muskens (eds.), Computing Meaning 3, Kluwer. (pdf)
  • Massimo Poesio, Patrick Sturt, Ron Artstein, and Ruth Filik 2006. Underspecification and anaphora: Theoretical issues and preliminary evidence. Discourse Processes, v. 42, n. 2, 157-175. (pdf)
  • Massimo Poesio, Amrita Patel, and Barbara di Eugenio, 2006. Discourse Structure and Anaphora in Tutorial Dialogues: an Empirical Analysis of two Theories of the Global Focus. Research in Language and Computation. v.4, 229-257. (pdf)
  • Massimo Poesio and Natalia Modjeska, 2005. Focus, Activation, and THIS-Noun Phrases. In A. Branco, T. McEnery and R. Mitkov (eds.), Anaphora Processing, John Benjamins. 429-456. (pdf)
  • Massimo Poesio, Rosemary Stevenson, Barbara di Eugenio, and Janet Hitzeman, 2004. Centering: A Parametric theory and its instantiations. Computational Linguistics, 30(3), p. 309-363. (pdf)
  • Massimo Poesio, 2004. Discourse Annotation and Semantic Annotation in the GNOME Corpus, Proc. of the ACL Workshop on Discourse Annotation, Barcelona, July (pdf).
  • Massimo Poesio, Rahul Mehta, Axel Maroudas and Janet Hitzeman, 2004. Learning to resolve bridging references, Proc. of ACL, Barcelona, July. (pdf)
  • Massimo Poesio, Tomonori Ishikawa, Sabine Schulte im Walde and Renata Vieira, 2002. Acquiring lexical knowledge for anaphora resolution. Proc. of LREC, Las Palmas, May. (ps) (pdf)
  • Poesio, M. and U. Reyle. 2001. Underspecification in anaphoric reference. In Proc. of the Fourth International Workshop on Computational Semantics, Tilburg, January 2001.
  • Pearson, J., Massimo Poesio, and Rosemary Stevenson, 2001. The effects of animacy, thematic role, and surface position on the focusing of entities in discourse. Proc. of the 1st Workshop on Cognitively Plausible Models of Semantic Processing, Edinburgh, July.
  • Massimo Poesio and Malvina Nissim, 2001. Salience and possessive NPs: the effects of animacy and pronominalization. Proc. of AMLAP (Poster).
  • Renata Vieira and Massimo Poesio, 2000. An Empirically-Based System for Processing Definite Descriptions. Computational Linguistics, v. 26, n.4, 539-593. (pdf)
  • Massimo Poesio and Renata Vieira, 1998. A Corpus-based Investigation of Definite Description Use, Computational Linguistics, v. 24, n.2, 183-216. (pdf)
  • Janet Hitzeman and Massimo Poesio, 1998. Long-distance pronominalisation and global focus. Proc. ACL/COLING 98. Montreal, August, 1998. (pdf)
  • Massimo Poesio, Sabine Schulte im Walde and Chris Brew, Lexical clustering and definite description interpretation. Proc. of the AAAI Spring Symposium on Learning for Discourse, Stanford, CA, March, 82--89. AAAI. pdf
  • Massimo Poesio, Renata Vieira, and Simone Teufel, 1997. Resolving Bridging Descriptions in Unrestricted Text. Proc. ACL-97 Workshop on Operational Factors in Practical, Robust, Anaphora Resolution For Unrestricted Texts. ACL Madrid, 7-11 July, pages 1-6, 1997. (.pdf)