I have been interested in the use of commonsense knowledge in support of natural language processing ever since my undergraduate work on Sowa's Conceptual Graphs and then my work on the TRAINS project, but until I started working with Renata Vieira on large scale anaphora resolution, and in particular on bridging references, I had only worked with hand-coded knowledge bases.
So-called bridging references are a class of anaphoric expressions whose interpretation often requires lexical and commonsense knowledge. For instance, in the following example from the GNOME corpus (reported by (Poesio et al, 2004)), `the handles' in 1b. is a bridging reference to the `egg vases' in 1a whose interpretation requires knowing that vases may have handles.
1a. These `egg vases' are of exceptional quality.
1b. Basketwork bases support egg-shaped bodies, and bundles of straw form the handles
Together with my former student Renata Vieira we first of all produced a classification of bridging references, and tested whether lexical sources such as WordNet provided sufficient information, with mediocre results ( Poesio, Vieira, and Teufel, 1997; Vieira & Poesio, 2000 ). In subsequent work, we started investigating whether the necessary lexical and commonsense knowledge could be acquired in a psychologically plausible way from corpora, using so-called `semantic space' or `distributional' models (Deerwester et al, 1990; Lund and Burgess, 1995; Schuetze, 1996; Landauer et al, 1998). In these models, the contexts in which a word occurs are used to build a vectorial representation of that word that encodes co-occurrence information. In our initial work (Poesio, Schulte im Walde, and Brew, 1998) we used a variant of the Hyperspace Analogue to Language (HAL) model (Lund and Burgess, 1995) to automatically extract lexical models that allowed us to predict synonymy with an accuracy similar to that obtained with WordNet. In subsequent work (Poesio, Ishikawa, Schulte in Walde, and Vieira, 2002; Poesio et al, 2004) we started using the semi-supervised relation extraction methods pioneered by Marti Hearst (1998) to extract automatically from corpora information about parts of objects (like that between vases and handles) that we could use to resolve associative bridging references involving mereological knowledge, like that shown in 1b.
The leading distributional models all extract either purely collocational information, or collocational information mediated by syntactic information (Grefenstette, 1994; Lin, 1998; Curran and Moens, 2002). Our work on semi-supervised extraction of mereological relations led us to wonder whether semantic space models in which the vectors encoded relational information would also achieve better performance than traditional distributional models at the tasks in which such models are usually evaluated (e.g., clustering).
I explored this question in work with my former PhD student Abdulrahman Almuhareb. We first showed that semi-supervised techniques for the extraction of attributes and values using only very simple patterns over words achieved higher performance that purely collocational models, with much more compact vectors (Almuhareb and Poesio, 2004). We then evaluated using a dependency parser to support attribute extraction, finding that this didn't lead to improved results, but much smaller corpora were needed (methods based on word patterns require Web-site corpora, which is cognitively implausible) (Almuhareb and Poesio, 2005a; Almuhareb and Poesio, 2005b) Finally, we developed a supervised approach, in which supervised methods like those used in relation extraction were used to extract from corpora relations that we identified as being typical of conceptual attributes, on the basis of a classification of attributes derived in part from Pustejovsky's Qualia theory (Pustejovsky, 2005), in part from Guarino's theory of attributes (Guarino, 2000) (Almuhareb and Poesio, 2005c). This research on attributes and values is summarized in ( Poesio and Almuhareb, 2008). This work was subsequently continued as a collaboration between myself, my then PhD student Eduard Barbu, and Claudio Giuliano and Lorenza Romano from Fondazione Bruno Kessler in Trento. In this work, the classification schema for attributes in feature norms from Wu and Barsalou (2009) was used (the same schema used by McRae et al, 2005 to classify their feature norms), as well as state-of-the art supervised relation extraction methods based on SVMs. ( Poesio et al, 2008)
Also with Abdulrahman, I developed one of the first methods for word-sense discrimination within semantic space models. The MSDA algorithm ( Almuhareb and Poesio, 2006) borrows ideas from Schuetze's and Pedersen's work, but makes crucial use of the attributes of concepts to discriminate between senses, instead of all collocations.
In addition to the supervised methods, we also explored the use of semi-supervised and unsupervised methods. In collaboration with Marco Baroni, Eduard Barbu, and Brian Murphy, we developed the STRUDEL model for the acquisition of distributional models, whose technique of counting patterns instead of occurrences has proven very effective' ( Baroni et al, 2010). With Eduard Barbu, we focused on the question of whether extracting from Wikipedia instead of from the Web or other corpora would improve the quality of the extracted ontology ( Barbu and Poesio, 2009).