Connectionist models of sentence production

Fitz, H. (2009). Neural Syntax. Amsterdam. ILLC publication series.

Hartmut Fitz has completed his dissertation on the Recursive Dual-path model. It addresses recursion in neural networks, poverty of the stimulus, systematicity in neural networks, messages with multiple propositions, learning of polar interrogatives, and the accessibility hierarchy for relative clause constructions.

Chang, F. (2009). Learning to order words: A connectionist model of heavy NP shift and accessibility effects in Japanese and English. Journal of Memory and Language, 61, 374-397.

Languages differ from one another and must therefore be learned. Processing biases in

word order can also differ across languages. For example, heavy noun phrases tend to be shifted

to late sentence positions in English, but to early positions in Japanese. Although these language

differences suggest a role for learning, most accounts of these biases have focused on processing

factors. This paper presents a learning-based account of these word order biases in the form of a

connectionist model of syntax acquisition that can learn the distinct grammatical properties of

English and Japanese while, at the same time, accounting for the cross-linguistic variability in

processing biases in sentence production. This account demonstrates that the incremental nature

of sentence processing can have an important effect on the representations that are learned in

different languages.

Chang, F., Dell, G. S., & Bock, K. (2006). Becoming syntactic. Psychological Review, 113, 2, 234-272

Psycholinguistic research has shown that the influence of abstract syntactic knowledge on performance is shaped by particular sentences that have been experienced. To explore this idea, the authors applied a connectionist model of sentence production to the development and use of abstract syntax. The model makes use of (a) error-based learning to acquire and adapt sequencing mechanisms and (b) meaning–form mappings to derive syntactic representations. The model is able to account for most of what is known about structural priming in adult speakers, as well as key findings in preferential looking and elicited production studies of language acquisition. The model suggests how abstract knowledge and concrete experience are balanced in the development and use of syntax.

Chang, F. (2002) Symbolically speaking: A connectionist model of sentence production. Cognitive Science, 26(5), 609-651.

The ability to combine words into novel sentences has been used to argue that humans have symbolic language production abilities. Critiques of connectionist models of language often center on the inability of these models to generalize symbolically (Fodor & Pylyshyn, 1988; Marcus, 1998). To address these issues, a connectionist model of sentence production was developed. The model had variables (role-concept bindings) that were inspired by spatial representations ( Landau & Jackendoff, 1993). In order to take advantage of these variables, a novel dual-pathway architecture with event semantics is proposed and shown to be better at symbolic generalization than several variants. This architecture has one pathway for mapping message content to words and a separate pathway that enforces sequencing constraints. Analysis of the model’s hidden units demonstrated that the model learned different types of information in each pathway, and that the model’s compositional behavior arose from the combination of these two pathways. The model’s ability to balance symbolic and statistical behavior in syntax acquisition and to model aphasic double dissociations provided independent support for the dual-pathway architecture.

Here is another figure for the model