SYMPOSIUM

The abstract or concrete nature of inner language is a controversial topic. Behavioural, psychophysical and neuroimaging data suggest that some inner speech episodes are phonologically expanded and have multimodal sensory properties. This has led to the embodied view that inner speech is a simulation of overt speech production, encompassing conceptualisation, formulation, phonological encoding and articulatory planning, only interrupted prior to motor execution. Other behavioural findings support the alternative view that some inner language occurrences are abstract or amodal: they are elliptical and condensed at the phonological, lexical and syntactic levels. According to some scholars, inner speech should therefore be conceived of as a compact pre-linguistic form, a verbal sketch, cut short before phonological encoding and articulatory planning. As argued in Grandchamp et al. (2019), these two views are not irreconcilable, however. They correspond to two extreme poles on the condensation dimension of inner language: expanded concrete vs condensed abstract forms. Variation in form may stem from situation (cognitive demands, emotional contexts). Recent studies on mental imagery suggest that there may also be some interindividual variability in the vividness of multisensory representations (from hyperphantasia to aphantasia) associated with various degrees of inner language condensation. The ConDialInt Model, a neurocognitive sketch of inner language, cast within a predictive control framework, accounts for such variation by positing activation/inhibition of internal models.

When people are asked to reflect on their conscious experience, they often report that it contains a fair amount of language: the everyday internal conversation that psychologists term inner speech. My interest as a developmental psychologist is in where these words in the head come from, what they are doing there, and what it is like to experience them. Empirical studies of self-directed speech point to it having important cognitive functions. Improved methodologies for studying these phenomena in children and adults support Vygotskian / Lurian conceptions of inner speech as constituting a functional system, whereby initially independent neural systems are ‘wired together’ in new ways by social experience. I present some recent findings relevant to this account, and consider prospects for a cognitive neuroscience of inner speech that is sensitive to its development and phenomenology.

A robust and growing body of evidence implicating the language system in various aspects of semantic memory – including its apparent contribution to some of our abstract concepts – has generated a great deal of interest in the idea that language might enhance grounded cognition. Many of the current versions of this idea treat the language system as fundamentally separate from, and independent of, the rest of the conceptual system. Essentially, these proposals treat language as if it were not itself a grounded phenomenon. In this talk, I develop and defend an unapologetically grounded account of the role played by language within a flexible, multimodal, and multilevel view of conceptual knowledge. This account provides a means of integrating embodied and distributional accounts of semantic memory and a framework for evaluating neuropsychological evidence.

What qualifies one concept as more abstract than another? I will review several competing answers in the literature and consider the prospects uniting them around a shared understanding. I will then explore potential confounds in working with that understanding when attempting to assess the relation of abstract conceptual abilities to language. Some of these problems derive from the fact that ratings of abstractness are usually tied to specific words, while nonverbal stimuli are often preferable when attempting to determine the dependence of abstract concept use on language. I then will recommend a context-relative (and task-relative) understanding of abstractness that avoids those pitfalls while remaining true to what researchers should want in an assessment of abstract conceptual abilities. This measure still allows abstractness to come in degrees.

The development of children’s word knowledge is an important testing ground for theories of abstract concepts. Abstract words tend to be acquired later than concrete words, with abstract vocabulary acquisition growth in the school years. I will describe our recent empirical work examining children’s acquisition and processing of abstract and concrete vocabulary. In this work, we have tested two proposals about how children acquire abstract vocabulary: 1) the emotion bootstrapping or affective embodiment account; and 2) the language competence hypothesis. Our results provide some support for both proposals. For instance, we found evidence that emotion information may be particularly important to children’s representations of abstract word meanings around ages 6-8 years. Further, around the same ages, we found that children’s language skills are related to their processing of abstract words. I will describe how the interplay of these factors across development might help characterize abstract meaning representation.

It is tempting to assume that abstract words like, “same”, “fun”, and “idea” map onto concepts that we have independently of learning those words. I will challenge this assumption on logical and empirical grounds, and present evidence for the causal role of language in the development of abstract concepts. I will then speak to what it is about language that helps to promote the development of abstract knowledge, and present new evidence that early knowledge of certain “seed” words promotes subsequent word learning and performance on a (putatively) nonverbal cognitive task. I will end by outlining the kinds of data that would be of enormous help in understanding the development of abstract concepts, but which is presently lacking.

Human concepts are naturally embodied--concrete and contextually situated. Whether we also form abstract concepts has been controversial. I will argue, first, that we do form abstract concepts; and, second, that do so via analogical comparison. Consistent with this claim, many of our abstract concepts are relational concepts—concepts whose membership is determined not by common intrinsic properties, but by common relational patterns, such as betrayal or carnivore. Finally, my third claim is that language is critical to forming, retaining and reusing abstract concepts. I’ll present evidence from children and adults to support these ideas.

We present a theory of dialogue as a form of cooperative joint activity. Dialogue is treated as a system involving two interlocutors and a shared workspace that contains their contributions and relevant non-linguistic context. The interlocutors construct shared plans and use them to “post” contributions to the workspace, to comprehend joint contributions, and to distribute control of the dialogue between them. A fundamental part of this process is to simulate their partner’s contributions and to use it to predict the upcoming state of the shared workspace. As a consequence, they align their linguistic representations and their representations of the situation and of the “games” underlying successful communication. The shared workspace is a highly limited resource, and the interlocutors use their aligned representations to say just enough and to speak in good time. We end by applying the account beyond the “minimal dyad” to augmented dialogue, multi-party dialogue, and monologue.

based on the new book:

Pickering, M.J., & Garrod, S. (2021). Understanding dialogue: Language use and social interaction. Cambridge: Cambridge University Press.

http://services.cambridge.org/us/academic/subjects/languages-linguistics/psycholinguistics-and-neurolinguistics/understanding-dialogue-language-use-and-social-interaction?format=HB&isbn=9781108473613

We have used inter-subject correlation (ISC) of fMRI signals to measure similarity of brain activity or connectivity in brain of subjects. ISC methods makes it possible to study how brain processes rich naturalistic stimulation, like viewing a film or listening to an auditory narrative. ISC is a data-driven method, where brain activity of one subject is used to predict brain activity of other subjects, thus there is no need a priori stimulus model. However, good understanding of the stimulus, how it is processed and what kind of experiences it generates is certainly crucial in interpreting the results.

ISC is higher in subjects who listen to a narrative from the same point of view 1 and in those subjects who interpret the story in a more similar way 2,3 . In our recent study, we showed that shared family cultural background enhanced similarity of narrative processing in the brain at prelexical, word, sentence, and narrative levels. Strength of social identity shaped such processing 4 . These among other results demonstrate that at least some aspects of mutual understanding are reflected in similarity of brain activity.

We recently scanned brain activity of a subject telling (“teller”) an emotional story in the scanner 5. We recorded the story and played it back to “listener” subjects. Both tellers and listeners rated the moment-to-moment valence and arousal of the stories. Telling and listening to the stories elicited similar emotions across speaker–listener pairs. Arousal was associated with increased speaker–listener neural synchronization in brain regions supporting attentional, auditory, somatosensory, and motor processing. Valence was associated with increased speaker–listener neural synchronization in brain regions involved in emotional processing, including amygdala, hippocampus, and temporal pole.

We recently examined the emotional and psychophysiological underpinnings of social interaction in the context of autism spectrum disorder, more specifically, involving participants diagnosed with Asperger syndrome (AS). We recorded participants’ autonomic nervous system (ANS) activation (electrodermal activity, heart rate, and heart rate variability) and facial muscle activation during conversations in two different types of male dyads: (1) ten dyads where one participant has been diagnosed with AS (AS/NT dyads) and (2) nine dyads where both participants are neurotypical (NT/NT dyads). We analyzed the conversations in this experiment to find out if there are quantitative differences in how AS and neurotypical subjects use lexical categories. Our preliminary analyses suggest that there are differences, the most conspicuous being that AS vs neurotypical subjects use more unique words.

References

1. Cooper, E. A., Hasson, U. & Small, S. L. Interpretation-mediated changes in neural activity during language comprehension. NeuroImage (2011), 55, 1314-1323.

2. Nguyen, M., Vanderwal, T. & Hasson, U. Shared understanding of narratives is correlated with shared neural responses. NeuroImage (2019) 184, 161–170.

3. Saalasti, S. et al. Inferior parietal lobule and early visual areas support elicitation of individualized meanings during narrative listening. Brain and Behaviour (2019), 9, e01288.

4. Hakonen, M. et al. Processing of a spoken narrative in the human brain is shaped by family cultural background. Submitted (2020).

5. Smirnov, D. et al. Emotions amplify speaker-listener neural alignment. Human Brain Mapping (2019),40, 4777–4788.

6. Stevanovic, M., Henttonen, P., Koskinen, E., Peräkylä, A., Nieminen-Von Wendt, T., Sihvola, E., et al. (2019). Physiological responses to affiliation during conversation: Comparing neurotypical males and males with Asperger syndrome. PLoS ONE (2011), 14, e0222084.

Decontextualized talk, i.e. talk beyond the immediate context, has been assumed to be only used rarely with children younger than 30 months of age. In our studies, however, we were able to show that this special kind of input is present already in talk to 12-month- old children. In my presentation, I will present some dimensions along which we can categorize different forms of decontextualized talk. I propose that this input scaffolds children’s understanding of talk about the “there-and-then” from early on. I discuss potentials of this kind of input to foster the development of abstract concepts.

When predicting and explaining behavior of other humans, we often adopt the intentional stance, and refer to their mental states in order to understand their actions. However, it is not clear whether and when we adopt the intentional stance also towards artificial agents, such as humanoid robots. This talk will provide an overview of our research addressing this question. I will present a tool that we developed for measuring adoption of the intentional stance, and findings showing that the likelihood of adopting the intentional stance is coded in specific patterns of neural activity at rest. Then, I will present studies which suggest that interactive scenarios influence adoption of the intentional stance more than mere observation of subtle human-like characteristics of a robot’s behavior. The talk will conclude with the discussion on the role of intentional stance for other mechanisms of social cognition, and implications for applied domains of social robotics in healthcare.

The key problem for the explanation of the understanding and use of abstract concepts and words in contemporary cognitive theory is posed as the problem of how they are grounded in the sensory experience and in language. Such a way to pose it is perhaps due to the legacy of the information processing approach, which treats symbols as a given. But what if we do not assume the existence of symbolic entities but rather look at the process of their emergence? Recognizing as a key challenge the insight into the very nature of symbols and how they may arise in the embodied and situated social cognition might enrich the theory of abstraction. It may bring us closer to thinking about abstraction in terms of human experience and practices: which values realized in human interactions are preserved and enhanced by abstraction and which are endangered? What kind of grounding remains in the symbolization process and – most importantly – how can the (partial) ungrounding happen? I briefly hint at a possible theoretical model of this process and illustrate it with language development data. The crucial aspect of the model is the necessary complementarity: both replicable structures and dynamical processes are needed to account for the transition from signaling to symbolization.

It has been a dogma in linguistics and the cognitive sciences that language is primarily arbitrary, with word forms not resembling their meanings. However, much recent work shows that iconicity, the resemblance between a word’s form and its meaning, is a dominant property of all human languages, spoken or signed. Moreover, iconicity has been shown to confer learning advantages, such as evidenced by the fact that young children use onomatopoeic words very frequently. Given the advantageous nature of iconicity, it is perhaps surprising that arbitrariness is so abundant, and that iconicity is rather limited. So, why aren’t the vocabularies of languages more iconic? After reviewing the existing evidence for iconicity and its functions in language acquisition, I present a series of arguments and empirical data suggesting that one of the limiting factors is the abstractness of much of everyday talk. Iconicity is particularly prominent for concrete words, and further evidence suggests that iconicity actively ties word forms to concrete representations, thus limiting their extension to abstract domains. Finally, I additionally review evidence suggesting that the acquisitional advantages of iconicity may be overstated.

This presentation will explore two distinct notions of abstraction: one that has its origins in psychology and philosophy and another that is derived from a consideration of cortical anatomy and information processing. Although they are often confused with one another, they actually represent opposite extremes in terms of how they are generated. Abstraction in logical and psychological terms (which I will call semiotic abstraction) is semiotically complex and constructed by a process we (with Joanna Rączaszek-Leonardi) describe as "ungrounding" because it involves progressively displacing symbols from their iconic and indexical grounding. In contrast, abstraction in a neurological sense (which I will call proto-abstraction) can more accurately be understood as an undifferentiated early stage of perceptual and action schema development. In this neurological sense abstraction is a minimally differentiated neurosemiotic process. Interestingly, the development of cognitively abstract symbolic forms may have provided humans with an unprecedented ability to represent and manipulate this undifferentiated proto-abstract mental content.

From the perspective of the situated conceptualization framework, the primary purpose of concepts is for categorizing and integrating elements of situations to support goal-directed action (including communication and social interaction). To the extent that important situational elements are categorized and integrated properly, effective goal-directed action follows. Over time, frequent patterns of co-occurring concepts within situations become established in memory as situated conceptualizations, conditioning the conceptual system and producing habitual patterns of conceptual processing. As a consequence, individual concepts are most basically represented within patterns of concepts that become entrained with specific kinds of physical situations. In this framework, the concrete versus abstract distinction between concepts is no longer useful, with two other distinctions becoming important instead: (i) external versus internal situational elements,(ii) situational elements versus situational integrations. Whereas concepts for situational elements originate in distributed neural networks that provide continual feeds about components of situations, concepts for situational integrations originate in association areas that establish temporal co-occurrence relations between situational elements, both external and internal. We propose that studying concepts in the context of situated action is necessary for establishing complete accounts of them, and that continuing to study concepts in isolation is likely to provide relatively incomplete and distorted accounts.

http://barsaloulab.org/Online_Articles/2018-Barsalou_et_al-PTRSB-beyond_concrete_abstract.pdf

http://barsaloulab.org/Online_Articles/2020-Barsalou-Jour_Cognition-challenges_opportunities.pdf

http://barsaloulab.org/Online_Articles/2013-Wilson_Mendenhall_et_al-JCN-abstract_concepts.pdf

Sensorimotor information is critical to grounding abstract concepts but its details are not well understood. I will discuss how sensorimotor grounding must be regarded as going beyond the basic five senses, and incorporate a multidimensional variety of perceptual experience from a range of distinct modalities and action experience from a range of distinct bodily effectors. Using large-scale data we collected from this multidimensional perspective, we find that head actions (i.e., actions of the face and head, but not the mouth) provide a vital source of grounding to all types of abstract concepts, but reliance on other sensorimotor dimensions varies. Interoception (i.e., sensations inside the body) is also essential to grounding abstract concepts that are internally-focused, such as emotions, personal qualities, injuries, and psychological illnesses. By contrast, vision is important to grounding abstract concepts that are externally focused, such as sciences, supernatural beings, seasons, and prime numbers; whereas externally-focused concepts that are social in nature, such as family relationships, professions, religions, and crimes, additionally rely on hearing. Finally, using modelling of a full-size adult conceptual system of approx 40,000 concepts, we find that the latent structure in sensorimotor experience produces a taxonomic hierarchy in abstract concepts as deep as that of concrete concepts. Abstract categories that spontaneously emerge from sensorimotor structure include emotions, moral and affective judgements, professions and expertise, and maths & analytics. Overall, these findings suggest that the distinction between abstract and concrete concepts is not as clearcut as ontological assumptions might suggest, and that abstract concepts rely just as firmly on sensorimotor grounding as concrete concepts have long been known to do.

Humans are poorer at identifying smells and communicating about them, compared to other sensory domains. They also cannot easily organize odor sensations in a general conceptual space, similar for instance to colour space, where geometric distance could represent how similar or different all odors are. Shall we grant exceptions to this claim based on experts and cross-cultural categorization studies, which show that training and culture are sufficient to master abstract olfactory concepts? Contrary to the exceptionalist move, I will argue that categorisation can occur locally - and carves out sensory domains in different manners when it comes to colours and odours.

It is widely assumed that thought processes involving conceptual representations draw on the content of the constituent concepts. The Representational Theory of Mind is our best account of how those processes work. But it is unclear whether its strictures will allow us to vindicate the assumption that thought processes draw on the content of concepts. Recent developments in the psychological and philosophical understanding of concepts make the challenge particularly pressing.

Here we show how some but not all transitions between conceptual representations draw on the content of the concepts involved. These ‘content-specific’ transitions work in two ways: via non-conceptual resources or between conceptual representations directly. They contrast with broadly logical transitions, which do not draw on the content of open class concepts. A surprising conclusion is that the formerly paradigmatic case, where a thinker reasons from explicit premises recalled from memory, turns out not to be a case where processing draws on the content of open class concepts.

Interoception is the sense of the physiological condition of the body.  It has long been known that interoception is key for our survival, as it supports the homeostatic regulation of drive states (e.g., hunger, thirst, pain) and somatically relevant states (e.g., effort and fatigue levels). More recently, several studies reveal that interoception is also crucial to higher-order cognition, such as emotional experience, decision making and consciousness; and furthermore, that a dysfunctional interoception may produce psychopathological conditions.

In my talk, I will introduce a predictive coding account of brain function and emotions, and present recent evidence showing that the state of our body affects how we categorize our own emotions and perceive those of others. In the final section I will discuss the hypothesis that an excessive abstraction - or disembodiment - of emotions might underlie some interceptive dysfunctions.

Learning and using abstract words like idea or freedom are essential to success in life as most societal, cultural and scientific endeavors require grasping and manipulating abstract ideas. Most current psychological theories (Paivio, 2007; Schwanenflugel et al., 1991; Gleitman et al., 2005; Lakoff & Johnson, 1980) suggest that the ability to learn abstract concepts depends on children’s linguistic and conceptual development.

However, our recent work suggests that emotion can provide a bootstrapping mechanism for learning abstract words: emotional valence reliably predicts age of acquisition (AoA) norms for abstract concepts (Ponari et al., 2017);and abstract words that are emotionally valenced are both processed (Ponari et al., 2017) and learnt (Ponari et al., 2020) better compared to neutral abstract words.

The above results converge to suggest that emotional valence has a crucial role in abstract language acquisition. The hypothesis is that children can support learning words that are abstract (hence lacking a concrete referent in the environment) by associating them to the feelings evoked by the situation and the facial expression of the caregiver. Thus, in general, the child’s appreciation of the emotional valence of a word would support the inference that the word refers primarily to an internal, rather than external state (Kousta et al., 2011; also see Barsalou et al., 2018).

We further assessed this hypothesis by testing abstract words’ knowledge in children with Autism Spectrum Disorder, with (ALI) and without (ALN) associated language impairment. If the social/emotional skills assumed to be impaired in ASD are particularly critical to learning abstract words, we would expect disproportionate deficits in abstract knowledge, as measured by a lexical decision and a definitions task. To address the variation in language ability within the ASD group, we compared the ASD children to both TD children and children with developmental language disorder (DLD). Contrary to our hypotheses, linear mixed models, complemented by single-case analyses, showed no disproportionate disadvantage for abstract words in the ASD samples in both tasks. However, qualitative analyses of the definitions revealed marked differences in defining abstract concepts: both ASD groups usesignificantly less emotional descriptions and descriptions of social interactions compared to TD and DLD children.

I will present the conjecture that metacognitive experiences can be used to ground the meaning of abstract concepts. Although this possibility has been suggested before, no process model has been proposed to clarify how concepts can be grounded in this way, how they might be learned, and how they can be expressed in on line processing. In my talk, I will present a neurocomputational model of abstract categorization in which metacognitive signals are used to represent an abstract category starting from concrete sensorimotor experiences. To exemplify this approach I will focus on ownership concepts like "mine" and "yours" and how they are grounded on the controllability of objects in a social context and on the metacognitive maps of competence that enables it.