The Dynamics of Early Social Interactions

The overarching goal of this research project was to investigate and expand our knowledge of social attention and interaction in early infancy and links to later typical and atypical development. This was done using a variety of methods and samples. First of all, dual eye tracking data was collected from young children and their caregivers, during a series of interactive and playful tasks. Although collecting and analysing dual eye tracking data was the primary goal of the research project, difficulties due to the COVID-19 pandemic led to other research projects being implemented as well. Charlotte worked with eye tracking data collected in the Babytwins Study Sweden (BATSS; Fack-Ytter et al., 2021), which she used to analyse the genetic and environmental influence on early social and cognitive measures. The work was also expanded to infant siblings of children with autism in an effort to compare these infants to typically developing children. This part of the project is linked to the EASE (Early Autism Sweden) study, which is an on-going longitudinal study of infant siblings of children with autism. This has led to publications in leading journals in our field, including first authorships in Journal of Child Psychology and Psychiatry, Developmental Science, Scientific Reports, and Journal of Autism and Developmental Disorders. 

The collaboration of Charlotte Viktorsson from Uppsala University with Irzam Hardiansyah from Karolinska Institute and Stockholm Health Care Services, and Niilo Valtakari from Utrecht University has contributed to integrate a wide range of disciplines from developmental cognitive neuroscience, twin studies, computer vision science and technology to advance our current knowledge regarding the interplay of genetic and environmental factors in shaping infant development, with a particular emphasis on brain, attentional, and behavioural aspects. Charlotte Viktorsson’s work in the Babytwins Study Sweden (BATSS; Falck-Ytter et al., 2021) has been particularly noteworthy. By integrating eye tracking data methodology within a classical twin design, her research revealed that 5-month-old infants (N = 535) preferential looking at different parts of the face represents a highly heritable and independent socio-communicative trait, which was specifically linked to language comprehension at 14 months (Viktorsson et al., 2023). 

In another study based on the BATSS data (N = 514), she found that individual differences in infants’ sense of approximate numerosity is partly due to genetic variation (Viktorsson et al., 2023). In addition to the published papers, there is one manuscript currently in review (Viktorsson et al.) and one manuscript in preparation (Viktorsson et al.). Charlotte Viktorsson has also investigated the potential of using polygenic scores to increase prediction of later outcome in infants and young children. This longitudinal twin design, combined with advanced technologies like eye-tracking and EEG methodologies, as well as genomics, provides a key example of the multidisciplinary efforts and ideas within the SAPIENS network. In addition, Charlotte Viktorsson has identified new ways of assessing social looking atypicalities in toddlers who later receive an autism diagnosis, a result which could have implications for future early detection of autism. In a study of 18-month-olds from EASE (N= 98), she found that children with later autism look less at the most socially informative aspects of the scene when watching other children interact (Viktorsson et al., 2023). This finding may also have an impact on interventions in childhood, suggesting that children with autism might need additional support in social situations involving other children, a common situation in schools, at playgrounds, and at home.

Niilo’s project involved the investigation of the role of gaze behavior to faces during interaction between parents and infants. The project can be split into two distinct parts: 1) building and optimizing a novel dual eye-tracking setup and then 2) applying it to research investigating gaze during parent-infant interaction.

In the first part of the project, Niilo built and piloted a novel dual eye-tracking setup that allowed for parents and infants to interact with each other. The goal of the setup was to investigate how parents and infants look at each other in a setting that represents the face-to-face interactions they partake in daily. While optimizing the setup, Niilo wrote a review paper to discuss the possibilities and limitations of eye tracking in human interaction, as well as an empirical paper to evaluate the performance of computer-vision-based gaze estimation in gaze research concerning adult and infant participants. Both papers were written in collaboration with Charlotte Viktorsson (ESR1) stationed at Uppsala University, where Niilo also spent his secondment at. The considerations from both papers allowed Niilo to further optimize the dual eye-tracking setup, build an analysis pipeline utilizing automated AOI assignment and pose detection, and design relevant experiments investigating gaze to faces in face-to-face social interaction involving both adult and infant participants.

In the second part of the project, Niilo applied the setup in three different experiments. First, Niilo and Charlotte Viktorsson (who was doing her secondment at Utrecht University) conducted a study to investigate how adults look at the face of a speaker while engaged in an explicit speech-processing task. Importantly, the study illustrated how the dual eye-tracking setup Niilo built can be applied to research concerning fine-grained details in how people look at each other’s faces during face-to-face interaction. The results of the study demonstrated that there is considerable stability in the way adults gaze at faces when engaged in a speech-processing task, regardless of whether the language being processed is familiar or unfamiliar.

Following, Niilo conducted two studies in which parents and infants interacted with each other face to face. In the first study, infants observed their parent perform a staged conversation between two hand puppets. With the study, Niilo was able to examine whether previous findings on infant turn following and turn anticipation from studies using screen-based paradigms generalize to face-to-face contexts, and whether parents’ nonverbal behaviors might play a role in the development of infant turn following and turn anticipation. The results showed that infants follow turns when observing conversations in face-to-face contexts, anticipate turns in conversations when their parent also orients their head toward the current speaker, and that older infants are better at utilizing nonverbal turn-related cues than younger infants.

In the second study, Niilo investigated whether previous findings on infant action prediction using screen-based paradigms generalize to face-to-face contexts, and whether the nonverbal behaviors of the parent may be related to infant action prediction. With the results, Niilo showed that infant action prediction in face-to-face interaction with their parent was better described by a bias for looking at the mouth of the parent than by prediction of action outcomes, although some evidence for repetition-based prediction was found. Moreover, parents were found to perform functional (i.e., performed in a way typical to the object, like taking a cup to the mouth) actions ending at the mouth in a more expressive manner than functional actions ending at the mouth or nonfunctional (i.e., performed in a way not typical for the object, like taking a cup to the ear) actions. The results further found that certain parent nonverbal behaviors were related to infants’ anticipatory looks to action target areas, suggesting that specific nonverbal cues provided by parents may guide infants’ gaze to the targets of those actions.

Taken together, Niilo’s work has provided researchers investigating gaze in human interaction, particularly those focused on parent-infant interaction, important considerations regarding the possibilities and limitations of the setups and tools they utilize. Moreover, Niilo’s work has shown that findings from studies using screen-based paradigms may not fully translate to face-to-face contexts. Finally, Niilo has provided future researchers with concrete ideas and suggestions of how to extend the investigation of gaze in parent-infant interaction from controlled paradigms to more free-flowing contexts.

Pamela Villar Gonzalez investigated early social features that may indicate neurodivergent developmental trajectories in children with an elevated likelihood of developing autism. One example from this line of research within SAPIENS ETN is Pamela Villar’s project carried out at the University of Warsaw entitled ‘’Can we predict Language Outcomes from EEG Frequencies?”. For the first study, her work focused on the relationship between infants, caregivers, and their environment in language communication. It was devoted to predicting the ‘amount of language’ and a prospective diagnosis of autism as early as possible. 

In a pilot study, the motor and vocal coordination between the baby and the parent (or caregiver), paying special attention to the movement, (proto)language, gazing, and turn-taking.  Babies and parents (or caregivers) were be monitored with wearables to track the movements and a head camera to register the gazing of both the baby and the parent. The camera will have an incorporated recording of the sounds and words emitted by the parent and the baby.  Because of the COVID-19 pandemic, full data collection could not be completed, so results are still preliminary. 

Due to the collaboration with Birbeck College, already collected data as part of the project BASIS (The British Autism Study of Infant Siblings) https://www.basisnetwork.org/ could be used further to answer the posed questions within this ESR3 package. The most important conclusion was that language is an important variable to consider for a diagnosis, even without other cognitive measures. Especially expressive language was important, with greater differences in what ASD infants can express than in what they can understand.

Eirini Papageorgopoulou joined the University of Warsaw from April till the end of July 2023 and conducted novel analysis to answer the question: how developmental changes in brain function are related to changes in language ability across time in infants who are at an Elevated Likelihood (EL) for Autism Spectrum Disorder.

In this longitudinal study, she aimed to investigate the longitudinal associations between electroencephalography (EEG) and language measures in a sample of infants at EL and Typical Likelihood (TL) for ASD (N = 159), and whether these associations differed in EL and TL infants. Of interest was also to examine whether the groups differed in the trajectories of brain function and language separately. Measures of EEG power in the theta and alpha ranges and language ability were collected at 10, 14, 24 and 36 months. 

Eirini Papageorgopoulou  used latent growth curve modeling (LGCM) for data analyses of the current study (Bollen & Curran, 2006), which is novel in the developmental child literature. Latent growth curve modeling is a flexible approach for modeling linear or nonlinear change, taking into account individual differences in the trajectories of change. Before examining the links between language skills and theta/alpha spectral power over developmental time with this novel method, a series of LGCMs were run to test for the optimal functional form of growth separately for language, theta power and alpha power. 

The results showed that theta power was a significant predictor of developmental change in language, but this association was not moderated by ASD likelihood group. In addition, the EL and TL groups differed in the slope of language, with the EL group showing less steep increases in language compared to the TL group. Overall, the findings suggest theta power as a candidate biomarker of general language ability, not specific to ASD likelihood and place language as a possible endophenotype of ASD. A journal article will be prepared for this study and future research directions are considered based on the present findings.

The main objective of this project was to monitor online infants’ neural oscillations and study its implications for learning and imitation. As a first step, Maria performed a study with adults that examined whether entraining of the brain to a theta frequency immediately before encoding influenced memory formation. Theta oscillations in the brain play a crucial role in memory encoding, a process essential for long-term memory formation. Research indicates that higher levels of naturally occurring theta power improve memory performance in both adults and children. Moreover, stimulating theta oscillations through sensory entrainment during the encoding phase has been demonstrated to enhance memory formation. 

Thirty adults were tested using EEG, and they watched a flicker followed by a picture of a cartoon animal in an environment. The flicker’s luminosity was modulated with a theta frequency of 5Hz in half of the trials and at random, i.e. at no fixed frequency, in the rest. After 15 minutes, the participants’ memory of the animals’ environment was tested. It was found that participants’ power at 5Hz was greater for the theta flicker than for the random flicker, and memory was better for the pictures preceded by the theta flicker than for those preceded by the random flicker. This work shed light on the neural mechanisms underlying an optimal theta learning state, and demonstrated the potential of investigating cognitive processes linked to this state in real-time analysis’ experiments. 

Based on these promising results, the next study with 4-year-old children was planned and performed. It investigated if inducing theta oscillations via sensory entrainment prior to memory encoding can facilitate memory formation in 4-year-old children. For this research, fifty-three children were tested in a very comparable experimental setup. Results indicate that there was no significant difference in memory performance or theta power between the two conditions (theta versus random flicker). These findings suggest we were unable to induce sensory entrainment, resulting in no improvement in memory performance. Furthermore, there was no correlation between the magnitude of theta power and memory performance, showing no natural variation in sensory entrainment among children. These contrasting findings for adults and children demonstrate a need for further research to identify alternative factors that influence sensory entrainment and memory encoding. A second goal of Maria's project was the improvement of EEG equipment to allow for online as well as more mobile measurements. This work was mainly performed by Robert Oostenveld and also this goal was attained.