Meetings

BERT-based Annotation of Oral Texts Elicited via Multilingual Assessment Instrument for Narratives

Abstract:

We investigate how NLP can help annotate the structure and complexity of oral narrative texts elicited via the Multilingual Assessment Instrument for Narratives (MAIN). MAIN is a theory-based tool designed to evaluate the narrative abilities of children who are learning one or more languages from birth or early in their development. It provides a standardized way to measure how well children can comprehend and produce stories across different languages and referential norms for children between 3 and 12 years old. MAIN has been adapted to over ninety languages and is used in over 65 countries. The MAIN analysis focuses on story structure and story complexity which are typically evaluated manually based on scoring sheets. We here investigate the automation of this process using BERT-based classification which already yields promising results.

Literacy Acquisition at School: German, French and Portuguese in Luxembourg. Presentation of the MAIN stories written in three languages by Grade 6 students 

Abstract:

The present study investigates the trilingual literacy acquisition at the end of primary school (grades 5-6) of students with Portuguese as a heritage language. This group of students learns to read and write in German (Grade 1) and French (Grade 2) as well as in Portuguese. While German and French are school languages, the Portuguese heritage language is taught 2 hours per week outside of school hours.  

The overall aim of the study is to explore the multilingual language and literacy acquisition of primary school students in Luxembourg. One research question is the influence of Portuguese writing on the written production of the school languages German and French. Another research question is the influence of the heritage language course on Portuguese writing. Overall, the study aims to identify the writing profiles of multilingual learners in a multilingual school context.  

The full dataset consists of 273 students from 27 classes with different language backgrounds. Three of the MAIN picture stories (dog, cat, butterfly) were used in the telling mode to elicit narratives in German, French, and the family language. In addition, standardized tests were used to assess spelling with a dictation in German and French. Only the students who attended heritage language classes (n = 65) were additionally assessed in reading (ELFE), vocabulary (PPVT-5), and sentence repetition tasks in the three languages, as well as in Portuguese spelling. 

Data collection ended only last June, and data processing has not yet been finished. Therefore, the presentation will present the theoretical framework and the methodological design of the study, as well as an example of the written production of the MAIN stories in the three languages. 

MAIN Image Sequences for Adults with Aphasia: Experience from the Aphasia in South Asian Languages Project (ASAL-Project)

Abstract:

Speech and language characteristics of connected speech provide a valuable tool for identifying, diagnosing and monitoring language impairments in neurological impairments (e.g., post-stroke aphasia, dementias). In English speaking populations, there is availability of a range of materials, including picture descriptions, story retelling, image sequences. However, in bilingual and multilingual adult populations, there is dearth of tasks which can consistently elicit high quality data for connected speech analyses across languages. With the ongoing development for MAIN for children, we were motivated to include two of the image sequences for our Aphasia in South Asian Language Project (ASAL). The aim of the ASAL project is to characterize language production impairments in post-stroke aphasia across different discourse genre in five Indian languages (i.e., two Indo-Aryan, Hindi, Bengali; three Dravidian, Tamil, Malayalam, Kannada). In this presentation, we would discuss our rationale for including these images in context of the other connected speech tasks, our experience in implementing them for adults speaking these languages, our planned linguistic analyses and some preliminary data across few languages. We look forward to discussing our data and receiving feedback for our ongoing research using the MAIN images. 

Narrative skills in Ukrainian-Russian bilingual children: The role of bilingualism and parental input

Abstract:

Development of narrative skills is a continuous process that extends well into the school years and beyond (Lindgren 2018; 2022; Govindarajan & Paradis 2019). While there are various factors that influence the acquisition of narrative skills, previous research emphasizes significant impact of parental input on children's narrative development (Armon-Lotem et al., 2011; Rowe, 2013; Yang et al., 2023). In bilingual language acquisition, a large number of studies showed that bilingual children perform similarly in both languages in terms of story structure (e.g., Bohnacker, 2016; Kunnari et al., 2016; Bohnacker et al., 2022; Fiani et al., 2022; Fichman et al., 2022; Lindgren & Bohnacker, 2022) and story complexity (e.g., Gagarina 2016; Kunnari et al., 2016; Lindgren, 2018; Bohnacker et al., 2022; Lindgren et al., 2022). Although, some studies also found differences between the languages (see Lindgren et al., 2023 for an overview).

Furthermore, while most research focuses on children, there are a few studies that tested solely adults (e.g., Gagarina et al., 2019a; Antonijevic et al., 2022) or included them as a control group (e.g., Hržica & Kuvač Kraljević, 2022; Vogels & Lindgren, 2022) using MAIN. The studies have shown that adults generally score higher in macrostructure than children. Despite these findings, to our knowledge, there are no studies investigating the relationship between parental macrostructure and their children’s macrostructure using MAIN in a bilingual context.

In the talk, we investigate narrative skills of Ukrainian-Russian bilingual children and their parents who arrived to Germany in 2022 as a result of the Russian aggression. In Ukraine almost everyone is bilingual, at least to a certain degree (Shevchuk-Kliuzheva, 2020; Sokolova, 2021). Hence, the investigated population in our study does not have a common dominant language: for some it is Ukrainian, for some it is Russian.

The data for the study were collected from Ukrainian-Russian bilingual 6-8-year-old children (n=25) and their parents (n=19), using the Multilingual Assessment Instrument for Narratives (MAIN) (Gagarina et al., 2012). Picture sequences, provided in MAIN, were used to elicit four narratives per each participant: two in Ukrainian and two in Russian. The narratives were transcribed and assessed for macrostructure based on story structure and story complexity using the Russian and Ukrainian scoring MAIN guidelines (Gagarina et al., 2019b; 2019c).

Building upon prior research, we ask two research questions:

1. Do children perform similarly in terms of story structure and story complexity when narrating in Ukrainian and Russian?

2. Assuming that narrative skills of parents can provide insights into the quality of parental input, we ask whether there is a correlation between the narrative skills of children and their parents, in terms of story structure and story complexity. 

Subordination in Turkish Heritage Children with and without Developmental Language Impairment

Abstract:

Recent research (Arosio et al., 2011; Hamann et al., 2007; Friedmann and Novogrodsky, 2011; Scheidnes and Tuller 2019 among others) has shown that complex constructions such as subordinate constructions are cross-linguistically vulnerable in bilingual DLD children. Therefore, they are identified as a potential clinical marker for identifying DLD in bilingual contexts, especially when the majority language is assessed. It is unclear, nonetheless, if this also holds true in heritage contexts, especially when L2-contact-induced phenomena have an influence on the heritage language, as is the situation with heritage Turkish in Germany. This study compares 13 Turkish heritage children with and without DLD (age range 5; 1–11; 6) to 10 late successive (lL2) BiTDs (age range 7; 2–12; 2) and 10 Turkish adult heritage bilinguals (age range 20; 3–25; 10) by analyzing subordinate constructions using both Standard and Heritage Turkish as reference varieties. Furthermore, the study investigates which background variables as predictors explain performance in subordinate constructions. The sentence repetition task (SRT) from the TODİL (Topbaş and Güven 2017) standardized test battery and the Multilingual Assessment Instrument for Narratives (MAIN; Gagarina et al. 2012) are used to elicit speech samples. The systematic analysis of a corpus of subordinate clauses constructed with respect to SRT, MAIN narrative production task, and MAIN narrative comprehension task designates that heritage children with TD and DLD may not be differentiated through these tasks, especially when their utterances are scored taking the Standard Turkish variety as a baseline. However, they may be differentiated if the Heritage Turkish as a baseline considered. The age of onset in the second language (AoO_L2) is the leading factor explaining performance difference subordinate clause production in SRT and in both tasks of MAIN disregarding using Standard Turkish or Heritage Turkish as reference varieties in scoring. The present study reports a similar overlap in DLD between Turkish and heritage Turkish to previous studies (Chilla and Şan 2017; Şan 2018; Chilla 2022). Therefore, a systematic analysis of the varieties of Standard Turkish is necessary to prevent bilingual children from being misdiagnosed in clinical practice (Herkenrath and Rehbein 2012; Chilla 2022). A linguistically controlled task such as the LITMUS-SRT in combination with the LITMUS-NWRT for the assessment of language impairment is recommended to prevent misdiagnosis in heritage bilingual children (Hamann and Abed-Ibrahim 2017; Hertel et al. 2022). Furthermore, the fact that lL2er BiTDs interact with Heritage Turkish must be considered in future studies as it may have an impact on the findings about their proficiency in Standard Turkish. 

Structural complexity patterns in 4-6-year old isiXhosa and Afrikaans speaking children in South Africa

Abstract:

The MAIN protocol is used by more than 3200 researchers from over 60 countries working with 92 languages. Despite the wealth of information available from international studies, differences in methodologies such as elicitation method (tell, retell), choice of stimulus pictures (BB, BG, C, D) and study populations (mono-, multilingual) make comparisons across languages difficult. There is a need for information about developmental patterns and norms, particularly concerning the development of structural complexity in both mono- and multilingual children. Our clinical focus is on 4–6-year-old children from lower SES environments because of our emphasis on early identification and intervention for children at-risk for language and literacy problems when they start formal schooling at age 6. We are therefore interested to know what patterns can be expected in this age group regarding narrative structural complexity.

As part of a larger research project, we collected MAIN narratives from 121 isiXhosa and 107 Afrikaans-speaking children, aged 4 to 5 years. This cohort was randomly selected from 52 early education centres situated in two lower socio-economic areas in the Western Cape, South Africa. We used the Cat and Dog stories and the telling mode for elicitation of the narratives.

Our presentation will focus on the analyses of the structural complexity of the narratives using the Westby binary decision tree. Additionally, we will draw comparisons between the levels of structural complexity and the children’s performances on a developmental screener, the Early Learning Outcomes Measure (ELOM)

Narrative Dynamic Assessment Training for School-Based Speech-Language Pathologists using the MAIN

Abstract:

There are numerous assessment methods available to school-based speech-language pathologists (SLPs). However, the lack of culturally and linguistically valid standardized assessments is of great concern. A complement to standardized assessment, which evaluates prior learning experiences, is dynamic assessment (DA), which analyzes the process of learning. Unfortunately, a lack of confidence, unfamiliarity in administering DA, and perceived time constraints have been reported as obstacles for SLPs. This study used a single-subject case design to investigate the effect of performance feedback on the implementation fidelity of narrative mediated learning (ML) sessions using the Multilingual Assessment Instrument for Narratives (MAIN, Gagarina et al., 2019) adapted as a DA measure. The three participant SLPs also completed initial and final questionnaires to determine whether their attitudes and perceptions of DA changed throughout the research investigation. The results suggested that performance feedback using a fidelity rubric effectively improved narrative ML procedures and fidelity. This is evidenced by a baseline decelerating trend to an intervention accelerating trend with 100% non-overlapping data between the A1 and B phases for all participants. By their own appraisal, the training and feedback refined their competency, efficiency, and capacity to perform narrative DA using the MAIN. Additionally, the participants demonstrated that the MAIN-DA could be conducted and scored in less than 40 minutes per session. Becoming more knowledgeable and comfortable with DA equipped the research subjects with a sound method for assessing culturally and linguistically diverse students and addressing possible misdiagnoses based on language differences.

Cross-lectal narrative skills in Arabic diglossia in children with typical versus atypical language development

Abstract:

Purpose: The study tested narrative skills in kindergarten Arabic-speaking children with Developmental Language Disorder (DLD) and in age-matched children with Typical Language Development (TLD). Specifically, it compared macro and micro-structure skills as well as the use of lexical words in Spoken Arabic (SpA) and Standard Arabic (StA) in two story modes: story telling in SpA versus retelling of a story heard in StA.

Method: 18 children with DLD (mean age=5.6) and 19 children with TLD (mean age=5.7) were tested. Two narrative tasks using picture prompts were administered: story telling in SpA and story retelling based on a narration in StA. The two stories were matched on complexity and episodic structure. Macro-structure was analyzed using the Goal-Attempt-Outcome (GAO) schema. Micro-structure analysis addressed productivity, lexical diversity, and morpho-syntactic accuracy. Analysis of lexical words was based on Saiegh-Haddad & Spolsky’s (2014) lexical distance framework which distinguishes three types of words: identical words, cognate and unique words.

Results: Children with TLD demonstrated significantly higher scores compared to children with DLD on macro-structure and on most micro-structure features as well as the use of StA lexical words, demonstrating higher productivity and fewer morpho-syntactic errors in subject-verb gender agreement. The findings also revealed a significant effect of narrative mode; despite higher linguistic productivity skills in the telling mode, both groups exhibited higher macro structure skills and higher type-token ratio as well more StA lexical words in the retelling mode.

Conclusion: The results support earlier reports of differences between children with TLD and DLD in narrative skills. Moreover, the results demonstrate the role of the retelling story mode in enhancing macro structure generation and lexical diversity and the use of StA lexical words for both groups of children, even when narration is in StA, a variety less familiar to children. The implications of these findings for assessment and intervention are discussed.

Narrative comprehension in adults with Down syndrome: understanding inferencing through MAIN

Abstract:

Language comprehension represents a relative strength for individuals with Down syndrome (DS), particularly in terms of vocabulary as opposed to syntax. However, difficulties have also been highlighted in areas of pragmatics. Despite apparent issues with recall, studies investigating storytelling report some sensitivity to the causal structures of narratives, while instances of poor inferencing skills may be associated with expressive issues. This study investigated narrative comprehension after retell in a group of 26 adults and young people with Down syndrome and 24 receptive vocabulary-matched typical controls. Inferencing abilities were assessed through 10 questions from the Multilingual Assessment Instrument for Narratives (MAIN, Gagarina et al., 2019), which targeted understanding of characters’ goals, internal states (IS), and the general plot of the story. Our analyses revealed a pattern of poorer global comprehension scores for participants with DS, characterised by more marked difficulties with IS questions and a significant effect of mental age. This suggests that individuals with Down syndrome show varying levels of sensitivity to different inferential aspects of stories, finding comprehension questions that require emotional labelling and hypothetical thinking more challenging. We discuss whether such difficulties could be the result of underlying cognitive or syntactic factors.

The narrative skills of Arabic-Swedish-speaking children (aged 4-7) in Sweden

Abstract:

Despite Arabic being the second most spoken home language (after Swedish) in Sweden, the present study is the first to analyze the narrative abilities of Arabic-Swedish-speaking bilingual children. In this talk, I will present results from my doctoral dissertation on the narrative skills (using MAIN, Gagarina et al., 2019) of 100 Arabic-Swedish-speaking bilingual children aged 4 to 7 residing in Sweden. Results are presented for both comprehension and production of narrative macrostructure in relation to age development, language differences (Arabic, Swedish), task effects, and story effects. Results are also reported regarding the children's performance on individual comprehension questions and their production of story components and story complexity. The present study is part of the BiLI-TAS project at Uppsala University.

Lesson learned from remote data collection in a Chinese bilingual kindergarten using MAIN

Abstract:

This time a PhD student from University Erlangen Hiu Ching Hung will present a study aiming to look into the relations between the L2 narrative comprehension of Chinese-English bilingual preschoolers and the prosody of teachers using MAIN. Conducted in July 2022 amidst the pandemic time, the study will provide insights into the tools and techniques used to obtain high-quality audio and video recordings. In addition, the presentation will cover observations on the use of questionnaires, teachers' behaviors, and children's language patterns. Please note that this meeting will not be recorded upon request.

Remote Online Language Assessment: Comparison of The Narrative Skills of Monolingual Turkish Speaking Children with Typical Development and Developmental Language Disorder Through MAIN-TR

Abstract:

The aim of this study is to assess the narrative skills of Turkish-speaking monolingual children with Typical Development (TD) and Developmental Language Disorder (DLD) through, the most noteworthy assessment tool, the Turkish version of MAIN (Multilingual Assessment Instrument for Narratives) remotely online and compare both children group. The research group consists of a total of 128 children aged 45-89 months; monolingual Turkish speaking children with typical development (n=96) and developmental language disorder (n=32). MAIN-TR was conducted remotely online in three elicitation modes (retell, tell, model story), and the narratives produced by the children were assessed within the framework of the macrostructure.  Besides, comprehension questions were asked to the children following these modes. All the stories (The Cat, The Dog, The Baby Birds & The Baby Goats) were conducted in the study. During this Text & Tea meeting, Semra will speak about her experiences about the remotely online narrative assessment process using MAIN-TR.

Experiences with using MAIN as a written and verbal task for children in Dutch (bilingual) primary education

Abstract:

The “FoTo” project is a research project following a pilot for bilingual primary education in the Netherlands. In this pilot, students are given 30-50% of their classes in English, starting in pre-kindergarten (Dutch grade 1). After several rounds of testing, the coming round of testing is the last one, in the last year of primary school (Dutch grade 8). In addition to several vocabulary and grammar proficiency texts, in grades 5 and 8 students are given a writing task based on the MAIN, in which they write a short story about one of the MAIN picture stories. In grade 8 a selection of students is also asked to do an adapted version of the MAIN task in which they tell 2 stories. During this Text & Tea meeting, Annelies will speak about our experiences with using the MAIN task as a written task and share some of the first results from the grade 8 testing round.

Using MAIN to validate a Mandarin sentence repetition task as a language screening tool

Abstract:

Sentence repetition (SR) is believed to be a clinical marker for developmental language disorder (DLD) across many languages. This study explored the potential of a self-designed Mandarin SR task (MSRT) to reflect Mandarin-speaking preschoolers’ language ability and to differentiate between children with and without DLD in this population. Method: In study 1, the MSRT was administered to 59 typically-developing (TD) children aged 3;6 (years; months) to 6;5 in China. The task was examined regarding its ability to correlate with language indices derived from children’s narrative samples, collected using the Mandarin version of the Multilingual Assessment Instrument for Narratives (MAIN). In study 2, both a TD and a DLD group were recruited to investigate the task’s sensitivity, specificity, and likelihood ratios to distinguish between children with and without DLD. Results: Study 1 showed that TD children’s performance on the MSRT significantly correlated with all the language measures derived from children’s narrative samples. Study 2 showed that the MSRT was able to differentiate between children with and without DLD. Conclusion: The MSRT is a promising tool to reflect language abilities and identify DLD in Mandarin-speaking preschoolers.

Narrative skills in the ambient language German in three to six-year-old bi- and multilingual children – 

Associations with German vocabulary and grammar skills, language exposure and general cognitive abilities

Abstract:

Narrative abilities are discussed as a promising tool for the assessment of language abilities in bi- and multilingal children, as they are assumed to be less dependent on the amount of exposure in a specific language and more closely associated with general cognitive abilities than language specific measures such as vocabulary or grammar. However, research concerning the question, how various factors shape narrative abilities in bi- and multilingual children has yielded differing results. Therefore, our goal was to conduct a comprehensive analysis of the relationship between narrative abilities and different child-internal and child-external factors. To this end, we analysed data from a subsample of the research project IMKi. The sample consists of 236 bi- and multilingual children between the ages of three and six, who grow up with German as their ambient language and several different heritage languages. Relationships between the narrative abilities in the ambient language German and German language abilities (expressive and receptive vocabulary, sentence repetition), language exposure in the home and general cognitive abilities are analysed.

Age and language exposure effects in the narratives of Basque school-age children 

Abstract:

The MAIN instrument has been designed to measure bi-/multilingual speakers’ narrative skills in one or more languages (Gagarina et al. 2012, 2020). The current study was conducted with 72 TD Basque-Spanish bilingual children, aged 4 to 8 years and living in two differentiated sociolinguistic areas: Basque-dominant environment (n=22) and (almost) monolingual Spanish-speaking environment (n=50). All participants narrated orally the Cat story following the telling procedure.

In addition to the MAIN A17 scoring, which measures narrative complexity, two more indicators of development were included in the analysis: degree of autonomy (number of speech turns), and speech fluency (number of words, total and by speech turn). The analysis revealed statistical differences between 4-5-y-o children vs. 6-8-y-o children in all the three indicators. Effects of the sociolinguistic environment arose only in the youngest children, both in total length and in narrative complexity.

Do education and age influence visual narrative comprehension processes? – A representative study

Abstract:

In this talk, I will present the results of a representative German online study designed to broaden our understanding of comprehension processes for visual narratives in different age and education groups. It investigated bridging inference generation, i.e., inferences generated to bridge gaps in the narration, during the viewing of pictorial and textual narratives generated from the MAIN material. The study provides answers to the questions of whether participants in various education and age groups significantly differ in narrative comprehension and whether there is a general comprehension skill with regard to the codality (text/picture) of the narrative.

What guides inference generation? A study of young Hindi learners studying in challenging contexts in India

Abstract:

In a study on 30 Indian learners studying in grade four and in Hindi medium schools from low SES background comprehension skills are assessed through MAIN instrument (the cat story) and a literacy test. The findings show that individual differences in inference generation, a sub-skill of comprehension, is affected by modality, gender and type of inferences.

CHILDES corpora and tools for studying child language development

Abstract:

CHILDES is the system used for the study of child language acquisition.  There have been over 8000 publications based on use of data from CHILDES.  This presentation will review the current status of resources in CHILDES and the related PhonBank project. It will focus on methods for morphological analysis, profiling commands, and the new TalkBankDB database search facility.

MAIN in Indian languages: From 2013 to 2021

Abstract:

In this presentation, we talk about the process of adapting MAIN into several Indian languages. Beginning with a historic overview of the multilingual nature of India and its linguistic diversity, we go on to discuss the chief linguistic features of the Indian languages into which MAIN has been adapted. The presentation also covers some of the challenges we faced while adapting the instrument – e.g., differences in vocabulary use, syntactic structures, storytelling conventions as well as cultural differences. Finally, we present a brief overview of findings from some of the studies that have used MAIN with young bilinguals in both the face-to-face and online modalities.

Adapting MAIN to Bosnian

Abstract:

The project of adapting MAIN to Bosnian was initiated in January 2021 by Dr. Perović. It is a language spoken in Bosnia and Herzegovina. This adaptation closely follows the Serbian version of MAIN – MTPN (Jeličić et al., 2020) as Bosnian is closely related to Serbian (and Croatian). Until recently, these languages were subsumed under the name Serbo-Croatian. Research of narratives in Bosnian is practically non-existent, both with monolinguals and bilinguals. During this project, data will be collected from two groups of participants: Bosnian-German bilingual adults. As this project also functions as a master’s thesis for the European Master’s in Clinical Linguistics+ program, a closer analysis of nominal morphology in the production of Bosnian narratives will be performed.

Language across lifespans: Researching linguistic skills with a focus on narratives

Abstract:

In this short presentation, I will introduce my current research interests. These include the testing of narrative skills with the help of MAIN in different groups of adult speakers: young adulthood, middle age and older adulthood (including possible cognitive impairments).

Similarities and differences between MAIN stories in comprehension and production of narrative macrostructure - results from studies of Swedish-speaking monolinguals and bilinguals

Abstract:

The four stories of the Multilingual Assessment Instrument for Narratives (MAIN; Gagarina et al., 2019) were created to be parallel in their narrative macrostructure; for all four stories (Cat, Dog, Baby Birds, Baby Goats), the same macrostructural components (e.g. goals, attempts, outcomes) were carefully depicted in the picture sequences and included in the story scripts. The ten accompanying comprehension questions also target the same macrostructural components in all four stories. Naturally, it is important to confirm that child performance is at the same level for all stories, e.g. to investigate empirically whether narratives elicited with the different stories are comparable in terms of narrative macrostructure and whether children score equally well on the comprehension questions irrespective of the story used. In this talk, I discuss similarities and differences between the stories based on results from a number of studies of narratives by Swedish-speaking monolinguals and bilinguals.

Slovak version of the online method 

Abstract:

In the presentation, we are introducing one of the various ways to use the MAIN method online. The primary idea is to keep the online space administration as close to face-to-face administration as possible. The shared knowledge that the adult and child have seems to be crucial in telling the story. Reflect this; the main point is to convince the child that the administrator sees something slightly different on their screen to motivate them to tell the story in as much detail as possible.

Our Slovak team prepares the individual story presentation for the use of other researchers.