The Interaction of Linguistic and Visual Cues for the Processing of Case in Russian by Russian-German Bilinguals: An Eye Tracking Study. 2024. Topics in Cognitive Science. (Minor S., Mitrofanova N., Westergaard M.)
In this study, we investigated the relationship between comprehension and prediction by testing how subtle changes in visual representations can affect the processing of grammatical case cues in Russian by Russian-German bilingual children (n = 49, age 8−13). The linguistic manipulation followed previous designs, contrasting SVO and OVS sentences, where the first NP (NP1) was marked with nominative or accusative case, respectively. Three types of visual displays were compared: (i) individual referents (potential agent/theme); (ii) pairs of referents (NP1 + potential agent/theme); and (iii) events (representing interactions between the referents). Participants were significantly more sensitive to the case manipulation when presented with events compared to the other two types of visual display. This suggests that they were able to quickly integrate the thematic role information signaled by grammatical case in the event representations. However, they were less likely to use the case information to anticipate upcoming arguments when the target pictures represented individual referents or pairs of noninteracting referents. We hypothesize that the process of argument anticipation is mediated by the activation of syntactic templates (SVO or OSV, depending on the case marking on NP1). The relatively weak anticipation effect observed may be attributed to the absence, or weak representation, of the noncanonical OVS template in the bilingual children's long-term memory.
Aspect processing across languages: A visual world eye-tracking study. 2023. Frontiers in Language Sciences, 1 (Minor S., Mitrofanova N., Guajardo G., Vos M., Ramchand G.)
The study employed a combination of a picture selection task and Visual World eye-tracking to investigate the processing of grammatical aspect (perfective vs. imperfective) in three languages: Russian, Spanish and English. In order to probe into the cognitive representations triggered by the aspectual forms we contrasted visual representations of different temporal portions of telic events—a snapshot of the process stage (ongoing event) and a snapshot of the immediate aftermath of the event/the result state (completed event). In all three languages, the gaze patterns and offline responses revealed a strong preference for representations of ongoing events in the imperfective condition. This confirms that the imperfective forms in all the three languages draw attention to the in-progress portion of a telic event. In the perfective condition, however, we found robust differences. Russian uses verbal prefixes to mark perfective aspect, and our results suggest that perfective telic verbs in Russian strongly highlight the result state of an event. In Spanish, the perfective past tense form (Preterite) also highlights event completion, but to a lesser extent than in Russian—in line with its less restrictive semantics in not requiring an inherent boundary. In contrast to Russian and Spanish, English speakers did not show a preference for representations of completed events in the perfective (Simple Past) condition. This suggests that the English Simple Past form does not encode a preferential cognitive salience for either the activity portion of an event or its result state, and lends support to the analysis of the English Simple Past as a non-aspectual tense form.
Temporal information and event bounding across languages: Evidence from visual world eyetracking. 2022. Proceedings of SALT 32. (Minor S., Mitrofanova N., Guajardo G., Vos M., Ramchand G.)
Comparing infrared and webcam eye tracking in the Visual World Paradigm. 2022. Glossa Psycholinguistics, 1(1), (Vos M., Minor S., Ramchand G.)
Visual World eye tracking is a temporally fine-grained method of monitoring attention, making it a popular tool in the study of online sentence processing. Recently, while infrared eye tracking was mostly unavailable, various web-based experiment platforms have rapidly developed webcam eye tracking functionalities, which are now in urgent need of testing and evaluation. We replicated a recent Visual World study on the incremental processing of verb aspect in English using ‘out of the box’ webcam eye tracking software (jsPsych; de Leeuw, 2015) and crowdsourced participants, and fully replicated both the offline and online results of the original study. We furthermore discuss factors influencing the quality and interpretability of webcam eye tracking data, particularly with regards to temporal and spatial resolution; and conclude that remote webcam eye tracking can serve as an affordable and accessible alternative to lab-based infrared eye tracking, even for questions probing the time-course of language processing.
Fine-grained time course of verb aspect processing, 2022, PLOS One, 17(2), (Minor S., Mitrofanova N., & Ramchand G.)
Sentence processing is known to be highly incremental. Speakers make incremental commitments as the sentence unfolds, dynamically updating their representations based on the smallest pieces of information from the incoming speech stream. Less is known about linguistic processing on the sub-word level, especially with regard to abstract grammatical information. This study employed the Visual World Paradigm to investigate the processing of grammatical aspect by Russian-speaking adults (n=124). Aspectual information is encoded relatively early within the Russian verb which makes this an ideal testing ground to investigate the incrementality of grammatical processing on the sub-word level. Participants showed preference for pictures of ongoing events when they heard sentences involving Imperfective verbs, and for pictures of completed events when they heard sentences involving Perfective verbs. Crucially, the analysis of the participants' eye-movements showed that they exhibited preference for the target picture already before they heard the end of the verb. Moreover, the latency of this effect depended on where the aspectual information was encoded within the verb. These results indicate that the processing and integration of grammatical aspect information can happen rapidly and incrementally on a fine-grained word-internal level. Methodologically, the study draws together a set of analytical techniques which can be fruitfully applied to the analysis of effect latencies in a wide range of studies within the Visual World eye-tracking paradigm.
Comparing Infrared and Webcam-based Eye Tracking in the Visual World Paradigm, 2021, talk, AMLaP 27, (Vos M., Minor S., & Ramchand G.)
The Processing and Acquisition of Russian Aspect: Evidence from VW Eye-Tracking, 2020, talk, AMLaP 26, (Minor S., Mitrofanova N., & Ramchand G.)
Processing of Grammatical Aspect in a Visual World: English vs Russian, 2020, poster presentation, AMLaP 26, (Minor S., Mitrofanova N., Vos M., & Ramchand G.)
A presentation from AMLaP 2020 . We investigated the processing of verbal aspect by Russian and English-speaking adults using a Visual World eye-tracking experiment. In both the investigated languages we manipulated the aspect of the verb in the audio stimulus. In the Russian version of the Experiment, half of the audio stimuli involved a Perfective verb and half involved an Imperfective verb. In the English version, half of the audio stimuli involved a verb in the Past Progressive form and half involved a verb in the Simple Past form. All test items involved telic predicates. Our results suggest that both Imperfective verbs in Russian and the Past Progressive form in English highlight the ongoing portion of the described event. However, we found a striking contrast in the processing of Perfective verbs in Russian and Simple Past verbs in English. Whereas Russian Perfective verbs clearly draw attention to the final portion and/or result state of the event, the English Simple Past form does not encode a preferential cognitive salience for either the activity portion of the event or its result state (in line with previous offline behavioural findings in Jeschull 2007, Arunachalam & Kothari 2011).
The Processing of Grammatical Aspect by Russian-speaking Adults and Children: A VW Eye-Tracking Study, 2020, poster presentation, CUNY 33, (Minor S., Mitrofanova N., & Ramchand G.)
Processing Aspect in Russian and English: Evidence from Visual World Eye-Tracking, 2020, poster presentation, CUNY 33, (Minor S., Mitrofanova N., Vos M., & Ramchand G.)