Embedded Files
Timo roettger
Timo roettger
Timo Roettger is a Professor of Linguistics at the University of Oslo. His scientific work has two goals: to understand how people communicate their intentions using speech and to help improve methodological practices across the language sciences. Among many other papers, his publications demonstrate that flexibility in choices in the research process, both in operationalization of theoretical constructs and the analytical pipeline, can have meaningful consequences for our narrative conclusions.
Against Replications
Timo Roettger
To effectively advance knowledge, science must produce data that can be reliably replicated using the original methods. However, coordinated efforts to replicate published findings across disciplines have revealed alarmingly low success rates, sparking what is now known as the replication crisis. This has prompted a constructive dialogue and practical recommendations, such as directly replicating studies and preregistering hypothesis-testing research.
Please don’t. In this talk, I will argue that most researchers in the language sciences should not prioritize replication and preregistration—and, in fact, that we should reconsider hypothesis testing altogether. I contend that large parts of our field are not yet ready for hypothesis testing. Instead, we need to focus on thoroughly exploring the quantitative systems we study, developing precise formal models of these systems, and only then moving toward testable predictions that are worth preregistering, testing, and replicating.
This talk will outline why exploration and quantitative theory development should take precedence and how such an approach can set the stage for more meaningful and reliable scientific progress in the language sciences.
Anne Beatty-Martínez
Anne Beatty-Martínez
Anne L. Beatty-Martínez is Assistant Professor of Cognitive Science and Director of the Center for Research on Language at the University of California, San Diego. Her research has two intertwined strands: one that examines how cognition supports language use and another that asks how language use impacts cognition itself. In her work, she capitalizes on the diversity and variability in people’s experiences to better understand how the mind and brain adapt to the demands of more than one language.
Research on Bilingualism as Discovery Science
Anne Beatty-Martínez
An important aim of research on bilingualism is to understand how the brain adapts to the demands of using more than one language. Over the past few decades, significant discoveries have been made regarding the consequences of bilingualism. However, concerns about replicability, fueled by the replication crisis of the previous decade, have shifted the focus of discussion, narrowing it primarily to issues like sample size and methodological uniformity. We critique this approach and reformulate the issues by placing them in the broader context of science as a discovery process, in which incremental understanding, methodological diversification, the framing of our questions, and even small sample studies, are essential for advance. We articulate what we believe are the goals of our scientific enterprise for the foreseeable future. We propose that such goals require practices and tools that provide a rich characterization of the participant sample to explore the cognitive and neural bases of bilingual phenotypes: the adaptive variety induced through the interplay of biology and the environment.
bodo winter
bodo winter
Bodo Winter is a Professor of Linguistics at the University of Birmingham and a UKRI Future Leaders Fellow. His research uses data science-driven linguistics to study multimodal communication, including iconicity, gesture and metaphor. His work is wide-ranging and consists of statistics training and tutorials for linguists, generalizability and the bouba/kiki effect.
The Generalizability Crisis in Linguistics: Rethinking Statistical Traditions
Bodo Winter
While the replication crisis has sparked much-needed scrutiny of methodological practices, the generalizability crisis may be an even deeper problem: many findings, even when replicable, fail to generalize because they neglect key sources of variation (Yarkoni, 2022; Winter & Grice, 2022). The generalizability crisis has many different origins, of which I will focus on three in my talk: (1) significance testing prioritizes testing mean differences over explicitly modeling variation, (2) researchers simplify random effects structures to avoid “convergence issues”, thereby limiting generalizability, and (3) experimental design, shaped by Ronald Fisher’s influential 20th century textbooks, often treats variation as mere noise and seeks homogeneous samples rather than focusing on whether findings hold across sources of variation.
I will argue that Bayesian methods offer a solution to the generalizability crisis, and an opportunity for us to free ourselves from deeply engrained statistical rituals (Gigerenzer, 2004). Among many other advantages, Bayesian methods provide more robust estimation than frequentist models like lme4, thereby reducing the pressure to simplify random effects. This also has ripple effects on the kinds of experiments we can design: unconstrained by the failure of our mixed models to estimate complex random effects structures, we can design new kinds of experiments with variation in mind. Finally, Bayesian inference conceptually emphasizes variation and uncertainty, thereby fostering a shift away from an over-emphasis of testing mean differences.
Angela De bruin
Angela De bruin
Angela de Bruin is a Senior Lecturer at the University of York. Her research interests include bilingualism, language switching, language production, executive control, and cognitive aging. Her work has examined the influence of publication biases on research studying bilingualism and executive control.
The importance of both direct and conceptual replication studies when studying language users from different language backgrounds
Angela de Bruin
Issues with (failures to) replicate and reproduce research have received substantial attention over the past decade, both within language sciences as well as throughout experimental research areas more broadly. In this talk, I will discuss the importance of replication studies and increasing reproducibility, as well as ways to improve both. I will focus on research studying different questions related to bilingualism, although it is very likely these points apply beyond this research area. A key part of my talk will focus on different factors that can explain why many studies cannot be replicated – but also why studies with bilinguals from different language backgrounds are not always expected to find the same results. I will therefore discuss the need for both direct and conceptual replication studies, especially to increase our theoretical understanding of the mechanisms involved in language comprehension, production, and control across the large variety of language users in monolingual and multilingual societies.
Page updated
Google Sites
Report abuse