Experimental Methods

ChatGPT has sparked a million dollar question for linguists and linguistic scientists: Is language fundamentally human? If a large set of training data can bring an AI close to human language ability, do we really need to assume any biological basis for language? Wait. But it would take a baby 100,000 years to match the amount of input that would be fed to train an AI model... Is that a fair comparison? The development of new models and datasets not only changes the methods to answer a question, but also changes the types of questions we ask. In this brainstorming session, we will read and discuss Vong et al. (2024, Science) to think about how a theory and a method can evolve together to create a paradigm shift in the study of language.

Even for an experienced researcher, empirical experiments are always full of trials and errors. There is no such thing as a perfect experiment, and we can only try to minimize the number of trials (and, more importantly, errors). In this hands-on session, we will take a sample project from a participant and think about how best to run the very first experiment to collect data. To make the content accessible and relevant to many of us, we will focus on a behavioral paradigm with simple measures such as forced-choice responses and response times. We will learn basic concepts such as dependent/independent variables and factorial design, as well as how to .

All experimentation is meant to be an iterative process with a "life cycle". You run your first experiment, which gives you the data and information you need to design another cycle of experiments. But, then, how do you design your first experiment? --- How many subjects do we need? What statistical tests should we use? How can we make predictions without even knowing what will happen in the experiment? --- This is a very common conundrum for many of us when writing a research proposal, a grant application, or an experimental protocol for institutional approval. In this session, we will walk through the process of study pre-registration using the same sample project from Session 2. We will use OSF (Open Science Framework: https://osf.io/) as an example platform for pre-registration and data sharing and learn about its key workflows and features.