A three-step tutorial on Running Experimens Online
with (absolutely) no coding.
A three-step tutorial on Running Experimens Online
with (absolutely) no coding.
Embedded Files
with (absolutely) no coding.
Step 1: Why run an online experiment?
Step 1: Why run an online experiment?
REO is fast and cost-efficient. But there is actually more that might benefit your research.
Step 2: Creating an experiment
Step 2: Creating an experiment
We use Gorilla.sc to make a new experiment that is browser-ready!
Step 3: Data collection
Step 3: Data collection
This part involves interacting with, and getting responses from, humans. Fun!
"Becoming familiar with these tools allows one to move from hypothesis to experimental data within hours"
(Grootswagers, 2020)
"Becoming familiar with these tools allows one to move from hypothesis to experimental data within hours"
(Grootswagers, 2020)
Useful resources
Useful resources
To get started:
MASON & SURI (2012)
MASON & SURI (2012)
A very detailed overview of Amazon Mechanical Turk: its advantages and pitfalls. Note that this article is from 2012. Much has changed since then. But the general caveats and recommendations are still very useful.
GROOTSWAGERS (2020)
GROOTSWAGERS (2020)
A primer on the architecture behind a web-based experiment (creating and hosting the experiment, interacting with servers, recruiting subjects). Not a "how-to" primer; it's more of a high-level description of what it takes to collect data through the web.
THEODORE (2020)
THEODORE (2020)
This is a very informative zoom tutorial by Rachel Theodore (U Conn). If you plan to run an online speech perception experiment, you should definitely check this out. You can download her slides from here.
The author, Ben Howell, is a creator of an online experiment builder, Psychostudio. This article and other posts of his cover a wide range of topics that are useful for all of us. (e.g., How to prevent cheating in online surveys and experiments., 51 places to find research participants for your study)
To learn more:
Smith (2021-2022)
Smith (2021-2022)
This is a course website created by Kenny Smith (University of Edinburgh). The course looks amazing! The technical components are beyond the level of what we are talking about. (Well, they do involve coding...) but the assigned readings touch on various issues of linguistic experimentations such as syntactic priming, partner-based interactive tasks, and iterative language learning. Highly recommend!
Dandurand et al (2008)
Dandurand et al (2008)
The authors were early adaptors of online methods. This article presents a side-by-side comparison of lab-based and web-based experiments. TLDR; People do better when they read instructions regardless of platforms, and online subjects may not be as attentive as you want them to be.
Bridges Et AL(2020)
Bridges Et AL(2020)
This is by far the largest comparison study on reaction times, using both in-lab and online experimental platforms. "Among the lab-based experiments, Psychtoolbox, PsychoPy, Presentation and E-Prime provided the best timing, all with meanprecision under 1 millisecond across the visual, audio and response measures." The online ones performed slightly worse but "That said, PsychoPy and Gorilla,broadly the best performers, were achieving very close to millisecond precision onseveral browser/operating system combinations".
This tutorial was created by:
This tutorial was created by:
Chigusa Kurumada
Chigusa Kurumada
I am a language researcher and an assistant processor at @UoR_BrainCogSci. My lab (https://kinderlab.bcs.rochester.edu/) studies language processing, learning, and communication. This tutorial was for the "Ph.D skills workshop" at the Stockholm University.
I thank my collaborator Dr. Loisa Bennetto (Psychology, University of Rochester) and our joint research team. Much of what I share here is what I learned through our joint efforts and explorations.
Page updated
Google Sites
Report abuse