Module 06
Paired Samples T-Test
***Reminder***
For paired samples t-test examples, we use another data file to demonstrate the statistical analysis.
If you want to try it yourself, please download the data file below (Link is also available at the homepage).
demo Stroop data file (.CSV)
Introduction
Many research questions are about comparison between two population means. To test the hypothesis generated from such research questions, we cannot use one-sample t-test to run the analysis, because it is only for comparing one population mean with a constant.
In this module, we focus on paired samples t-test, which is mainly used when we are testing the within-subject difference. Because the two means that we are going to compare are not independent- they are from the same participant but in different conditions, this test is also known as dependent sample t-test.
1. What is paired samples t-test?
Paired samples t-test compares two means from the same sample that has been put into two conditions. For example, suppose we are interested in how horror movie affects our heart rate. We recruit a group of participants and record their heart rates before and after watching a horror movie. We can pair up the before and after conditions for each participant in the sample and test whether the heart rate has changed after watching the horror movie.
[Try it yourself: Essentially, the computation of paired samples t-test is the same as one-sample t-test, except that, before conducting the one-sample t-test, we need to compute the difference between the two measurements for each participant (also known as the D scores). Then, we test whether the average difference is different from a test value (typically set at zero).]
2. Example 1: Comparing different conditions in Stroop Task
Stroop task, one of the classical experiments in psychology, is to measure the reaction time of responding to the ink color of a color word. There are two conditions, namely, congruent and incongruent. In the congruent condition, the color word is same as the ink of the word. For example, “red” is written in red ink color. In the incongruent condition, the color word is different from the ink of the word. For example, “red” is written in blue color. We want to measure the task performance between congruent and incongruent conditions. We use reaction time in millisecond (ms) to measure their performance.
Q: Does congruence between the color word and its ink color affect reaction time? (α = .05)
A: We used paired samples t-test (two-tailed) to examine this.
Step 1: Set the research hypothesis: The reaction times in the two conditions are different.
Step 2: Based on your research hypothesis, write down the null and alternative hypotheses:
H0 : μD = 0
H1 : μD ≠ 0
Step 3: Perform statistical analysis in jamovi (Please use full screen mode).
Example6.1_Stroop11.mp4Based on the results from jamovi, we can draw the conclusion that there was a difference between two conditions.
Conclusion / Interpretation (APA format):
There was a significant difference in reaction time between the congruent (M = 500, SD = 98.8) and incongruent (M = 677, SD = 136.7) conditions, t(23) = -7.24, p < 0.001, d = -1.48.
3. Example 2: Comparing two sessions Stroop task in the same condition
In addition to the comparison between the congruent and incongruent conditions, we want to examine whether the reaction time in the incongruent condition can be shortened through training. We provided a week of training to the participant and invited the participant to do the Stroop task again after the training. In this example, we focused on the change in reaction time in the incongruent conditions in the two sessions.
Q: Does training have an effect on reaction time in the incongruent condition? (α = .05)
A: We used paired samples t-test (one-tailed) to examine.
Step 1: Set the research hypothesis: The reaction time of the first session is longer than that of the second session.
Step 2: Based on your research hypothesis, write down the null and alternative hypotheses:
H0: μD ≤ 0
H1: μD > 0
Step 3: Perform statistical analysis in jamovi (Please use fullscreen mode).
Example6.2_12mp4.mp4Based on the results from jamovi, we can draw the conclusion that the reaction time of the first session was longer.
Conclusion / Interpretation (APA format):
Reaction time of the first incongruent session (M = 677, SD = 137) were significantly longer than that of the second incongruent session (M = 578, SD = 159), t(23) = 2.77, p = 0.005, d = 0.565.
4. Effect size for paired-samples t-test
Similar to the effect size for the one-sample t-test, we are interested in the standardized mean difference between the mean of D scores (i.e., the difference in measurements between the two conditions for each observation) and the test value (usually = 0 for paired t-test):
The interpretation of the paired-sample Cohen's d is same as the one-sample Cohen's d.
Module Exercise
Complete the exercise!
Now, if you think you're ready for the exercise, you can check your email for the link.
Remember to submit your answers before the deadline in order to earn the credits!