Module 08
One-Way ANOVA
Introduction
In some studies, there are more than two groups of participants or conditions.
To answer the question "Does the DV differ across ALL groups/conditions?", we cannot use t tests, because the question involves three or more means.
In that case, we can conduct a test called ANOVA, which is the acronym for ANalysis Of VAriance.
1. What is One-Way ANOVA?
In general, ANOVA (analysis of variance) compares the variability of DV across groups with the variability of DV within groups. If the across-group variability is much larger than the within-group variability, it is considered as statistical evidence to show that the group means differ in the population.
One-Way ANOVA means we consider how a single factor affects the DV.
For example, if we want to know whether the stress levels of students differ across universities, we can collect a sample of students from different universities and ask about their stress, and use ANOVA to answer the question.
2. Example 1: Faculty and GPA
Some people believe that the faculty you belong to determines your academic results. For example, it may be difficult to achieve the full marks in courses in arts faculty, while in courses related to mathematics it may be possible to achieve the full marks. Is it true or not?
Q: Does the faculty that a student belongs to have any effect on the student's GPA?
A: One-Way ANOVA can be used to examine this hypothesis.
Step 1: Perform statistical analysis in jamovi (Please use fullscreen mode).
Example 7.1 OneWayANOVA_GPA_woPlot.mp4Based on the results from jamovi, we can draw the conclusion that there was no difference in GPA across faculties.
Conclusion/ Interpretation (APA format):
There was no difference in GPA result across faculties, F(2, 197) = 2.64, p = 0.074, η2 = 0.026.
3. Example 2: Hostel and Sleeping hours
In Lingnan University, students can choose to live in one of the 10 hostels. Generally, hostels are divided in 3 categories which are South, North, and New hostels. Some people believe that students' behavior varies across hostels, especially in sleeping hours. Therefore, we collected the data to test the hypotheses.
Q: Do sleeping hours vary across hostels?
A: We used One-Way ANOVA to examine.
Step 1: Perform statistical analysis in jamovi (Please use full screen mode).
Example 7.2 OneWayANOVA_Sleep_woplot.mp4Based on the results from jamovi, we can draw the conclusion that there was a difference in sleeping hours across hostels.
Conclusion/ Interpretation (APA format):
There was a significant difference in sleeping hours across hostels, F(3, 196) = 2.94, p = 0.034, η2 = 0.043.
4. Visualization
Sometimes, we may want to visualize the data by plotting some graphs.
Let's use the example 1: Faculty and GPA to illustrate how to do it in jamovi.
Example 7.1 Exploration.mp45. Effect size in ANOVA
Same as t-test, we can measure the effect size in ANOVA. We measure the magnitude of effect by calculating how much of the variance in the dependent variable can be explained by the independent variable. The statistical term we use is the correlation ratio, denoted as eta-squared (η2 ). Theoretically, the range of η2 is from zero to one (0 - 1). E.g., if η2 = .35, it means that 35% the variance in the DV can be explained by the IV, and 65% of the variance in the DV is attributed to other factors that are not related to the IV.
Below is a table for judging the size of η2 in ANOVA in general.
6. Post-hoc tests
In ANOVA, we compare at least 3 means in the test. If the result is significant, it means that at least 2 group means are significantly different from each other. However, we do not know the difference comes from which specific groups. Therefore, we can conduct post-hoc tests to perform one-to-one comparison between any pair of group means (it's like a t-test).
Performing multiple hypothesis tests leads to an inflation of the probability for Type I error (i.e., α, see the lecture slides for details). In order to make sure the overall α (also known as the experiment-wise α) remains at a low level (e.g., α = .05), the p values in post-hoc tests are corrected (typically increased) so that each of the p values can still be compared with the original criterion of α = .05 without inflating the overall α.
Each post-hoc test has its own calculation method to adjust the p-value based on the probability of committing type I error. We are going to introduce some of the common post-hoc tests.
Bonferroni: the most conservative method; typically by multiplying each uncorrected p value by the total number of post-hoc comparisons
Tukey: less conservative than Bonferroni, while still ensuring that the overall α remains at the desired value even for comparisons across all possible pairs of group means
Let's use the demo dataset to demonstrate how to conduct post-hoc tests in jamovi.
Example 3: Now, we want to know students in BA, BBA and BSS have different perceived IQ. Therefore, we conduct an One-Way ANOVA to evaluate the statement. We set perceived IQ as the dependent variable and faculty as the factor.
Example 7.3 OneWayANOVA_Post-hoc.mp4Conclusion/ Interpretation (APA format):
An One-Way ANOVA showed that the difference in perceived IQ was significant, F(2, 197) = 3.88, p = .022, η2 = 0.038.
Post hoc analyses using the Tukey post hoc criterion for significance indicated that
the average perceived IQ was significantly higher in BA students (M = 74.4, SD = 12.2) than that in BSS students (M = 69.0, SD = 11.3), t(197) = 2.78, p = .016,
there was no difference in the average perceived IQ between BSS students and BBA students (M = 71.6, SD = 9.78), t(197) = -1.28, p = .405, and
the average perceived IQ did not differ between BA and BBA students, t(197) = 1.49, p = .298.
Module Exercise
Complete the exercise!
Now, if you think you're ready for the exercise, you can check your email for the link.
Remember to submit your answers before the deadline in order to earn the credits!