ANOVA
Purpose:
An analysis of variance. This test will answer if the group means differ, but not "how" they differ (although, a post-hoc test can answer this!). There are three kinds of ANOVA: a one-way ANOVA (for between-subjects manipulation where the IV has 3+ levels), repeated measures ANOVA (for within-subjects manipulation where the IV has 3 levels), and factorial ANOVA (where there are 2+ IVs).
Note: Between-group manipulation (between-subjects study design) means that each group received a different IV treatment condition (level). Within-group manipulation (within-subjects study design) means that every participant experiences every IV treatment condition (level).
Context used:
There must be ONE CATEGORICAL IV with 3+ levels, and ONE CONTINUOUS DV
OR
There must be TWO OR MORE CATEGORICAL IVs with 2+ levels, and ONE CONTINUOUS DV
Jamovi Walkthrough:
One-way/Between-subjects ANOVA:
Data set-up: Column 1 should be code levels of IV1, Column 2 should be code levels for IV2, and Column 3 should be DV scores.
Click "ANOVA"
Click "ANOVA"
Move the DV into the "Dependent Variable" box
Move the IV into the "Fixed Factors" box
Check "η2" under "Effect Size"
Check "Homogeneity test" in the "Assumption Checks" menu
Move the IV into the right box in the "Post Hoc Tests" menu
Check "Tukey" under "Corrections" in the "Post Hoc Tests" menu
Check "Cohen's d" under "Effect Size" in the "Post Hoc Tests" menu
Repeated Measures/Within-subjects ANOVA:
Data set-up: Column 1 should be DV scores for IV level 1, Column 2 should be DV scores for IV level 2, and Column 3 should be DV scores for IV level 3.
Click "ANOVA"
Click "Repeated Measures ANOVA"
Label the IV in the bolded text box under "Repeated Measures Factors"
Label the IV levels in the remaining text boxes under "Repeated Measures Factors"
Move the IV levels from the left box into the corresponding box under "Repeated Measures Cells"
Label the DV in the text box under "Dependent Variable Label"
Check "η2" under "Effect Size"
Check "Homogeneity test" in the "Assumption Checks" menu
Check "None" and "Greenhouse-Geisser" under "Sphericity corrections" in the "Assumption Checks" menu
Move the IV into the right box in the "Post Hoc Tests" menu
Check "Tukey" under "Corrections" in the "Post Hoc Tests" menu
Factorial ANOVA:
Data set-up: Column 1 should be code levels of IV1, Column 2 should be code levels for IV2, and Column 3 should be DV scores.
Click "ANOVA"
Click "ANOVA"
Move the DV into the "Dependent Variable" box
Move the IV into the "Fixed Factors" box
Check "η2" under "Effect Size"
IF the IV has 3+ levels, OR you reject the null hypothesis: Move the IV into the right box in the "Post Hoc Tests" menu and check "Tukey" under "Corrections" in the "Post Hoc Tests" menu
Click "Exploration"
Click "Descriptives"
To get cell means, move the DV into the "Variables" box, and move all IVs into the "Split by" box.
Click "Exploration"
Click "Descriptives"
To get marginal means, move the DV into the "Variables" box, and move ONYL ONE IV into the "Split by" box.
Repeat steps 11-13 for the other IV.
Output Interpretation:
Levene's Test: Test homogeneity assumption
If p < 0.5, Assumption violated, select Welch's test under "Variances" to continue
If p > 0.5, Assumption satisfied (this is what we want, continue as normal)
f-value: Variation between samples means/variation within the samples.
If the f-value is high, there is high variability and a lower p-value.
If the f-value is low, there is low variability.
p-value: The probability you detect a meaningful relationship/difference when there is none. We are typically looking for a small value (p < 0.5).
If p < 0.5, reject the null hypothesis. There IS a difference.
If p > 0.5, accept the null hypothesis. There is NO difference.
η2: Percent of variance caused by the treatment. Even 20% is a large amount for human subjects research.
"X% of variance in the DV is caused by the IV"
Post-Hoc Comparisons:
Cohen's D: A measure of effect size, which measures the SIZE of the difference between two groups.
If d is 0.20-0.49, the effect size is small.
If d is 0.50-0.79, the effect size is medium.
If d is 0.8+, the effect size is large.
p-value: The probability you detect a meaningful relationship/difference when there is none. Looking for a small value (p < 0.5)
If p < 0.5, reject the null hypothesis. There IS a difference between this pairwise comparison.
If p > 0.5, accept the null hypothesis. There is NO difference between this pairwise comparison.
APA Format:
Appropriate data visualization: Bar graphs (with error bars)
Sample table: https://apastyle.apa.org/style-grammar-guidelines/tables-figures/sample-tables#anova
Sample write-up:
A one-way ANOVA compared the average number of apples eaten by participants in high school, their freshman year of college, and their senior year of college. This test was found to be statistically significant at an alpha level of .05, F(df,df)=X p<0.5, n²= . A Tukey HSD test indicated the average number of apples eaten by high school students (M= SD=) were significantly greater than apples eaten by college freshmen (M= SD=) and apples eaten by college seniors (M= SD=). The average number of apples eaten by college freshmen and college seniors did not differ.
Note: plugin the appropriate test used, means, standard deviations, whether the test was significant, f-value, df, alpha and p-level, the strength of the relationship (n²), and Tukey Test results.