ANOVA

Purpose:

An analysis of variance, used to determine if there are any statistically significant differences between the means of three or more independent (unrelated) groups. However, it does not specify "how" they differ from each other (a post-hoc test can answer this!). There are three kinds of ANOVA: a one-way ANOVA (for between-subjects manipulation where the IV has 3+ levels), repeated measures ANOVA (for within-subjects manipulation where the IV has 3 levels), and factorial ANOVA (where there are 2+ IVs).

Note: Between-group manipulation (between-subjects study design) means that each group received a different IV treatment condition (level). Within-group manipulation (within-subjects study design) means that every participant experiences every IV treatment condition (level).

Context Used:

• There must be ONE CATEGORICAL IV with 3+ levels, and ONE CONTINUOUS DV

OR

• There must be TWO OR MORE CATEGORICAL IVs with 2+ levels, and ONE CONTINUOUS DV

Jamovi Walkthrough:

One-way/Between-subjects ANOVA:

Data set-up: Column 1 should be code levels of IV1, Column 2 should be code levels for IV2, and Column 3 should be DV scores.

1. Click "ANOVA"

2. Click "ANOVA"

3. Move the DV into the "Dependent Variable" box

4. Move the IV into the "Fixed Factors" box

5. Check "η2" under "Effect Size"

6. Check "Homogeneity test" in the "Assumption Checks" menu

7. Move the IV into the right box in the "Post Hoc Tests" menu

8. Check "Tukey" under "Corrections" in the "Post Hoc Tests" menu

9. Check "Cohen's d" under "Effect Size" in the "Post Hoc Tests" menu

Repeated Measures/Within-subjects ANOVA:

Data set-up: Column 1 should be DV scores for IV level 1, Column 2 should be DV scores for IV level 2, and Column 3 should be DV scores for IV level 3.

10. Click "ANOVA"

11. Click "Repeated Measures ANOVA"

12. Label the IV in the bolded text box under "Repeated Measures Factors"

13. Label the IV levels in the remaining text boxes under "Repeated Measures Factors"

14. Move the IV levels from the left box into the corresponding box under "Repeated Measures Cells"

15. Label the DV in the text box under "Dependent Variable Label"

16. Check "η2" under "Effect Size"

17. Check "Homogeneity test" in the "Assumption Checks" menu

18. Check "None" and "Greenhouse-Geisser" under "Sphericity corrections" in the "Assumption Checks" menu

19. Move the IV into the right box in the "Post Hoc Tests" menu

20. Check "Tukey" under "Corrections" in the "Post Hoc Tests" menu

Factorial ANOVA:

Data set-up: Column 1 should be code levels of IV1, Column 2 should be code levels for IV2, and Column 3 should be DV scores.

1. Click "ANOVA"

2. Click "ANOVA"

3. Move the DV into the "Dependent Variable" box

4. Move the IV into the "Fixed Factors" box

5. Check "η2" under "Effect Size"

6. IF the IV has 3+ levels, OR you reject the null hypothesis: Move the IV into the right box in the "Post Hoc Tests" menu and check "Tukey" under "Corrections" in the "Post Hoc Tests" menu

7. Click "Exploration"

8. Click "Descriptives"

9. To get cell means, move the DV into the "Variables" box, and move all IVs into the "Split by" box.

10. Click "Exploration"

11. Click "Descriptives"

12. To get marginal means, move the DV into the "Variables" box, and move ONYL ONE IV into the "Split by" box.

13. Repeat steps 11-13 for the other IV.

Output Interpretation:

Levene's Test: Test homogeneity of variances assumption

If p < 0.5, Assumption violated, select Welch's test under "Variances" to continue

If p > 0.5, Assumption satisfied (this is what we want, continue as normal)

f-value: The ratio of variation between sample means to variation within the samples.

If the f-value is high, there is high variability and a lower p-value.

If the f-value is low, there is low variability.

p-value: The probability of detecting a meaningful relationship/difference when there is none. We are typically looking for a small value (p < 0.5).

If p < 0.5, reject the null hypothesis. There IS a difference.

If p > 0.5, accept the null hypothesis. There is NO difference.

η2: The proportion of variance in the DV caused by the IV. Even 20% is a large amount for human subjects research.

"X% of variance in the DV is caused by the IV"

Post-Hoc Comparisons:

Cohen's d: A measure of effect size, which measures the SIZE of the difference between two groups.

If d is 0.20-0.49, the effect size is small.

If d is 0.50-0.79, the effect size is medium.

If d is 0.8+,  the effect size is large.

p-value: The probability of detecting a meaningful relationship/difference when there is none. Looking for a small value (p < 0.5)

If p < 0.5, reject the null hypothesis. There IS a difference between this pairwise comparison.

If p > 0.5, accept the null hypothesis. There is NO difference between this pairwise comparison.

APA Format:

Appropriate data visualization: Bar graphs (with error bars)

Sample table: https://apastyle.apa.org/style-grammar-guidelines/tables-figures/sample-tables#anova

Sample write-up:

A one-way ANOVA compared the average number of apples eaten by participants in high school, their freshman year of college, and their senior year of college. This test was found to be statistically significant at an alpha level of .05, F(df,df)=X p<0.5, η2=   . A Tukey HSD test indicated that the average number of apples eaten by high school students  (M=  SD=) were significantly greater than apples eaten by college freshmen (M=  SD=) and apples eaten by college seniors  (M=  SD=). The average number of apples eaten by college freshmen and college seniors did not differ.

Note: plugin the appropriate test used, means, standard deviations, whether the test was significant, f-value, df, alpha and p-level, the strength of the relationship (η2), and Tukey Test results.