ANOVA: Comparing 3+ groups

When do we use ANOVA?

When your experimental design contains multiple (3+) groups, we use Analysis of Variance (ANOVA) instead of a t-test to statistically compare them. ANOVA compares all groups at once. You may think that doing many t-tests to compare all possible pairs of means would be a good approach, but so doing would increase the risk of concluding a significant difference when none actually exists.

A significant ANOVA p-value tells you that the null hypothesis is rejected (that there is a significant difference among your groups). However, you do not know which group or groups are causing this to be true. Therefore, if the ANOVA is significant, it can be followed by a Tukey's post hoc test to examine differences between each possible pair of groups. The Tukey's pairwise comparisons are displayed as letters on the column graph: If two groups share a letter, it means they are not significantly different from each other. See an example of a figure displaying ANOVA and Tukey's post hoc comparisons below:

Example: This experiment examined the development of the immune system in house sparrows. Researchers took blood samples and measured the level of IgY antibodies circulating in the blood. Birds were sampled at a range of ages from 3 days after hatching through adulthood. (Killpack, Oguchi, & Karasov, 2013).

The ANOVA test was significant, which told the researchers that at least one pair of age groups differed. A Tukey's test was run to learn which pair(s) significantly differed. Examine the letters above the bars in the figure. Which age groups significantly differed in antibody levels?

Performing an ANOVA test

Watch this video overview about performing ANOVA and follow the steps provided below

1. Using Excel, calculate the mean and standard deviation from the replicates for each of your experimental groups. You will also need to determine the sample size (N) for each group by counting the number of replicates in each group.

This example shows the data from a study that aimed to compare species richness (# of different species) in Salem, Nahant, and Lynn. Replicate samples (N) were collected from each location and the mean and standard deviation of species richness was calculated.

2. Navigate to the webpage http://statpages.info/anova1sm.html for the ANOVA statistics calculator. Enter your data for Group name, N, mean, and Std Dev for each group in the appropriate cells. Keep the default confidence level of 95%. Click 'Compute.'

3. Analyze the ANOVA results.

3a. The p-value in the ANOVA table tells you the overall significance. If there is at least one significant difference among the groups, then the p-value will be <0.05.

In the ANOVA table we see that p=0.0104, indicating that the there is at least one pair of experimental locations that differs significantly. We move on to look at the Tukey's post-hoc tests...

3b. If the p-value in the ANOVA table is significant (less than 0.05), then look at the Post-hoc tests to see which pairs of experimental groups significantly differ from each other. A p-value will be provided for each possible pairwise comparison between experimental groups.

In the Post-hoc tests data we see each pairwise comparison between locations.

Salem vs. Nahant p=0.0092, indicating a significant difference in species richness
Salem vs. Lynn p=0.6216, indicating no significant difference in species richness
Nahant vs. Lynn p=0.1182, indicating no significant difference in species richness

4. If you would like, you can add letters to your figure based on the ANOVA and Tukey's post-hoc results. Remember the rule that if two groups share a letter, it means they are not significantly different from each other. It's a fun puzzle to determine which letters to put above the bars based on the p-values!

Based on our data, Salem and Nahant should not share a letter. Let's label Salem "a" and Nahant "b" to indicate that they are significantly different from each other.

Based on our data, Lynn should share a letter with Salem. Lynn should also share a letter with Nahant. So, we will label Lynn as "a, b" to indicate that it does not significantly differ from either Salem or Nahant.

4a. Copy your figure from Excel and paste into Word.

4b. To add labels to your graph in Word, go to the top menu and click on 'Insert' and then scroll down to choose 'Text Box.' Click on your graph to drag a text box that is the length of the bars.

4c. Add the appropriate letters above each bar, according to your ANOVA and post-hoc data.

Describing ANOVA and Tukey's post-hoc results in text

Watch this video overview about writing ANOVA paragraphs and follow the steps provided below

Results sections include 3 important components:

Figure
- Properly labeled with error bars
Figure caption
- Numbered in order of appearance in the text
- Briefly describe the data presented in the graph. Make sure it includes:
  - Your model organism for the experiment
  - Your experimental groups (independent variable) displayed on the X-axis
  - What you measured (dependent variable) displayed on the Y-axis
  - That the columns represent means and the error bars represent standard deviations
Results text paragraph
- Briefly introduce the experiment that is depicted in your figure.
- State the data pattern presented in the figure, including whether the comparison was significant.
- In parentheses: results of the statistical test
- In parentheses: a reference to the figure where the comparison is displayed

Google Sites