statrefs home‎ > ‎Main‎ > ‎Fundamentals‎ > ‎Types of Errors‎ > ‎

Inflating Type 1 Error

Inflating experiment-wise Type 1 error

Interesting link demonstrating the effect -

What does it mean to inflate Type 1 error?
  • If we run multiple individual hypothesis tests on the same set of data, each test has an alpha; for this example, alpha = 0.05.
  • If we run twenty statistical comparisons, each with alpha = 0.05 (1/20), then we have a much larger chance of experiencing Type 1 error. 
    • (Wrongly concluding that an effect or difference exists due to sampling error.)
One example would be to run multiple t-tests to compare many potential factors instead of using an ANOVA-based method that incorporates all comparisons of-interest in the same study.

More information is available in the StatNotes paper attached on the bottom of the page for ANOVA post-hoc tests.