statrefs home‎ > ‎Main‎ > ‎Fundamentals‎ > ‎Types of Errors‎ > ‎

Inflating Type 1 Error


 
Inflating experiment-wise Type 1 error

Interesting link demonstrating the effect - http://xkcd.com/882/


What does it mean to inflate Type 1 error?
  • If we run multiple individual hypothesis tests on the same set of data, each test has an alpha; for this example, alpha = 0.05.
  • If we run twenty statistical comparisons, each with alpha = 0.05 (1/20), then we have a much larger chance of experiencing Type 1 error. 
    • (Wrongly concluding that an effect or difference exists due to sampling error.)
 
One example would be to run multiple t-tests to compare many potential factors instead of using an ANOVA-based method that incorporates all comparisons of-interest in the same study.

More information is available in the StatNotes paper attached on the bottom of the page for ANOVA post-hoc tests.




Comments