21: One-Factor Experiments

"To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of." - Ronald Fisher."The world’s best statistical analysis cannot rescue a poorly planned experimental program." - Gerry Hahn."It (ANOVA) is probably the most useful technique in the field of statistical inference." – Doug Montgomery (author of ‘Design and Analysis of Experiments’).

Lecture outline: how to conduct experiments in which one-factor is changed at a time?

1. Comparing multiple treatments

Comparing two samples (t tests)

ANOVA: used for comparing more than two treatments

2. ANOVA example

One-factor three-level comparison example

F-distribution and F-statistic

Follow up t tests.

Statistical vs. Practical significance.

3. ANOVA model

ANOVA means model

ANOVA effects model

Visual tests to validate ANOVA assumptions

Primary reference for this lecture:

“The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling” by Raj Jain; Chapter 20: “One-Factor Experiments”.