20: Experimental Design
"The fundamental principle of science, the definition almost, is this: the sole test of the validity of any idea is experiment." - Richard Feynman."All experiments are designed experiments; unfortunately, some of them are poorly designed" - Douglas Montgomery."If you’re trying to establish cause-and-effect relationships, do try to do so with a properly designed experiment." - Robert Hooke.
Lecture outline: how to build valid experimental designs?
1. Design of experiments (DoE): motivation, history and basic designs
Science, causality, and experiments
Where are designed experiments used? (mainly in agriculture, industry, and quality control; great diversity in application areas)
Importance of using designed experiments
Hypothesis testing and design of experiments.
Some basic experimental designs (Edison’s trial and error method; one-factor-at-a-time (OFAT) designs; Fisher’s factorial designs;
2. BoE basic terminology
Factors (predictor); levels (treatments); experimental unit; replication; interaction; blocking.
Fisher proposed DoE fundamentals: Factorial principle; Blocking; Randomization; Replication.
Common errors of DoE
Example: Design of experiment of the ‘best’ paper helicopter.
Primary reference for this lecture:
“The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling” by Raj Jain; Chapter 16: “Introduction to Experimental Design”.
Secondary references for this lecture:
1. “Cartoon Guide to Statistics” by Larry Gonick; Chapter 10: “Experimental Design”
2. “A First Course in Design and Analysis of Experiments” by Gary Oelhert; Chapter 1: “Introduction”
3. "Modern Engineering Statistics", by Ryan.