09: Confidence Intervals
"Most quantities of interest, most projections, most numerical assessments are not point estimates. Rather they are rough distributions — not always normal, sometimes bi-modal, sometimes exponential, sometimes something else." - John Allen Paulos."Variation is the hard reality, not a set of imperfect measures for a central tendency. Means and medians are the abstractions." — Stephen Jay Gould.
"Although this may seem a paradox, all exact science is dominated by the idea of approximation." - Bertrand Russell.
Lecture outline: study the issues relating to the confidence in measurements of sampled data.
1. Intuition of confidence intervals
Population and Sampling distribution
Z-distribution vs. t-distribution
Point estimates and Interval estimates
Interval estimates: Margin of error
Critical values: t* and z*
Sample size for desired margin of error
90%, 95%, 99% confidence intervals
2. Types of confidence intervals
Confidence intervals of sample mean
Confidence intervals of proportion
Confidence intervals of difference of mean (2 samples)
Primary reference for this lecture:
1. “The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling” by Raj Jain; Chapter 12: “Summarizing Measured Data”
2. "Statistics for Experimenters", Box and Hunter.
3. "Probability and Statistics with Reliability, Queueing and Computer Science Applications", Kishor Trivedi, Chapter 10: Statistical Inference
Secondary references for this lecture:
1. “Measuring Computer Performance: A Practitioner's Guide” by David Lilja; Chapter 4: “Errors in Experimental Measurements”
2. “Performance Evaluation of Computer and Communication Systems” by Le Boudec, Chapter 2: “Summarizing Performance Data, Confidence Intervals”
3. “Cartoon Guide to Statistics” by Larry Gonick; Chapter 7: “Confidence Intervals”