Regression Analysis
The videos, notes, etc. below are instructional
materials and not to be use commercially. They
are to be used in conjunction with the texts for
the course.
Unit 0: Preliminaries
Permutations and Combinations 10:06
Power Rule for Polynomials 2:56
Moments of Random Variables 8:33
Unit 1: Simple Linear Regression
Linear Correlation Coefficient 17:36
Simple Linear Regression 16:42
Least Squares Regression Coefficients 24:26
Estimating the Variance in the Error Terms 4:55
Bias and Mean Squared Error 14:42
Unit 2: Inferences for Simple Linear Regression
Hypothesis Tests for p bet1 beta0 21:35
ANOVA Approach to Regression 20:01
Unit 3: Diagnostics and Remedial Measures for
Simple Linear Regression
Diagnostics for Residuals 1 9:04
Diagnostics for Residuals 2 22:52
Diagnostics for Residuals 3 4:04
Diagnostics for Residuals 4 9:47
Diagnostics for Residuals 5a 10:02
Diagnostics for Residuals 5b 13:01
Diagnostics for Residuals 5c 3:33
Diagnostics for Residuals 6 10:08
Diagnostics for Residuals 7 5:56
Unit 4: Matrix Algebra
Unit 5: The General Linear Model
Unit 6: Diagnostics
Unit 7: Problems with the Predictors
Unit 8: Problems with the Errors
Weighted Least Squares 1 19:51
Weighted Least Squares 1 notes
Unit 9: Transformations
The first set of notes are mine (and yours). Please work
through those and watch the videos. In addition, refer to
Chapter 9 from Linear Models with R, by Julian Faraway.
Read/work through this chapter with R as needed.
Homework: submit solutions to problems 1, 3, and 6 from
the end of Faraway's chapter on transformation (chapter 9).
Also, submit solutions to at least one of the MLE problems
at the end of the MLE notes. Grad students: choose at least
one additional MLE problem. Anyone (undergrads and grads)
can submit solutions to additional MLE problems for extra credit.
log(y + alpha) and Broken Stick 11:19
Quadratic Regression Excel Spreadsheet (in the files list at the bottom of this webpage)
Unit 10: Building the Regression Model I:
Variable Selection
Please mainly work through my notes and videos below this
week. You will also want to refer to Chapter 10 of Faraway.
However, I have found An Introduction to Statistical Learning
with Applications in R, by James, Witten, Hastie, and Tibshirani
extremely helpful (chapters 2, 5, 6). This is an excellent, excellent
text, and you should feel free to look to it for reference as needed.
You can find a freed pdf version of this text online.
The homework for this week appears AT THE END OF MY
PDF NOTES. Please submit solutions to problems 1, 2, 3, 6, and
either 4 or 5. You may do the other one (5, or 4) for extra credit.
Model Selection Code for Faraway
Unit 11: Building the Regression Model II:
Shrinkage Methods
For this week, please refer to my notes (below, called
ModelBuildingIIShrinkageMethods). Please submit solutions
to all five homework problems that appear at the end of the notes.
These notes (and most examples therein) are taken from
various sources, mainly your Faraway text, Introduction
to Statistical Learning with R, Applied Linear Statistical Models,
Linear Models and Generalizations by Rao and Tautenburg,
the online statistics course notes at Penn State, and
Andrew Ng's machine learning notes (Stanford).
Model Building II: Shrinkage Methods
Shrinkage R Code for Faraway PCR 1
Shrinkage R Code for Faraway PCR 2
Partial Least Squares, Ridge, and Lasso
Unit 12: Categorical Predictors
For homework, open the infmort data from the faraway
package. Read the online documentation for this data
set (do a web search, or type ?infmort at the R prompt
after loading the faraway package). Use R to find a simple
model for the infant mortality rate in terms of the other
variables. Watch out for outliers and influential observations,
(you know ways of detecting these) and definitely check
and correct for any model assumptions that might not
hold (e.g., homoskedasticity). Make plenty of graphs,
and be sure to interpret your model by explaining what
your regression parameters mean in layman's terms. If
you make any kind of transformation, be sure to explain
how your final model should be interpreted.
Categorical Predictors 1 16:29
Categorical Predictors 2 35:36
Categorical Predictors 3 42:41
Unit 13: Logistic Regression
Here's the last part! Please work through the first 10
pages of Chapter 2 of Extending the Linear Model
with R, by Julian Faraway (see the pdf below). Please
do the homework I've posted below and submit to the
dropbox. Again, you won't need to read the entire
chapter to do this homework- just the first few pages.
Please submit this homework any time before the final exam.