Reflections on Ridge Regression
Ridge regression refers to a class of biased linear estimators used in a multiple linear regression context when the explanatory variables are highly correlated. In such instances, the optimal least squares estimator often yields regression coefficient estimates that have large variances. In addition, the direction of the least squares estimates may be reversed from prior knowledge and thus rendered meaningless from a practical perspective. Ridge regression is one approach to statistical estimation within this context. The motivations for considering this methodology will be reviewed reflecting the speakers’ experience using such methods for over three decades. Ridge regression estimators are rational functions in the so-called ridge parameter and this property leads to useful insights into the behavior of a ridge trace, i.e., a plot of ridge regression coefficients as a function of the ridge parameter. Several illustrative examples will be used to highlight various properties of the ridge estimator.