statrefs home‎ > ‎Main‎ > ‎Methods‎ > ‎Regression‎ > ‎Multiple Linear Regression‎ > ‎

Multicollinearity




(Contributed by George S.)

Collinearity of the predictors variables can also cause Ill-conditioned matrices. In other words, if one predictor is linearly related to another predictor, then you will have a ill-conditioned matrix

From StatSoft's online textbook (http://www.statsoft.com/textbook/multiple-regression/#amulticollinearity)
  • Imagine that you have two predictors (X variables): (1) weight in pounds and (2) weight in ounces. Obviously, our two predictors are completely redundant; weight is one and the same variable, regardless of whether it is measured in pounds or ounces. If you were to attempt to regress these two variables to a persons height, then you would have an ill-conditioned matrix since one weight variable (say pounds) is linearly related to the other weight variable (ounces).

Three ways to check multicollinearity:
  • check pairwise correlations between all the predictor variables
  • compute the Variance Inflation Factor statistic
  • mean center all predictor variables.
See page 16 here for more info


See http://www.statisticalhorizons.com/multicollinearity for an interesting article of when multicollinearity may not be an issue.
 

 





Comments