statrefs home‎ > ‎Main‎ > ‎Methods‎ > ‎Regression‎ > ‎

Linear Regression


General Notes

 
What is meant by "linear" regression?  (Definitions of Linear


See the documents attached at the bottom of this page.
  • Stat Notes - Linear Regression Basics.pdf
  • Stat Notes - Linear Regression Inferences.pdf


Simple Linear Regression
Multiple Linear Regression
Polynomial Regression
Nonlinear Regression

 

Assumptions

 
For standard regression methods, the values of the explanatory variable ("x") are assumed to be constants, meaning that "x" is not a random variable, but instead has known values.  If x and y are both random variables, then the model is not strictly a regression model, but is rather a bivariate distribution (possibly bivariate normal distribution).


The main assumptions include:

The residuals are "iid" N(0,s2).

This means that the model residuals are independent and identically distributed ["iid"], having a Normal distribution with mean = 0 and constant variance across the domain of the regression [N(0,s2)].

The term "independent" means that the value of the residual at one point is not influenced by the value of the residuals at any other points.  Residuals that are not independent are often autocorrelated.

Anytime regression methods (or any statistical methods) are used, the model assumptions should be checked using standard diagnostic measures for linear regression.
 



References

 
See also NIST for more information on model assumptions.

Arsham - Regression Modeling and Analysis