Linear Regression
General Notes
What is meant by "linear" regression? (Definitions of Linear)
See the documents attached at the bottom of this page.
Stat Notes - Linear Regression Basics.pdf
Stat Notes - Linear Regression Inferences.pdf
Simple Linear Regression
Multiple Linear Regression
Polynomial Regression
Nonlinear Regression
Assumptions
For standard regression methods, the values of the explanatory variable ("x") are assumed to be constants, meaning that "x" is not a random variable, but instead has known values. If x and y are both random variables, then the model is not strictly a regression model, but is rather a bivariate distribution (possibly bivariate normal distribution).
The main assumptions include:
The residuals are "iid" N(0,s2).
This means that the model residuals are independent and identically distributed ["iid"], having a Normal distribution with mean = 0 and constant variance across the domain of the regression [N(0,s2)].
The term "independent" means that the value of the residual at one point is not influenced by the value of the residuals at any other points. Residuals that are not independent are often autocorrelated.
Anytime regression methods (or any statistical methods) are used, the model assumptions should be checked using standard diagnostic measures for linear regression.
References
Applied Linear Regression Models (Kutner)
See also NIST for more information on model assumptions.