statrefs home‎ > ‎Main‎ > ‎Methods‎ > ‎Regression‎ > ‎Regression Supporting Concepts‎ > ‎

Definitions of Linear

There seem to be different ways of interpreting what is meant by "linear" in linear regression.  One is more common, and one is more correct in terms of the statistical theory behind the method. 

Interpretation 1 (most common):
  • "Linear" means "linear in the predictor variable", where the form of the explanatory variable has an exponent equal to 1.

E(Y) = B0 + B1*x1       where E(Y) is the expected value of Y

Interpretation 2 (most statistically relevant):
  • "Linear" in this sense means that the expected value of the response is a linear function of the parameters, or that it is "linear in the betas". 

Intrinsically linear regression models include:
  • polynomial models, and
  • models that are nonlinear in the parameters, but that can transformed into a model that is linear in the parameters.

A model that is linear in the parameters and is also linear in the predictor variable is called a first-order model.


Examples of models that are "linear in the betas" include:
E(Y) = e(B0 + B1*x)  which can be transformed to  ln(y) = B0 + B1*x 
E(Y) = B0 + B1*x1 + B2*x2  where  x1 and x2 can take many forms, such as x2 or log(x)
E(Y) = B0 + B1*x + B2*x2 is a linear statistical model where E(Y) is a 2nd order polynomial function of the independent variable "x", with x1 = x and x2 = x2.
E(Y) = B0 + B1*log(x) is also a linear model.

An example of "not linear in the betas":
E(Y) = B0 + B1e(B2*x) 
An important difference of nonlinear models (Kutner chapter 13 p. 513) ...
"... is that the number of regression parameters is not necessarily directly related to the number of the explanatory (X) variables in the model.  In linear regression models, if there are "p-1" explanatory (X) variables in the model, then there are "p" regression coefficients in the model.  For the exponential regression model above, there is one explanatory variable but there are three regression coefficients."

Another very important aspect of "non-linearity" is that any transform that might be needed does not affect the error terms (ei terms) and their assumptions of independent, identically distributed (iid).

(See StatSoft for other examples of intrinsically nonlinear models.  Also see the NIST reference above.)