Bias-Variance Trade-off

From An Introduction to Statistical Learning with Applications in R (Hastie Tibshirani) - Section 2.2.2 "The Bias-Variance Trade-Off"

E(y₀ - f-hat(x₀))² = Var(f-hat(x₀)) + [Bias(f-hat(x₀))]² + Var(ɛ)

Quoted: It is possible to show that the expected test MSE, for a given value x₀, can always be decomposed into the sum of three fundamental quantities;

• Variance of f-hat(x₀)

• Squared bias of f-hat(x₀)

• Variance of the error terms ɛ

Dr. Hyndman provides a break-down of the math behind the bias-variance decomposition.

There are some situations where allowing a little bias can provide a useful reduction in variance.