What do
the experts have to say about checking model assumptions?

The notes below contain excerpts from many references on various statistical methods and topics, written by credible authors and recognized experts.

Chapter 3 (page 100)

When a
regression model is considered for an application, we can usually not be
certain in advance that the model is appropriate for that application. Any one, or several, of the features of the
model, such as linearity of the regression function or normality of the error
terms, may not be appropriate for the particular data at hand. Hence, it is important to examine the aptness
of the model for the data before inferences based on that model are undertaken.

Chapter 3 (page 76)

The
decomposition of the variability in the observations through an analysis of
variance identity is a purely algebraic relationship. However, the use of the partitioning to test
formally for no differences in treatment means requires that certain
assumptions be satisfied. Specifically,
these assumptions are that the observations are adequately described by the
model [ y_{ij} = mu + tau_{i} + e_{ij} ] and that the errors are normally and
independently distributed with mean zero and constant but unknown
variance. If these assumptions are
valid, the analysis of variance procedure is an exact test of the hypothesis of
no difference in treatment means. In
practice, however, these assumptions will usually not hold exactly. Consequently, it is usually unwise to rely on
the analysis of variance until the validity of these assumptions has been
checked. Violations of the basic
assumptions and model adequacy can be easily investigated by the examination of
residuals. […] Examination of the residuals should be an
automatic part of any analysis of variance.

Chapter 3 (page 83)

The ANOVA
techniques considered […] are theoretically based on independence, random
samples, normal distributions, and equal population variances. In practice, we do not expect the model
assumptions to be satisfied exactly. For
the procedures to yield reliable results, however, those assumptions must be
reasonably satisfied.

Chapter 6 (page 182)

If we
could rely on the exactness of the IID assumption, we could say that after
these [analysis of variance] statistics
had been calculated no further relevant information remained in the
original data, and we could, therefore, ignore the residuals and the original observations
and concentrate entirely on the interpretation of these statistics. In practice it would be very unwise to do
this without further checks because data may contain valuable information not
allowed for in the assumed mathematical model and therefore not revealed by the
associated analysis of variance table.
[…] Discrepancies of many kinds
between a tentative model and the data can be detected by studying residuals. […]
When assumptions concerning the adequacy of the model are true, we expect
to find that, apart from restrictions introduced by the analysis itself, the
residuals vary randomly. If, however, we
find that the residuals contain unexplained systematic tendencies, we shall be
suspicious of the model. Therefore, as
an automatic preliminary to further statistical analysis, a table of residuals
should always be constructed and studied.
[particular discrepancies] should be looked for as a matter of routine.

Chapter 4 (page 58)

The
decomposition of the total variability in the observations through an ANOVA is
purely algebraic. However, the use of
ANOVA to construct confidence intervals requires that certain assumptions on
the observations be satisfied. In
general, ANOVA procedures are relatively robust to these underlying
assumptions. In experiments where some
or all of the factors are random (as is frequently the case in gauge R&R
experiments), the assumptions are more critical. […] In
practice these assumptions usually will not hold exactly. Consequently, it is usually unwise to rely on
the ANOVA results without carefully checking the validity of these
assumptions. Violations of the basic
assumptions and model adequacy can be easily investigated by the examination of
residuals.

Chapter 2
(page 29)

(Regarding
SPC charts) The most critical assumption
is that successive deviations are independent, so that variation within
subgroups can tell us what the variation of the averages between subgroups
should be.

Chapter 2
(page 51)

[…] ARLs
are extremely sensitive to violations of assumptions.

Chapter 3
(page 61)

Difficulties
frequently occur in the application of the control methods [discussed]. They relate to the inadequate approximations
that underlie the use of these techniques and, in particular, the supposition
that noise can be modeled as a stationary random variation about a fixed mean.

All
models are approximations. Assumptions,
whether implied or clearly stated, are never exactly true. All models are wrong, but some models are
useful. So the question […] is not “Is
the model true?” (it never is) but “Is the model good enough for this
particular application?” The answer
depends on two factors: (1) How close to
the reality the assumptions are likely to be, and (2) How robust (insensitive)
the derived statistical methods are to these likely departures from assumption.

Chapter 3
(page 63)

So in the
practice of quality control charting or in the application of any other
statistical method, you must ask: What
are we assuming? How large can we expect
deviations to be from these ideal assumptions?
Will the conclusions be seriously affected by such deviations?

Chapter 3 (page 137)

The
analysis of variance assumes that the model errors (and as a result, the
observations) are normally and independently distributed with the same variance
in each factor level. These assumptions
can be checked by examining the residuals.

Chapter 1
(page
20-21)

Care
should be used with [process capability] indices […]. Statistical properties and assumptions should
be understood and met before using them, and index estimations should be
qualified via confidence intervals.

(page 22)

Care
should always be taken to ensure that the underlying assumptions are met […].

Reliability: Statistical Methods for Reliability Data (Meeker)

Chapter 17 (page 443)

An
important part of any statistical analysis is diagnostic checking for
departures from model assumptions. In
conducting a failure-time regression analysis we recommend the use of graphical
methods, using generalizations of usual regression diagnostics (including
residual analysis). These diagnostic
methods can be used in a manner that is similar to their use in ordinary
regression analysis, except that interpretation is often complicated by the
censoring. The analysis can also be
complicated when fitting underlying nonnormal distributions.

Chapter 1 (page 14)

Of
course, any quantitative analysis relies upon a series of assumptions. When the conditions behind the assumptions no
longer apply, the analyses should no longer be employed.

## Assumptions behind lot-based sampling inspection plans.

See the
paper “**Statistical Methods and Regulatory Compliance - Sampling Plans.doc**”.

(view document here)

(view as web page)

(Role of Statistics in QMS - paper is attached at the bottom of this page)

(Chapter
1, page 3)

The sources
of error in applying statistical procedures are legion and include all of the
following: […] Failing to validate models.

(Chapter
1, page 4)

Here is a
partial prescription for the error-free application of statistics.

(7)
Know
the assumptions that underlie the tests that you use. Use those tests that require the minimum of
assumptions and are most powerful against the alternatives of interest.

(Chapter
5, page 51)

Every
statistical procedure relies on certain assumptions for correctness. Errors in testing hypotheses come about
either because the assumptions underlying the chosen test are not satisfied or
because the chosen test is less powerful than other competing procedures.

[… page
52 …]

[B]efore
we choose a test statistic, we check which assumptions are satisfied, which
procedures are most robust to violation of these assumptions, and which are
most powerful for a given significance level and sample size.

(page 78)

An
experienced data handler develops a nose for strange features of data and
statistical summaries. An odd pattern,
irregularity, or coincidence is often a clue that something fishy may be going
on – a mistake in recording the data, a statistical miscalculation, a departure
from the assumptions and conventions of a particular type of analysis, or -- in
unusual cases -- even downright fraud.

(page 14)

A
traditional assumption made in financial study is that the simple returns are
independently and identically distributed as normal with fixed mean and
variance. This assumption makes
statistical properties of asset returns tractable. But it encounters several difficulties.

(page 44)

A fitted
model must be examined carefully to check for possible model inadequacy. If the [time series] model is adequate, then
the residual series should behave as a white noise. If a fitted model is found to be inadequate,
it must be refined.

(page 193)

Many
things can, and often do, go wrong when data are analyzed. There may be data that were entered
incorrectly, one might not be analyzing the data set one thinks, the variables
may have been mislabeled, and so forth.
[…] Besides problems with the
data, the assumed model may not be a good approximation to reality. [… most] practitioners use regression
diagnostics to detect problems and then attempt to remedy any problems that
were detected. [… the] residuals can be
used to check the assumptions behind regression.

(page 51)

**Assumption [...] must be verified, not merely stated, in practice to ensure good results.**