See the attached document (at the bottom of this page) for thoughts and guidance on model assumptions. See also the subpages:
Here is the introduction to the document:
Most statistical methods have a theoretical framework, and corresponding assumptions that need to be met before any results obtained by using the method can be considered valid.
The assumptions behind a statistical model or method need to be tested as a part of the overall analysis and included in the report. Diagnostic tests are available for nearly all model assumptions.
If the results of a diagnostic test of model assumptions show that the assumptions are met, then this supports the claim of validity of the results obtained using the statistical method.
If the results of a diagnostic test of model assumptions show that the assumptions are NOT met, then the validity of the results obtained by using the model are in question.
For data that has heavy tails, a violation of model assumptions often means that the statistical method is underpowered … that, if an effect exists, the method is less likely to find it.
This protects from Type 1 error but increases Type 2 error. In these cases where the data has heavy tails, equivalent nonparametric methods often offer more statistical power for the same sample sizes. (See the paper providing a summary of nonparametric methods at link.)
Some statistical methods are fairly robust against violations of certain assumptions. For example:
The significance level of the t-test is somewhat robust against departures from the assumption of normality. The power of the t-test is not.
ANOVA is somewhat robust against the assumption of equal variances.
Methods to verify the robustness of a statistical method against violations of an assumption include:
Perform a corresponding nonparametric test. If the results and conclusions are equivalent, then there may be adequate robustness.
Perform a simulation to test the limits of robustness of an assumption.
The results of diagnostic tests of model assumptions should be included in any report. While a simple statement that model assumptions were tested and were found to be valid is necessary, it may not be sufficient – particularly in a regulated industry. The results of the diagnostic tests should also be included in a report in a manner that supports reproducibility of the results.
When using inferential methods, one common assumption is that the sample represents the characteristics of the population. This is facilitated by ensuring that the sample is randomly drawn from the population, and that any necessary stratification is considered in constructing the sampling plan.
Common areas where model assumptions seem to be routinely ignored include applications with data that has autocorrelation (requiring time series methods), financial data (higher volatility with heavy tails, often autocorrelated), and even basic regression methods.
One good reference (of several) is “Common Errors in Statistics (and How to Avoid Them)”, published by Wiley Press, authored by Phillip I. Good.
When in doubt, consult with a (degreed) statistician.