Where did the Test Matrix come from and what purpose does it serve?
Common statistical methods share a basic structure that ties the application of the method to the underlying theory. (see link)
Origin and Purpose
The Test Matrix was created to promote a standard format for stating a hypothesis and specifying the manner in which it will be tested, that was tied to the underlying theory without requiring everyone to be an expert in the theory behind the method.
The Test Matrix was the result of several years of observations of protocols and experimental designs where many of the necessary details were vague, at best, and often missing.
See attached Test Matrix template (RD-05980_Att1).
See attached Test Matrix template (RD-05980_Att4) with examples and additional information.
The Headers
The headers and entries in the Test Matrix are important. The headers should be used as they are. The following text shows the relationship between the theory and the Test Matrix columns.
Assumptions
These are not covered in the Test Matrix, but should be tested in the analysis and included in the report.
Hypotheses
The column in the Test Matrix is intended to house a statement about the null hypothesis. The word "null" was left out as it confused a lot of people.
Test Statistic
The associated column in the Test Matrix is titled "Statistical Method". The phrase "Test Statistic" was too cryptic for many people, although it is technically more correct.
A given test statistic has an associated null distribution. This, combined with the alpha level, defines the rejection region.
P-value
The associate column in the Test Matrix is titled "p-value". This is the p-value calcuated from the test statistic.
There are cases where another type of test statistic might be reported, such as a value of Cpk or UPF.
Conclusion
Even though the p-value calculated from test statistic is reported, we still need a statement regarding whether or not the a-priori action standard (documented in the Hypothesis) was met.
The Entries
For non-expert users, the entries in the examples in the attached template were created to ensure appropriate wording that relates directly to the theory of the chosen statistical method.
Because these example entries have specific meaning and purpose, they should be used as they are.