Hypothesis testing is a way for researchers to determine whether they will accept or reject the null hypothesis in favor of an alternate hypothesis. P-value is a method of hypothesis testing that provides evidence against the null hypothesis.
P-value Basics:
Although p-values are expressed in decimals, converting to percentages helps to interpret their meaning.
In hypothesis testing, the obtained p-value is compared to the selected alpha level.
As p-value decreases, evidence to support rejection of the null hypothesis grows stronger.
Graphical Interpretation of P-value:
In a probability distribution graph, the p-value is the area in the tail outside of the test statistic.
Alpha Level:
Alpha level is selected by the researcher. If the researcher wants to be 95% confident in their research, then the alpha level is 5% (100% - 95%).
Comparing P-Value to Alpha Level:
If the p-value is less than or equal to the alpha level (p ≤ 0.05), then the null hypothesis is rejected in favor of the alternate hypothesis.
If the p-value is greater than the alpha level (p > 0.05), then the null hypothesis is not rejected.
If the Ho is rejected at the 5% significance level, then the evidence against the Ho is "strong". If the Ho is rejected at the 1% significance level, i.e. p ≤ 0.01, then the evidence against the Ho is "very strong".
Example: 60 students are randomly assigned to a gamified (experimental group, n=30) or traditional (control group, n=30) educational setting. The course content and final exam are the same for both groups. The researcher seeks to compare the mean exam scores for the two groups to determine whether there is a significant difference in outcomes.
Null hypothesis:
Ho: μ1 = μ2
Ho: There is no significant difference between exam scores of the control and experimental groups.
Alternate hypothesis:
H1: μ1 ≠ μ2
H1: There is a significant difference between exam scores of the control and experimental groups.
Level of significance:
α = 0.05
Decision rule:
If the p-value is less than the level of significance (α = 0.05), then we reject the null hypothesis (Ho) in favor of the alternate hypothesis (H1).
Data analysis and test statistic:
Group Control Experimental
Mean 71.67000 75.50000
SD 11.69500 12.75400
SEM 2.13521 2.32855
N 30 30
Unpaired t test results:
P-value and statistical significance:
The two-tailed p-value equals 0.2303 (The p-value is obtained from a table using the t statistic, degrees of freedom, and confidence level to identify the corresponding p-value)
By conventional criteria, this difference is considered to be not statistically significant.
Confidence interval:
The mean of Control minus Experimental equals -3.83000
95% confidence interval of this difference: From -10.15405 to 2.49405
Intermediate values used in calculations:
t = 1.2123
df = 58
standard error of difference = 3.159
Conclusion:
Since the p-value (0.2303) is greater than α = 0.05, we cannot reject the null hypothesis.
