Practical v. Statistical Significance

The total length of the videos in this section is approximately 8 minutes.

Practical v. statistical significance

PracticalvStatisticalSignificance.1.PracticalvsStatisticalSignificance.mp4

Question 1: For a particular study, we expect that larger sample size will lead to

  • bigger t-statistic and smaller p-value

  • smaller t-statistic and larger p-value

Show answer

bigger t-statistic and smaller p-value

Question 2: The sample size determines:

  • practical significance

  • statistical significance

Show answer

Statistical significance. The larger the sample size, the larger the t-statistic will be, and the more likely it will be to be statistically significant. The sample size has no bearing on practical significance. Try writing out the expression for a t-statistic, choosing some arbitrary numbers if you like. When you increase n, t gets bigger, which means that the p-value would get smaller, even if the numerator of the t-statistic stays the same.

Question 3: Check all of the statements that are true.

  • Irrelevant but non-zero population differences will be statistically significant, in large enough samples

  • Important population differences may not be statistically significant, but only because the sample is too small

  • p-value cut-offs for rejecting null hypothesis are arbitrary

  • p-value is the probability that the null hypothesis is true

Show answer

The first three, but not the fourth.

That's all.

During this tutorial you learned:

  • The definition of p-value

  • The difference between practical significance vs statistical significance

  • The relationship between a t-statistic and sample size

  • The cutoff threshold of p-values is arbitrary


Terms and concepts:

P-value, practical significance, statistical significance