Parametric Hypothesis Tests, Part 1:
Z-tests and t-tests
The total length of the videos in this section is approximately 42 minutes. You will also spend time answering short questions while completing this section.
You can also view all the videos in this section at the YouTube playlist linked here.
Please note: You have likely heard of t-tests and the surrounding concepts in your intro stats course. These videos are intended for students who (like you) have already been exposed to these ideas at an introductory level. They move quickly. Don't skip this!
Using the CLT to generate a reference distribution
ParamHypTestsP1.1.Hypothesis Tests.mp4Parametric tests (like t-tests and Z-tests - they have parameters like μ and σ) differ from non-parametric tests and several ways. In general, you might need a non-parametric test if you are concerned about the assumptions underlying the parametric tests due to a small sample size or any other reason. The questions below underline some of the differences between the two types of tests.
Question 1: Which is the null hypothesis of a randomization test?
Population means are the same
Population distributions are the same
Show answer
The null hypothesis of a randomization test (including the rank sum, the sign test, the signed rank test, etc.) is that the population distributions are the same. In contrast, t-tests use null hypotheses related to population means.
Question 2: Which uses the set of all possible allocations of units to groups as the reference distribution?
Randomization/permutation test
t-test
Show answer
Randomization/permutation test
Question 3: Which allows you to choose any test statistic you like?
non-parametric tests
parametric tests
Show answer
Non-parametric tests
Question 4: Which assumes normal population distributions?
t-tests
rank sum tests
Show answer
t-tests
Z-tests
ParamHypTestsP1.2.Z-Test.mp4Question 5: Which of these test statistics will lead to the same p-value? Check all that apply.
x̄
x̄ − 0
(x̄ − 0) / (σ / √n)
Show answer
All three test statistics will lead to the same p-value. For each, we can find a normal distribution to use as the reference distribution using the CLT.
Standard normal distribution
ParamHypTestsP1.3.Standard Normal Distribution.mp4Question 6: What is the standard deviation of the standard normal distribution?
Show answer
1
Toward the t-test: sample variance
ParamHypTestsP1.4.S^2.mp4Question 7: Which are reasons for n-1 to appear in the denominator of the sample variance, rather than n? Check all that apply.
One piece of information has been used up to estimate the population mean
Calculating deviations from the estimated mean instead of the population mean increases the overall uncertainty
On average over all possible samples, we want the sample variance to be equal to the population variance
Show answer
All 3.
Question 8: Which are different for various possible samples from a population?
μ
x̄
σ
s
Show answer
x̄ and s. In contrast, μ and σ are population parameters, not statistics that you can calculate based on a sample.
One-sample t-test
ParamHypTestsP1.5.T-Test.mp4Question 9: Which of the following assume that the population distribution is normal? Check all that apply.
Z-test
t-test
Show answer
t-test. The only random element of the Z-statistic is the sample mean, and the reference distribution for the sample mean follows from the Central Limit Theorem: it's normal, regardless of the population distribution. The t-statistic includes both the sample mean and the sample variance, and so the CLT isn't enough - in order for the t-statistic to exactly follow the t-distribution, we need the population distribution to be normal. As we'll see, though, this assumption of normality does not turn out to be crucial.
Question 10: Suppose the test statistic is equal to 1.5. Would the p-value be bigger if this test statistic came from
a Z-test
a t-test
Show answer
t-test
A t-distribution has longer tails than a normal. You can think of it as if we pushed down on top of the normal distribution and spread it out a little. So, if we ask what proportion of the curve is greater than 1.5, there will be more of the t-distribution >1.5 than the normal.
Two-sample test
ParamHypTestsP1.6.2 Sample Z-Test.mp4Question 11: What is the null hypothesis of a two-sample Z-test?
Show answer
The population mean in the first group is equal to the population mean in the second group.
Two-sample t-test, unpooled
ParamHypTestsP1.7.2 Sample t-test Unpooled Welch.mp4Question 12: Is the null hypothesis of a two-sample t-test different from the null hypothesis of a two-sample Z-test?
Show answer
No.
Two-sample t-test, pooled
ParamHypTestsP1.8.2 Sample t-test Pooled.mp4Question 13: Why are we calculating a pooled sample variance instead of just using the two separate within-group sample variances?
Show answer
We are assuming that the true variance is the same in the two populations. So, we should use all of our data to estimate that one number.
Nice job.
During this tutorial you learned:
More about the Central Limit Theorem and its connection to hypothesis testing
About the difference between parametric tests and non-parametric tests
The definition of a standard normal distribution
How to perform a Z-test and find a Z-statistic
How to estimate the population variance with the sample variance
That sample variance is unbiased for the population variance
How to conduct a one-sample t-test
How to conduct an unpooled two-sample t-test and pooled two-sample t-test, including assumptions
How to calculate the pooled sample variance
Terms and concepts:
Parametric test, Z-test, standard normal distribution, sample variance, degrees of freedom, one-sample t-test, t-statistic, reference distribution, unpooled (or Welch or unequal variance) two-sample t-test, pooled two-sample t-test