Introduction

• Hypothesis testing is a statistical method for us to evaluate whether evidence obtained from a sample is sufficient for rejecting a null hypothesis about the population.

• There are various types of hypothesis tests, with each having its own properties and applicable conditions.

• In this module, we are going to introduce one-sample t-test, one of the most basic types of hypothesis tests.

1. What is one-sample t-test?

We use one-sample t-test when we want to test whether the population mean (μ) is equal to/smaller than/larger than a specific value, when we DO NOT know the value of the population variance.

One-sample t-test can be further divided in two types: directional (also known as one-tailed) or non-directional (also known as two-tailed). 

• A directional or one-tailed t-test is to test whether μ is smaller than or larger than a specific value. Because the hypothesized effect has a specific direction (i.e., larger than or smaller than), a difference in the opposite direction does not count as an "effect".

For example,  we want to know whether obesity is a problem in America. To test the hypothesis, we collect a sample of Americans and measure their BMI (Body Mass Index). We examine whether the average BMI of this sample is larger than 30, which is defined as Obese Class I. In this case, the hypothesis is directional (BMI > 30) and we will use a directional (or one-tailed) one-sample t-test.

• A non-directional or two-tailed t-test is to test whether μ  is different from a specific value. Because the hypothesized effect only concerns whether the difference exists or not, a difference in either direction (larger than or smaller than) will be counted as an "effect".

For example, suppose we are interested in determining whether exercise duration of university students is different from 30 minutes per day. We collect a sample of students and record their exercise durations. We compare the mean exercise durations of the sample to 30 minutes. This is non-directional (or two-tailed) because we are only interested in whether the mean is different from 30 minutes, but not whether it is specifically longer than or shorter than 30 minutes.

2. Degrees of freedom

The t distribution depends on a number called the degrees of freedom (df). We need to know df in order to specify the t distribution we refer to when we conduct a t-test.

For one-sample t-test, the formula of degrees of freedom is:

Degrees of freedom = Sample size - 1

[For your interest: In general, degree of freedom is calculated by [number of known piece of information] minus [number of unknown parameter].  The sample size (n) gives you the information about your data, so you have n pieces of information can be used to estimate the unknown parameter.  The one-sample t-test has only one unknown parameter, that is, the population mean (μ).]

5. Effect size 

• Effect size is the magnitude of an effect relative to the variability in the population.

• It describes the effect in the POPULATION. Therefore, effect size is independent of the details of a specific hypothesis test, such as sample size (n) and alpha level (α).

• Cohen's d is a standardized measure of effect size for indicating the magnitude of a difference effect. 

• Since the mean difference (i.e., true population mean - hypothesized value in the null hypothesis) depends on the unit of the measure, we need to standardize the mean difference using the standard deviation (SD) for comparison across scenarios. In statistical terms, Cohen’s d can be understood as a standardized mean difference. 

• Theoretically, the range of Cohen's d is from negative infinity to positive infinity. In jamovi, if the sample mean M is smaller than the test value, the Cohen's d in the one-sample t-test output will be negative. Otherwise, it will be positive.

Below is an effect size interpretation table for general situations.

The basic formula of Cohen's d is: 

(M - μ) / SD*

M = sample mean; μ = test value (stated in the null hypothesis); SD = standard deviation. 

For one-sample t-test, since we only compare the observed score (i.e., sample mean) to the hypothetical value of μ, the denominator (i.e., standard deviation; SD*) equals to the SD of the sample:

SD* = SD

Cohen's d can be extended to calculate the effect size for different kinds of t-tests, where the difference lies in the denominator (SD*). 

• difference between a sample value (M) and a population target value (μ), used in one-sample t-test as illustrated above; 

• difference in the mean scores between two conditions, used in paired-sample t-test (see Module 6);

• difference of the mean scores between two groups, used in independent samples t-test (see Module 6).

We will come back to these different types of Cohen's d later. 

Module Exercise

Complete the exercise!

• • Now, if you think you're ready for the exercise, you can check your email for the link.

  • Once you download your own dataset, do not forget to manage the data (e.g., select the suitable levels of measurement).

  • Remember to submit your answers before the deadline in order to earn the credits!