Hypothesis Testing

Overview 

Hypothesis testing is a fundamental concept in statistics that allows researchers to make informed decisions about unknown population parameters based on sample data. It involves formulating a null hypothesis and an alternative hypothesis, and then using sample data to test the validity of these hypotheses. A key aspect of hypothesis testing is determining the appropriate significance level, which represents the probability of rejecting the null hypothesis when it is true.

The null hypothesis and alternative hypothesis are fundamental concepts in statistical hypothesis testing. 

The null hypothesis (H0) represents the default or initial assumption that there is no significant relationship or difference between variables or groups in a population. It is often formulated as a statement of equality or no effect. Researchers aim to test the null hypothesis to determine if there is evidence to reject it in favor of an alternative hypothesis.

The alternative hypothesis (H1 or Ha) is the opposite of the null hypothesis. It states that there is a significant relationship, effect, or difference between variables or groups in a population. Researchers seek evidence to support the alternative hypothesis, indicating that there is a true effect or relationship present.

In hypothesis testing, statistical analysis is performed on the data to determine the likelihood of observing the results under the assumption of the null hypothesis. If the evidence suggests that the observed results are highly unlikely to occur by chance under the null hypothesis, the null hypothesis is rejected in favor of the alternative hypothesis.

There are two types of hypothesis tests: one-tailed and two-tailed. 

One-tailed test: In a one-tailed test, the alternative hypothesis specifies the direction of the difference between the sample mean and the hypothesized population mean. This means that we are only interested in detecting if the sample mean is significantly greater than or less than the hypothesized population mean in a specific direction. One-tailed tests are typically used when there is a clear directional prediction or when the cost of a Type II error (failing to reject a false null hypothesis) is high.

Two-tailed test: In a two-tailed test, the alternative hypothesis simply states that there is a difference between the sample mean and the hypothesized population mean, without specifying the direction. This means that we are interested in detecting if the sample mean is significantly different from the hypothesized population mean in either direction. Two-tailed tests are typically used when there is no clear directional prediction or when the cost of a Type I error (rejecting a true null hypothesis) is high.

Significance level: The significance level (alpha) is the probability of rejecting the null hypothesis when it is actually true. It is typically set at 0.05 or 0.01, which means that we are willing to accept a 5% or 1% chance of making a Type I error. The significance level determines the critical value or region for the test statistic.

Critical region: The critical region is the range of values for the test statistic that would lead to the rejection of the null hypothesis. The critical region is determined by the significance level and the degrees of freedom, which depend on the sample size and the method used to estimate the population parameter. If the calculated test statistic falls within the critical region, we reject the null hypothesis; if it falls outside the critical region, we fail to reject the null hypothesis.

Exploring a Powerful Hypothesis Testing Tool with Shiny App

The shiny app is a user-friendly tool that allows you to perform hypothesis testing for both binomial and normal distributions. With this app, you can conveniently analyze your data and determine whether the observed results support or reject a specific hypothesis.

When you open the app, you will see a user interface with two tabs: "Binomial Distribution" and "Normal Distribution." These tabs correspond to the two types of hypothesis tests available.

In the "Binomial Distribution" tab, you can input the sample size, the number of successes, the expected probability, and select the tail type for your test (one-tail or two-tail). The app will generate a bar plot displaying the binomial distribution based on your inputs.

To perform the binomial hypothesis test, click the "Calculate Binomial Test" button. The app will calculate the test statistic and p-value based on your inputs. The result will be displayed in the "Binomial Hypothesis Test Result" tab, indicating whether you should reject or fail to reject the null hypothesis.

Switching to the "Normal Distribution" tab, you can provide the sample size, the mean, the standard deviation, the test value, and select the tail type for your test. The app will generate a line plot showing the normal distribution with the test value marked in red.

To conduct the normal hypothesis test, click the "Calculate Normal Test" button. The app will calculate the test statistic and p-value based on your inputs. The result will be displayed in the "Normal Hypothesis Test Result" tab, indicating whether you should reject or fail to reject the null hypothesis.

Throughout the app, the distinction between one-tail and two-tail tests allows you to specify the directionality of your hypothesis. A one-tail test examines whether the observed results are significantly greater or smaller than the expected values in a specific direction. On the other hand, a two-tail test investigates whether the observed results differ significantly from the expected values in any direction.

By using this Shiny app, you can easily perform hypothesis testing for both binomial and normal distributions, empowering you to make informed decisions based on statistical evidence.

Expore!

Hypothesis Testing for Binomial Distribution

The binomial distribution is a probability distribution that describes the number of successes in a fixed number of independent Bernoulli trials. It is commonly used in hypothesis testing to analyze categorical data and determine the likelihood of observing a specific outcome or set of outcomes.

In the binomial distribution, each trial has two possible outcomes: success or failure. The probability of success (often denoted as p) remains constant across all trials, while the probability of failure is given by q = 1 - p. The distribution is characterized by two parameters: the number of trials (n) and the probability of success (p).

To use the binomial distribution in hypothesis testing, certain conditions must be met:

1. Independent Trials: The trials must be independent of each other, meaning the outcome of one trial does not affect the outcome of another.

2. Fixed Number of Trials: The number of trials (n) must be fixed in advance. Each trial is considered as a separate and identical experiment.

3. Two Possible Outcomes: Each trial should have only two possible outcomes, often referred to as success and failure. These outcomes are mutually exclusive.

4. Constant Probability of Success: The probability of success (p) should remain constant across all trials. This assumption ensures that the binomial distribution accurately reflects the underlying process being studied.

By satisfying these conditions, the binomial distribution can be applied to analyze the observed data and calculate probabilities associated with different outcomes. This helps in hypothesis testing by determining the likelihood of obtaining the observed results under the assumption of the null hypothesis, and subsequently evaluating the strength of evidence against the null hypothesis.

One-tailed tests

If you want to carry out a one-tailed hypothesis test, you need to:

• Formulate a model for the test statistic

• Identify suitable null and alternative hypotheses

• Calculate the probability of the test statistic taking the observed value (or higher/lower), assuming the null hypothesis is true

• Compare this to the significance level

• Write a conclusion in the context of the question

Alternatively, you can find the critical region and see if the observed value of the test statistic lies inside it.

Task 1 

The standard treatment for a particular disease has a  2/5 probability of success. A certain doctor has undertaken research in this area and has produced a new medicine which has been successful with 11 out of 20 patients. The doctor claims that the new medicine represents an improvement on the standard treatment.

Test, at the 5% significance level, the claim made by the doctor.

Two-tailed tests

Sometimes we may be asked if it has changed. We are not told how it has changed and so there is no direction to H1.  This is called a two-tailed test.

For a two-tailed test, halve the significance level at the end you are testing.

You need to know which tail of the distribution you are testing. If the test statistic is X ~ B(n, p) then the expected outcome is np. If the observed value x is lower than this then consider P(X ≤ x). If the observed value is higher than the expected value, then consider P(X≥x).

Task 2

Over a long period of time, it has been found that at Enrico's restaurant the ratio of non-vegetarian to vegetarian meals is 2 to 1. At Manuel's restaurant, in a random sample of 10 people ordering meals. only one ordered a vegetarian meal. Using a 5% level of significance, test whether or not the proportion of people eating vegetarian meals at Manuel's restaurant is different to that at Enrico's restaurant.

Approximating Binomial Distribution with Normal Distribution

Introduction:

The normal distribution can be used as an approximation for the binomial distribution under certain conditions. This approximation is based on the Central Limit Theorem, which states that the sum of a large number of independent and identically distributed random variables approaches a normal distribution.

Conditions for Approximation:

1. The number of trials (n) in the binomial distribution should be large, typically n ≥ 20.

2. The probability of success (p) should be moderate, neither too close to 0 nor too close to 1. A rule of thumb is to ensure np ≥ 5 and n(1 - p) ≥ 5.

Calculation Method:

To approximate a binomial distribution with a normal distribution, follow these steps:

Step 1: Calculate the mean of the binomial distribution as μ = np.

Step 2: Calculate the standard deviation of the binomial distribution as σ = √(np(1 - p)).

Step 3: Convert the binomial distribution to a normal distribution by applying a continuity correction. For discrete binomial variables, use the continuity correction by subtracting 0.5 from the lower bound and adding 0.5 to the upper bound.

Step 4: Use the mean and standard deviation calculated in Step 1 and Step 2 to determine the corresponding values in the normal distribution.

Parameter Transformation:

When using the normal distribution as an approximation for the binomial distribution, the parameters are transformed as follows:

Mean (μ) of the normal distribution = np

Standard Deviation (σ) of the normal distribution = √(np(1 - p))

Application in Hypothesis Testing:

The approximation of binomial distribution with a normal distribution is commonly used in hypothesis testing. It allows us to use the properties of the normal distribution to calculate probabilities and critical values.

Exercise 1: One-Tailed Test

A manufacturer claims that 80% of their products meet a certain quality standard. A random sample of 100 products is taken, and 75 of them meet the standard. Perform a one-tailed hypothesis test at a significance level of 0.05 to determine if the manufacturer's claim is justified. 

Exercise 2: Two-Tailed Test

A website claims that 70% of its users are satisfied with its service. A random sample of 200 users is taken, and 140 of them express satisfaction. Perform a two-tailed hypothesis test at a significance level of 0.01 to examine the claim.