Probability & Levels of Significance

Specification

• Decision making and interpretation of inferential statistics

• Probability and levels of significance (p≤.10 p≤.05 p≤.01).

• Observed and critical values, and sense checking of data.

• Type I and type II errors.

Inferential statistics

Inferential statistics are ways of analysing data using statistical tests that allow the researcher to make conclusions about whether a hypothesis was supported by the results.

Levels of Significance

The null hypothesis states that there is no relationship between the two variables being studied (one variable does not affect the other). It states the results are due to chance and are not significant in terms of supporting the idea being investigated. Thus, the null hypothesis assumes that whatever you are trying to prove did not happen.

The alternative hypothesis is the one you would believe if the null hypothesis is concluded to be untrue.

The alternative hypothesis states that the independent variable did affect the dependent variable, and the results are significant in terms of supporting the theory being investigated (i.e. not due to chance).

How do you know if a p-value is statistically significant?

The level of statistical significance is often expressed as a p-value between 0 and 1. The smaller the p-value, the stronger the evidence that you should reject the null hypothesis.

• A p-value less than 0.05 (typically ≤ 0.05) is statistically significant. It indicates strong evidence against the null hypothesis, as there is less than a 5% probability the null is correct (and the results are random). Therefore, we reject the null hypothesis, and accept the alternative hypothesis.
  However, this does not mean that there is a 95% probability that the research hypothesis is true. The p-value is conditional upon the null hypothesis being true is unrelated to the truth or falsity of the research hypothesis.
  

• A p-value higher than 0.05 (> 0.05) is not statistically significant and indicates strong evidence for the null hypothesis. This means we retain the null hypothesis and reject the alternative hypothesis. You should note that you cannot accept the null hypothesis, we can only reject the null or fail to reject it.
  

• A statistically significant result cannot prove that a research hypothesis is correct (as this implies 100% certainty). Instead, we may state our results “provide support for” or “give evidence for” our research hypothesis (as there is still a slight probability that the results occurred by chance and the null hypothesis was correct – e.g. less than 5%).

Type 1 & Type 2 Errors

A statistically significant result cannot prove that a research hypothesis is correct (as this implies 100% certainty). Because a p-value is based on probabilities, there is always a chance of making an incorrect conclusion regarding accepting or rejecting the null hypothesis (H₀).

Anytime we make a decision using statistics there are four possible outcomes, with two representing correct decisions and two representing errors.

The chances of committing these two types of errors are inversely proportional: that is, decreasing type I error rate increases type II error rate, and vice versa.

How does a Type 1 error occur?

A type 1 error is also known as a false positive and occurs when a researcher incorrectly rejects a true null hypothesis. This means that your report that your findings are significant when in fact they have occurred by chance.

The probability of making a type I error is represented by your alpha level (α), which is the p-value below which you reject the null hypothesis. A p-value of 0.05 indicates that you are willing to accept a 5% chance that you are wrong when you reject the null hypothesis.

You can reduce your risk of committing a type I error by using a lower value for p. For example, a p-value of 0.01 would mean there is a 1% chance of committing a Type I error.

However, using a lower value for alpha means that you will be less likely to detect a true difference if one really exists (thus risking a type II error).

How does a Type II error occur?

A type II error is also known as a false negative and occurs when a researcher fails to reject a null hypothesis which is really false. Here a researcher concludes there is not a significant effect, when actually there really is.

The probability of making a type II error is called Beta (β), and this is related to the power of the statistical test (power = 1- β). You can decrease your risk of committing a type II error by ensuring your test has enough power.

You can do this by ensuring your sample size is large enough to detect a practical difference when one truly exists.

Why are Type I and Type II Errors Important?

The consequences of making a type I error mean that changes or interventions are made which are unnecessary, and thus waste time, resources, etc.

Type II errors typically lead to the preservation of the status quo (i.e. interventions remain the same) when change is needed.

Statistical Tables

Statistical tables provide information to help psychologists make decisions in relation to the statistical significance of the results of statistical tests. Statistical tables contain ‘critical values’ that are used when assessing significance.

Critical values are a numerical value which researchers use to determine whether or not their calculated/observed value (from a statistical test) is significant. Some tests are significant when the observed (calculated) value is equal to or greater than the critical value, and for some tests the observed value needs to be less than or equal to the critical value.

Past Paper Questions

• Jayla decided to check her data using a statistical test at p≤0.05 in order to avoid a type I error. Define the term ‘type I error’. (1) June 2017

• State which statistical test Oti would have used for her data. (1) June 2018 P2

• Sacha carried out a Spearman’s rank test. He wanted to see if his results were significant at p≤0.05. Define what is meant by p≤0.05. (2) January 2019 P2

• Explain which statistical test Cherry could use to analyse her data. (3) January 2020 P2

• Explain the difference between p=0.05 and p=0.01 for Hassan's study. (2) June 2019

• Calculate df using Table 3. The formula can be found in the formulae and statistical tables at the front of the exam paper. (1) June 2018

• Define what is meant by p≤0.05. (2) January 2019

• Explain what is meant by p<0.05 in relation to this study. (2) June 2016

• Philippa decides to use a p≤0.01 level of significance on her results. State what is meant by ‘≤’. (1) October 2018

• Describe what is meant by p≤0.01 in relation to Cha and Dao’s investigation. (2) October 2019

• Explain one strength of Cha and Dao applying a ‘sense check’ to the data gathered in their investigation. (2) October 2019