MTUPower Module

Developed by: Brittany Nelson (Correlations), Dave Schreifels (T-Tests), and Katy Roose (Proportions)

What is Power? Power is a measure of the probability of making a Type-II Error in an experiment with other known parameters – that is to say, the probability of correctly rejecting the null hypothesis when in fact there is an effect of whatever you're testing. The current standard for experimental power at the time of writing is 80%, or 0.80. This package consists of three power analysis functions – correlations, proportions, and t-tests. These are described hence.

Correlational Power

Description

Correlation is a simple metric of similarity between two observed variables, but this simple metric is well known to lead to erroneous conclusions of all sorts. However, with regard to two of these errors - Type-I, and Type-II - we can calculate the required parameters to make a robust correlational assessment (by solving for sample size or correlation) or, alternatively, to assess how robust an established correlation is (by solving for significance level or power).

Variables

• Sample Size – The number of observations in each group
• Correlation – The smallest correlation that can be confidently detected with the given parameters
• Significance Level – aka, “p,” which usually =.05
• Power – Experimental Power, typically .80

Proportional Power

Description

Proportion comparisons are used to find a difference between, for example, the likelihood of rolling a 1 versus the likelihood of rolling a 6 at the casino. Most times this will be used to determine the sample size needed to detect a real difference between two measured proportions (to continue the casino example, if you've rolled the die 13 times and 6 has come up three times while the other numbers have all come up 2 times, it could be that the die is slightly biased - how large of a sample would you need to collect to prove it?). Note that the probabilistic difference in proportions is not based on the mathematical difference between them (i.e. the difference between 10% and 20% will give different results than the difference between 20% and 30% - make sure to use your real calculated values instead of simplifying!).

Variables

• Proportion/Accuracy 1 - The proportion of occurrences of the first group
• Proportion/Accuracy 2 - The proportion of occurrences of the second group
• Sample Size – The number of observations in each group
• Significance Level – aka, “p,” which usually =.05
• Power – Experimental Power, typically .80
• Alternative Hypothesis - Greater Than the null hypothesis, Less Than the null hypothesis, or test in both directions (Two-Tailed)?

Power of a T-Test

Description

The t-test power analysis solves for one of four things: the number of observations required in each group, the effect size, the significance level, or the power. In addition, you will need to specify the alternative hypothesis – less than the null hypothesis, greater than, or two-tailed, which tests in both directions.

Variables

• Sample Size – the number of observations in each group
• Cohen's d – the size of the effect (decimal between 0 and 1)
• Significance Level – aka, “p,” which usually =.05
• Power – Experimental Power, typically .80
• Type of T-Test – One-sample, two-sample, or paired-samples
• Alternative Hypothesis – Greater Than the null hypothesis, Less Than the null hypothesis, or test in both directions (Two-Tailed)?