p-values

A lot of statistics boil down to a p-Value but what is a p-value?  The p-value is the probablity of getting result that high (or low) or more extream.  I think the ITRC Groundwater Statistics for Monitoring and Compliance provides a good description of p-values: 

In hypothesis testing, the p-value gives an indication of the strength of the evidence against the null hypothesis, with smaller p-values indicating stronger evidence.  If the p-value falls below the significance level (the alpha value) of the test, the null hypothesis is rejected.

Think about a coin flip.  So the following is our test:

• Null hypothesis the coin is fair.

• Alternate hypotheses the coin is rigged.

By fair we mean is that there is an equal chance of head or tails during any given toss.  So we will toss the coin a few times and see what the results are.  First we have to determine how many flips in a row of the same result we are going to find suspicious and reject the null hypothesis and declare the coin to not be fair aka rigged.  Let's say we want to be 95% sure so we are willing to risk a 5% chance of being wrong and call coin rigged when it was fair.  So we set our alpha at 0.05.  We toss the coin five times.  If the result are: 

• 4 heads and 1 tail that p-value is 0.16.  0.16 is greater than 0.05 so we Accept the null hypotheses and declare/judge the coin to be fair.

• 5 heads and 0 tails that is a  p-value is 0.03.  0.03 is less than 0.05 so we Reject the null hypotheses and declare/judge the coin to be rigged.

The following table provides some p-values and how many flips you would have to have in a row.

IMPORTATANT - a low p-value does not "Prove" anything.  It just show you how unlikely that result or more extreme is.  A p-value of 0.000001 (one in a million) looks very low but lets think about that.  Only one man in a million os over 7 feet tall.  There must be something wrong with them.  But there are around 2.3 million men in Kentucky is around so there should be approx. 2 men in Kentucky over 7 feet tall that are completely normal.  They are just way out on the tail of the bell shape curve.  We run a lot of test at the 0.05 level so 95% confidence.  So we will be wrong about 1 in 20 test.  Back to the example of men heights....that limit would be 6'3" or taller.

References:

Gaetano Raiola June 2012 Statistical study on bodily communication skills in volleyball to improve teaching methods Journal of Human Sport and Exercise DOI:10.4100/jhse.2012.72.12