p-values

A lot of statistics boil down to a p-Value but what is a p-value?  I think the ITRC Groundwater Statistics for Monitoring and Compliance provides a good description of p-values: 

In hypothesis testing, the p-value gives an indication of the strength of the evidence against the null hypothesis, with smaller p-values indicating stronger evidence.  If the p-value falls below the significance level (the alpha value) of the test, the null hypothesis is rejected.

Think about a coin flip.  So the following is our test:

• Null hypothesis the coin is fair.

• Alternate hypotheses the coin is rigged.

By fair we mean is that there is an equal chance of head or tails during any given toss.  So we will toss the coin a few times and see what the results are.  First we have to determine how many flips in a row of the same result we are going to find suspicious and reject the null hypothesis and declare the coin to not be fair aka rigged.  Let's say we want to be 95% sure so we are willing to risk a 5% chance of being wrong and call coin rigged when it was fair.  So we set our alpha at 0.05.  We toss the coin five times.  If the result are: 

• 4 heads and 1 tail that p-value is 0.06.  0.06 is less than 0.05 so we Accept the null hypotheses and declare/judge the coin to be fair.

• 5 heads and 0 tails that is a  p-value is 0.03.  0.03 is greater than 0.05 so we Reject the null hypotheses and declare/judge the coin to be rigged.

The following table provides some p-values and how many flips you would have to have in a row.

References:

Gaetano Raiola June 2012 Statistical study on bodily communication skills in volleyball to improve teaching methods Journal of Human Sport and Exercise DOI:10.4100/jhse.2012.72.12