L6 C4 S2 Probability Versus Confidence
Conventional treatments of the theory of confidence intervals are very confusing because they fail to distinguish clearly between pre-experimental and post-experimental. Given any probabilistic situation, e.g. a Bernoulli coin flip B such that P(B=1)=p, the probability exists only pre-experimentally. Post-experimentally, after the coin has been flipped and the outcome has occurred, it is NO LONGER true that P(B=1)=p. Now, either B=1 is TRUE, or it is FALSE; there is no probability associated with this event any more. However, there is one case where something like probability applies in the post-experimental situation. This is the case where the coin is flipped, the outcome has occurred, but I don’t know what the outcome is. Due to my ignorance of the outcome, my knowledge of the post-experimental situation is not different from my pre-experimental knowledge. In this situation, we will use the word CONFIDENCE, to replace probability. I can say that B=1 with 50% Confidence – This is shorthand for saying that we are in a post-experimental situation where probabilities no longer apply. BUT, because the outcome is NOT KNOWN to me, my knowledge about B is LIKE the probabilistic knowledge of someone in the pre-experimental situation. The word “confidence” refers to a state of mind, a feeling, and therefore accurately displays the subjectivity of the concept, which can be changed if I get a glimpse of the coin, and so my state of knowledge about the coin changes. The uncertainty or probability is now in my head, and not in the physical world out there. The physical uncertainty has been removed, because the experiment has taken place, and the outcome has occurred. The psychological uncertainty remains because I did not learn the outcome.
This is exactly the situation which arises when we use a random sample to estimate population probabilities. Before the random sample of size N is collected, we can say that with 95% probability, the proportion to be observed in the random sample to be collected will be within 2SE=2sqrt(p*(1-p))/sqrt(N). After the random sample has been collected and the outcome has been observed, the probability is extinguished. Now either the event of 95% probability took place, or it did not take place. Either the true p fell within 2SE of the observed proportion in the sample, or it did not fall within the interval. The probability no longer exists; the event is either TRUE or FALSE. However, if we do not KNOW the true value of p, then we do not know whether or not the event occurred. In this case, we can use the CONFIDENCE theory to assert that we have 95% confidence that the event took place.
NOTE: The terminology above is new and non-standard, but conforms closely to standard usage. In fact, it would be correct to say that pre-experimentally, we have objective probability. Post-experimentally, lack of knowledge of outcomes converts objective probabilities to subjective probabilities. However, the words “subjective probability” have been adopted by another school of thought, which has given a very misleading and confusing meaning to them. So we avoid the use of this terminology, which would be the natural one in other circumstances.