P&S L1 C5 S2
ntro Stats: Islamic Approach -- Part 2 Prob & Stat
Lecture 1: Random Samples -- Concept 5: Law of Large Numbers
S2: Problems with the Limiting Observed Frequency Definition of Probability
The most widely accepted definition of probability, used in nearly all modern textbooks (with few exceptions) is called the frequency theory of probability. This definition is based on the Law of Large Numbers. As we have developed the topic, the probability of some set E is the proportion of elements of E in the target population being sampled. The observed proportions in simple random samples converge to this true probability as the size of the simple random sample goes to infinity. This standard definition does not make sense for many reasons.
EXPLANATION OF FREQUENTIST DEFINITION OF PROBABILITY: The frequentist definition was invented at a time when the Logical Positivist Philosophy was dominant. This philosophy said that "unobservables" do not belong in scientific theories, which deals with the real world we can touch and see. The intuitive definition of probability involves the idea of "what might have happened". Even though we chose one particular person, we could have chosen someone else. This is UNOBSERVABLE. Therefore, frequentists tried to find something which was observable and would be a suitable definiton of probability. From the Law of Large Numbers, the observed frequency in a large samples converges to the probability, so this was taken as an OBSERVABLE way to define probability.
CRITIQUE OF FREQUENTIST DEFINITION: The frequentist definition was MEANT to define probability in such way that it is observable. HOWEVER, the infinite limit of trial is NOT observable. The only way to assess what will happen in the limit is to use theoretical properties of probability. But this makes the definition CIRCULAR. First we define probability. Then we use the properties of probability to prove the Law of Large Numbers, which shows that observed frequencies converge to probability. Then we make this limit the definition of probability. We have to start with a definition of probability in order to get to this definition via frequency. There are many other difficulties with a limiting frequency definition.
Conclusion -- The Standard Frequentist DEFINITION of probability is wrong. The definition is ACTUALLY a theorem, the Law of Large Numbers, which follows from some other definition of probability. Probability must be defined FIRST, and then we can prove that the limiting frequency equals the probability.