L2 C3 S4

ADDITION LAW applies ONLY to UNION of sets. We are trying to calculate the probability of choosing a member from A OR from B, so members of BOTH sets are part of the UNION of the two sets. This is like ADDING the two sets to each other. CORRESPONDINGLY, the probabilities of the two sets are added.

THERE ARE TWO CASES: The sets are MUTUALLY EXCLUSIVE, this means they have NO OVERLAP. Then we just add the two probabilities:

MULTIPLICATION LAW applies only to the INTERSECTION of sets. We are trying to compute the probability of choosing a member belong to BOTH sets at the same time. This is obviously a SMALLER set. Multiplying probabilities always makes them smaller, since they are all less than or equal to 1.

The following multiplication law applies to ANY TWO events A and B regardless of whether or not they are independent:

The second, more difficult case arises when the two sets have overlap. For example, if A is female students and B is students who wear glasses. Just adding the probabilities of the two sets will DOUBLE COUNT the female students who wear glasses. They will be counted once as females and another time as students who wear glasses. To correct for this, the intersection of the two sets must be subtracted from the sum, This leads to the following formula:

When the events A and B are independent, than P(A|B)=P(A) -- conditioning on B does not change the probability of A -- This means that knowledge of whether or not the event B has happened does not affect our knowledge of the probabiilty of B. Similarly, independence means that P(B|A)=P(B). Knowledge about the event A does not affect the probability of B. In this case the general formula above simplifies to :

This formula is valid for ALL sets, whether or not they are mutually exclusive. The definition of mutually exclusive is that the two sets do not have any common members so the intersection is empty, which means that the last term in the above expression is ZERO.

This formula defines independence. That is A and B are independent if and only if this formula holds.

Note that if the two sets A and B are non-empty than the product of their probabilities on the Right Hand Side above will be greater than 0. On the other hand, if these sets are Mutually Exclusive, then the intersection of the sets on the Left Hand Side will have ZERO probability. So independent sets cannot be mutually exclusive, and vice versa -- mutually exclusive sets cannot be independent.

In terms of information, if A and B are mutually exclusive then when we know that A has occurred, we know that B has NOT occurred, so knowledge of A gives us perfect information about B. This means that the two sets cannot be independent.