Lecture 5: conditional probability

Conditional probability

Definition: If \(A\) and \(B\) are events, then the probability of A given B, written \(Pr(A|B)\) is given by \[Pr(A|B) := \frac{Pr(A \cap B)}{Pr(B)}\] Note that \(Pr(A|B)\) is only defined if \(Pr(B) \neq 0\).

Intuitively, \(Pr(A|B)\) is the probability of \(A\) in a new sample space created by restricting our attention to the subset of the sample space where \(B\) occurs. We divide by \(Pr(B)\) so that \(Pr(B|B) = 1\).

Note: \(A|B\) is not defined, only \(Pr(A|B)\); this is an abuse of notation, but is standard.

Probability trees

Using conditional probability, we can draw a tree to help discover the probabilities of various events. Each branch of the tree partitions part of the sample space into smaller parts.

For example: suppose that it rains with probability 30%. Suppose that when it rains, I bring my umbrella 3/4 of the time, while if it is not raining, I bring my umbrella with probability 1/10. Given that I bring my umbrella, what is the probability that it is raining?

One way to model this problem is with the sample space

\[ S = \{raining (r), not raining (nr)\} \times \{umbrella (u), no umbrella (nu)\} = \{(r,u), (nr,u), (r,nu), (nr,nu)\} \]

Let \(R\) be the event "it is raining". Then \(R = \{(r,u), (r,nu)\}\). Let \(U\) be the event "I bring my umbrella". Then \(U = \{(r,u), (nr,u)\}\).

The problem tells us that \(Pr(R) = 3/10\). It also states that \(Pr(U|R) = 3/4\) while \(Pr(U|\bar{R}) = 1/10\). We can use the following fact:

Fact: \(Pr(\bar{A}|B) = 1 - Pr(A|B)\). Proof left as exercise.

to conclude that \(Pr(\bar{U}|R) = 1/4\) and \(Pr(\bar{U}|\bar{R}) = 9/10\).

We can draw a tree:

Probability tree (LaTeX source)

Probability tree (LaTeX source)

We can compute the probabilities of the events at the leaves by multiplying along the paths. For example, \[Pr(\{(r,u)\}) = Pr(U \cap R) = Pr(R)Pr(U | R) = (3/10)(3/4) = (9/40)\]

To answer our question, we are interested in \(Pr(R|U) = Pr(U \cap R)/Pr(U)\). We know \(U = \{(u,r), (u,nr)\}\). We can compute \(Pr(U)\) using the third axiom; \(Pr(U) = Pr(\{(u,r)\}) + Pr(\{(u,nr)\}) = (3/10)(3/4) + (7/10)(1/10)\). We can then plug this in to the above formula to find \(Pr(R|U)\).

Note we could also answer this using Bayes's rule and the law of total probability (see below); it would amount to exactly the same calculation. The tree just helps organize all of the variables.

Bayes's Rule

Bayes's rule is a simple way to compute \(P(A|B)\) from \(P(B|A)\).

Claim: (Bayes's rule): \(P(A|B) = P(B|A)P(A)/P(B)\).

Proof: Write down the definitions; proof left as exercise.

Law of total probability

Claim: (law of total probability) If \(A_1, \dots, A_n\) partition the sample space \(S\) (that is, if \(A_i \cap A_j = \emptyset\) for \(i \neq j\) and \(S = \cup A_i\)), then

\[Pr(B) = \sum_{i} Pr(B|A_i)Pr(A_i)\]

Proof sketch: Write \(B = \cup_{i} (B \cap A_i)\). Apply third axiom to conclude \(Pr(B) = \sum_{i} Pr(B \cap A_i)\). Apply definition of \(Pr(B | A_i)\).