Law of Total Probability

Context: FIT1058_MOC · break an event’s probability across a partition of the sample space · weighted sum of conditionals · the denominator of extended Bayes’ Theorem

Quick Revision

• 🎯 Objective: partition $U=⨆A_i$ ⟹ $\mathrm{Pr}(B)={\sum }_i\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$ ➔ weighted sum of conditionals.
• 📦 Core Components: distribute over the partition ➔ additivity ➔ multiplication rule.
• ⚡ Key Constraint: partition must be disjoint + exhaustive; supplies Bayes’ denominator.

📝 Core

1. The Law

• Formula ➔ $\mathrm{Pr}(B)={\sum }_i\mathrm{Pr}(B\cap A_i)={\sum }_i\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$.
• Partition ➔ $U=A_1\bigsqcup \cdots \bigsqcup A_n$ (mutually exclusive + exhaustive).

2. Why It Works

• Distribute ➔ $B={⨆}_i(B\cap A_i)$ (disjoint pieces).
• Add ➔ $\mathrm{Pr}(B)={\sum }_i\mathrm{Pr}(B\cap A_i)$ (additivity).
• Multiply ➔ $\mathrm{Pr}(B\cap A_i)=\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$.

3. When It Helps

• Easy conditionals ➔ choose a partition where each $\mathrm{Pr}(B\mid A_i)$ is simple.
• Causes/scenarios ➔ turns one hard probability into a sum of simple ones.

Key identities:

$\mathrm{Pr}(B)={\sum }_{i=1}^n\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i),U={⨆}_i{A}_i$

When It Flips: the sum ${\sum }_i\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$ is exactly the denominator (normaliser) of the extended Bayes' Theorem.

📊 Exam Execution Trace

Manual Execution Trace

Three coins (Fair, DoubleHead, DoubleTail), $\mathrm{Pr}(\text{Heads})$:

Step / State	Cause $A_i$	$\mathrm{Pr}(A_i)$	$\mathrm{Pr}(H\mid A_i)$	Product
0 (Init)	—	—	—	—
1	Fair	$\frac{1}{3}$	$\frac{1}{2}$	$\frac{1}{6}$
2	DoubleHead	$\frac{1}{3}$	1	$\frac{1}{3}$
3	DoubleTail	$\frac{1}{3}$	0	0

⚠️ Common Mistakes

• 💡 Partition must cover with no overlap ➔ a gap or overlap breaks the equality; each $\mathrm{Pr}(A_i)>0$ so the conditionals exist.

🧠 Active Recall

State the law of total probability and derive it from additivity.

• Hint: Distribute + add + multiply.

Answer

• Short answer: $\mathrm{Pr}(B)={\sum }_i\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$ over a partition.
• Why: $B=⨆(B\cap A_i)$ ➔ additivity then the multiplication rule.

Why is the law useful, and where does it appear in Bayes' Theorem?

• Hint: Easy conditionals; the normaliser.

Answer

• Short answer: Converts a hard $\mathrm{Pr}(B)$ into a sum of simple conditionals over causes $A_i$.
• Why: Bayes’ denominator ➔ the same sum normalises the posteriors to sum to 1.