Bayes’ Theorem

Context: FIT1058_MOC · reverses a conditional · updates a prior belief into a posterior given evidence · denominator from the Law of Total Probability

Quick Revision

• 🎯 Objective: convert $\mathrm{Pr}(B\mid A)$ into $\mathrm{Pr}(A\mid B)$ ➔ $\frac{\mathrm{Pr}(B\mid A)\mathrm{Pr}(A)}{\mathrm{Pr}(B)}$.
• 📦 Core Components: prior ➔ likelihood ➔ posterior; denominator via total probability.
• ⚡ Key Constraint: $\mathrm{Pr}(A\mid B)\ne \mathrm{Pr}(B\mid A)$ — confusing them is the base-rate fallacy.

📝 Core

1. The Theorem

• Formula ➔ $\mathrm{Pr}(A\mid B)=\frac{\mathrm{Pr}(B\mid A)\mathrm{Pr}(A)}{\mathrm{Pr}(B)}$.
• Belief updating ➔ prior $\mathrm{Pr}(A)$ → posterior $\mathrm{Pr}(A\mid B)$ after observing $B$.

2. Proof & Extended Form

• Proof ➔ two expressions for $\mathrm{Pr}(A\cap B)$; equate and divide by $\mathrm{Pr}(B)$.
• Extended ➔ over a partition, denominator $={\sum }_i\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$ (Law of Total Probability).

3. Prior → Posterior

• Can rise/fall/hold ➔ observing Heads: DoubleHead $\frac{1}{3}\to \frac{2}{3}$, DoubleTail $\frac{1}{3}\to 0$, Fair unchanged.
• Normalisation shortcut ➔ posteriors over a partition sum to 1.

When It Flips: posteriors over a partition sum to 1, so a missing one can be found by subtraction. Bayesian updating underpins statistics and machine learning; if $A,B$ independent the posterior equals the prior.

📊 Exam Execution Trace

Manual Execution Trace

Three coins, observe Heads:

Step / State	Hypothesis	prior	likelihood	posterior
0 (Init)	—	—	—	—
1	Fair	$\frac{1}{3}$	$\frac{1}{2}$	$\frac{1}{3}$
2	DoubleHead	$\frac{1}{3}$	1	$\frac{2}{3}$
3	DoubleTail	$\frac{1}{3}$	0	0

⚠️ Common Mistakes

• 💡 Direction matters ➔ $\mathrm{Pr}(A\mid B)\ne \mathrm{Pr}(B\mid A)$; Bayes is the correction factor $\mathrm{Pr}(A)/\mathrm{Pr}(B)$ (base-rate fallacy).

🧠 Active Recall

State and prove Bayes' Theorem.

• Hint: Two forms of $\mathrm{Pr}(A\cap B)$.

Answer

• Short answer: $\mathrm{Pr}(A\mid B)=\frac{\mathrm{Pr}(B\mid A)\mathrm{Pr}(A)}{\mathrm{Pr}(B)}$.
• Why: Equate ➔ $\mathrm{Pr}(B)\mathrm{Pr}(A\mid B)=\mathrm{Pr}(A)\mathrm{Pr}(B\mid A)$, divide.

What are prior and posterior, and how does the extended form get its denominator?

• Hint: Before/after + total probability.

Answer

• Short answer: Prior = before evidence, posterior = after; denominator $={\sum }_i\mathrm{Pr}(B\mid A_i)\mathrm{Pr}(A_i)$.
• Why: Normalise ➔ the total-probability sum makes posteriors sum to 1.