Chapter 5: Bayes’ Theorem

Meike Niederhausen and Nicky Wakim

2023-10-04

Learning Objectives

  1. Calculate conditional probability of an event using Bayes’ Theorem

  2. Utilize additional probability rules in probability calculations, specifically the Higher Order Multiplication Rule and the Law of Total Probabilities

Introduction

  • So we learned about conditional probabilities

    • We learned how the occurrence of event A affects event B (B conditional on A)

  • Can we figure out information on how the occurrence of event B affects event A?

  • We can use the conditional probability (\(\mathbb{P}(A|B)\)) to get information on the flipped conditional probability (\(\mathbb{P}(B|A)\))

Bayes’ Rule for two events

Theorem: Bayes’ Rule (for two events)

For any two events \(A\) and \(B\) with nonzero probabilties,

\[\mathbb{P}(A| B) = \frac{\mathbb{P}(A) \cdot \mathbb{P}(B|A)} {\mathbb{P}(B)}\]

Calculating probability with Higher Order Multiplication Rule

Example 1

Suppose we draw 5 cards from a standard shuffled deck of 52 cards. What is the probability of a flush, that is all the cards are of the same suit (including straight flushes)?

Higher Order Multiplication Rule

\[\mathbb{P}(A_1\cap A_2 \cap \ldots \cap A_n)=\mathbb{P}(A_1)\cdot\mathbb{P}(A_2|A_1) \cdot \\ \mathbb{P}(A_3|A_1A_2)\ldots \cdot\mathbb{P}(A_n|A_1A_2\ldots A_{n-1})\]

Calculating probability with Law of Total Probability

Example 2

Suppose 1% of people assigned female at birth (AFAB) and 5% of people assigned male at birth (AMAB) are color-blind. Assume person born is equally likely AFAB or AMAB (not including intersex). What is the probability that a person chosen at random is color-blind?

Law of Total Probability for 2 Events

For events \(A\) and \(B\),

\[%\left( \begin{array}{ccl} \mathbb{P}(B)&=&\mathbb{P}(B \cap A) + \mathbb{P}(B \cap A^C)\\ &=& \mathbb{P}(B|A) \cdot \mathbb{P}(A)+ \mathbb{P}(B | A^C)\cdot \mathbb{P}(A^C) \end{array} %\right)\]

General Law of Total Proability

Law of Total Probability (general)

If \(\{A_i\}_{i=1}^{n} = \{A_1, A_2, \ldots, A_n\}\) form a partition of the sample space, then for event \(B\),

\[%\left( \begin{array}{ccl} \mathbb{P}(B)&=& \sum_{i=1}^{n} \mathbb{P}(B \cap A_i)\\ &=& \sum_{i=1}^{n} \mathbb{P}(B|A_i) \cdot \mathbb{P}(A_i) \end{array} %\right)\]

Calculating probability with generalized Law of Total Probability

Let’s revisit the color-blind example

Example 4

Recall the color-blind example (Example 2), where

  • a person is AMAB with probability 0.5,

  • AMAB people are color-blind with probability 0.05, and

  • all people are color-blind with probability 0.03.

Assuming people are AMAB or AFAB, find the probability that a color-blind person is AMAB.

Calculate probability with both rules

Example 5

Suppose

  • 1% of women aged 40-50 years have breast cancer,

  • a woman with breast cancer has a 90% chance of a positive test from a mammogram, and

  • a woman has a 10% chance of a false-positive result from a mammogram.

What is the probability that a woman has breast cancer given that she just had a positive test?

Bayes’ Rule

Theorem: Bayes’ Rule

If \(\{A_i\}_{i=1}^{n}\) form a partition of the sample space \(S\), with \(\mathbb{P}(A_i)>0\) for \(i=1\ldots n\) and \(\mathbb{P}(B)>0\), then

\[\mathbb{P}(A_j | B) = \frac{\mathbb{P}(B|A_j) \cdot \mathbb{P}(A_j)} {\sum_{i=1}^{n} \mathbb{P}(B|A_i) \cdot \mathbb{P}(A_i)}\]

Chapter 5 Slides