Lesson 4: Rules of probability

Meike Niederhausen and Nicky Wakim

2025-10-08

Learning objectives

  1. Define independence of 2-3 events and decide whether two or more events are independent

  2. Define key facts for conditional probabilities and calculate conditional probabilities.

  3. Calculate the probability of an event using Bayes’ Theorem, Higher Order Multiplication Rule, and the Law of Total Probabilities

Where are we?

General Process for Probability Word Problems

  1. Clearly define your events of interest

  2. Translate question to probability using defined events OR Venn Diagram

  3. Ask yourself:

    • Are we sampling with or without replacement?

    • Does order matter?

  4. Use axioms, properties, partitions, facts, etc. to define the end probability calculation into smaller parts

    • If probabilities are given to you, Venn Diagrams may help you parse out the events and probability calculations

    • If you need to find probabilities with counting, pictures or diagrams might help here

  5. Write out a concluding statement that gives the probability context

  6. (For own check) Make sure the calculated probability follows the axioms. Is is between 0 and 1?

Learning objectives

  1. Define independence of 2-3 events and decide whether two or more events are independent

  1. Define key facts for conditional probabilities and calculate conditional probabilities.

  2. Calculate the probability of an event using Bayes’ Theorem, Higher Order Multiplication Rule, and the Law of Total Probabilities

Independent Events

Definition: Independence

Events \(A\) and \(B\) are independent if \[\mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B).\]

Notation: For shorthand, we sometimes write \(A \mathrel{\unicode{x2AEB}} B,\) to denote that \(A\) and \(B\) are independent events.

 

  • Also note: \[\begin{aligned} \mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B) & \implies A \mathrel{\unicode{x2AEB}} B \\ A \mathrel{\unicode{x2AEB}} B & \implies \mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B) \end{aligned}\]

Example of two dice

Example 1

Two dice (green and blue) are rolled. Let \(A =\) event a total of 7 appears, and \(B =\) event green die is a six. Are events \(A\) and \(B\) independent?

Example of two dice: simulating in R (1/2)

Example 1

Two dice (green and blue) are rolled. Let \(A =\) event a total of 7 appears, and \(B =\) event green die is a six. Are events \(A\) and \(B\) independent?

set.seed(1002)
reps = 10000
rolls = replicate(reps, sample(x = 1:6, size = 2, replace = TRUE))
rolls[, 1:10]
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]    4    5    5    5    6    5    2    4    3     4
[2,]    1    4    6    3    1    1    5    5    6     3
event_A = ( rolls[1, ] + rolls[2, ] == 7 )
head(event_A, 10)
 [1] FALSE FALSE FALSE FALSE  TRUE FALSE  TRUE FALSE FALSE  TRUE
event_B = ( rolls[1, ] == 6 )
head(event_B, 10)
 [1] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE

Example of two dice: simulating in R (1/2)

Example 1

Two dice (green and blue) are rolled. Let \(A =\) event a total of 7 appears, and \(B =\) event green die is a six. Are events \(A\) and \(B\) independent?

( sum(event_A) / reps ) * ( sum(event_B) / reps )
[1] 0.0286608
event_A_and_B = ( rolls[1, ] + rolls[2, ] == 7 ) & ( rolls[1, ] == 6 )
head(event_A_and_B, 10)
 [1] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE
sum(event_A_and_B) / reps
[1] 0.0284

Independence of 3 Events

Definition: Independence of 3 Events

Events \(A\), \(B\), and \(C\) are mutually independent if

    • \(\mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B)\)

    • \(\mathbb{P}(A \cap C) = \mathbb{P}(A) \cdot \mathbb{P}(C)\)

    • \(\mathbb{P}(B \cap C) = \mathbb{P}(B) \cdot \mathbb{P}(C)\)

  2. \(\mathbb{P}(A \cap B \cap C) = \mathbb{P}(A) \cdot \mathbb{P}(B) \cdot \mathbb{P}(C)\)

Remark:

On your homework you will show that \((1) \not \Rightarrow (2)\) and \((2) \not \Rightarrow (1)\).

Probability at least one smoker

Example 2

Suppose you take a random sample of \(n\) people, of which people are smokers and non-smokers independently of each other. Let

  • \(A_i =\) event person \(i\) is a smoker, for \(i=1, \ldots ,n\), and

  • \(p_i =\) probability person \(i\) is a smoker, for \(i=1, \ldots ,n\).

Find the probability that at least one person in the random sample is a smoker.

Learning objectives

  1. Define independence of 2-3 events and decide whether two or more events are independent
  1. Define key facts for conditional probabilities and calculate conditional probabilities.
  1. Calculate the probability of an event using Bayes’ Theorem, Higher Order Multiplication Rule, and the Law of Total Probabilities

Conditional Probability facts (1/2)

Fact 1: General Multiplication Rule

\[\mathbb{P}(A\cap B)=\mathbb{P}(A)\cdot\mathbb{P}(B|A)\]

Fact 2: Conditional Probability Definition

\[\mathbb{P}(A|B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}\]

Conditional Probability facts (2/2)

Fact 3

If \(A\) and \(B\) are independent events (\(A \unicode{x2AEB}B\)), then \[\mathbb{P}(A|B) = \mathbb{P}(A)\]

Fact 4

\(\mathbb{P}(A|B)\) is a probability, meaning that it satisfies the probability axioms. In particular, \[\mathbb{P}(A|B) + \mathbb{P}(A^C|B) = 1\]

Conditional probability with two dice

Example 3

Two dice (green and blue) are rolled. If the dice do not show the same face, what is the probability that one of the dice is a 1?

Conditional probability with two dice: simulations

Example 3

Two dice (green and blue) are rolled. If the dice do not show the same face, what is the probability that one of the dice is a 1?

set.seed(1002)
rolls = replicate(reps, sample(x = 1:6, size = 2, replace = TRUE))
rolls[, 1:10]
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]    4    5    5    5    6    5    2    4    3     4
[2,]    1    4    6    3    1    1    5    5    6     3
event_A = ( rolls[1, ] != rolls[2, ] )
head(event_A, 10)
 [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
event_B = ( rolls[1, ] == 1 | rolls[2, ] == 1 )
head(event_B, 10)
 [1]  TRUE FALSE FALSE FALSE  TRUE  TRUE FALSE FALSE FALSE FALSE
sum(event_B & event_A) / sum(event_A)
[1] 0.3315328

Monty Hall Problem

Survivor Season 42

With the Wiki page on it!

Learning objectives

  1. Define independence of 2-3 events and decide whether two or more events are independent

  2. Define key facts for conditional probabilities and calculate conditional probabilities.

  1. Calculate the probability of an event using Bayes’ Theorem, Higher Order Multiplication Rule, and the Law of Total Probabilities

Bayes’ Rule for two events

  • We can use the conditional probability (\(\mathbb{P}(A|B)\)) to get information on the flipped conditional probability (\(\mathbb{P}(B|A)\))

Theorem: Bayes’ Rule (for two events)

For any two events \(A\) and \(B\) with nonzero probabilties,

\[\mathbb{P}(A| B) = \frac{\mathbb{P}(A) \cdot \mathbb{P}(B|A)} {\mathbb{P}(B)}\]

Calculating probability with Higher Order Multiplication Rule

Example 4

Suppose we draw 5 cards from a standard shuffled deck of 52 cards. What is the probability of a flush, that is all the cards are of the same suit (including straight flushes)?

Higher Order Multiplication Rule

\[\begin{aligned} \mathbb{P}(A_1\cap & A_2 \cap \ldots \cap A_n)= \\ & \mathbb{P}(A_1)\cdot\mathbb{P}(A_2|A_1) \cdot \\ & \mathbb{P}(A_3|A_1A_2)\ldots \cdot\mathbb{P}(A_n|A_1A_2\ldots A_{n-1}) \end{aligned}\]

Calculating probability with Law of Total Probability

Example 5

Suppose 1% of people assigned female at birth (AFAB) and 5% of people assigned male at birth (AMAB) are color-blind. Assume person born is equally likely AFAB or AMAB (not including intersex). What is the probability that a person chosen at random is color-blind?

Law of Total Probability for 2 Events

For events \(A\) and \(B\),

\[%\left( \begin{array}{ccl} \mathbb{P}(B)&=&\mathbb{P}(B \cap A) + \mathbb{P}(B \cap A^C)\\ &=& \mathbb{P}(B|A) \cdot \mathbb{P}(A)+ \mathbb{P}(B | A^C)\cdot \mathbb{P}(A^C) \end{array} %\right)\]

General Law of Total Proability

Law of Total Probability (general)

If \(\{A_i\}_{i=1}^{n} = \{A_1, A_2, \ldots, A_n\}\) form a partition of the sample space, then for event \(B\),

\[%\left( \begin{array}{ccl} \mathbb{P}(B)&=& \sum_{i=1}^{n} \mathbb{P}(B \cap A_i)\\ &=& \sum_{i=1}^{n} \mathbb{P}(B|A_i) \cdot \mathbb{P}(A_i) \end{array} %\right)\]

Calculating probability with generalized Law of Total Probability

Calculate probability with both rules

Example 6

Suppose

  • 1% of people who are AFAB aged 40-50 years have breast cancer,

  • an AFAB person with breast cancer has a 90% chance of a positive test from a mammogram, and

  • an AFAB person has a 10% chance of a false-positive result from a mammogram.

What is the probability that an AFAB person has breast cancer given that they just had a positive test?

Bayes’ Rule

Theorem: Bayes’ Rule

If \(\{A_i\}_{i=1}^{n}\) form a partition of the sample space \(S\), with \(\mathbb{P}(A_i)>0\) for \(i=1\ldots n\) and \(\mathbb{P}(B)>0\), then

\[\mathbb{P}(A_j | B) = \frac{\mathbb{P}(B|A_j) \cdot \mathbb{P}(A_j)} {\sum_{i=1}^{n} \mathbb{P}(B|A_i) \cdot \mathbb{P}(A_i)}\]