Lesson 18: Moment Generating Functions

Meike Niederhausen and Nicky Wakim

2025-12-03

Learning Objectives

  1. Learn the definition of a moment-generating function.
  2. Find the moment-generating function of a random variable.
  3. Use a moment-generating function to find the mean and variance of a random variable.

Where are we?

What are moments?

Definition 1

The \(j^{th}\) moment of a r.v. \(X\) is \(\mathbb{E}[X^j]\)

Okay, but what are they?

Example 1

\(1^{st}-4^{th}\) moments

  1. 1st moment:

 

  1. 2nd moment:

 

  1. 3rd moment:

 

  1. 4th moment:

What is a moment generating function (MGF)??

Definition 3

If \(X\) is a r.v., then the moment generating function (MGF) associated with \(X\) is: \[M_X(t)= \mathbb{E}[e^{tX}]\]

Remarks

  • For a discrete r.v., the MGF of \(X\) is \[M_X(t)= \mathbb{E}[e^{tX}]=\sum_{all \ x}e^{tx}p_X(x)\]

  • For a continuous r.v., the MGF of \(X\) is \[M_X(t)= \mathbb{E}[e^{tX}]=\int_{-\infty}^{\infty}e^{tx}f_X(x)dx\]

  • The MGF \(M_X(t)\) is a function of \(t\), not of \(X\), and it might not be defined (i.e. finite) for all values of \(t\). We just need it to be defined for \(t=0\).

Example

Example 4

What is \(M_X(t)\) for \(t=0\)?

How do MGFs give us moments?

Theorem 5

The moment generating function uniquely specifies a probability distribution. AKA all moments can be found from the MGF through its derivatives at \(t=0\).

Theorem 6

\[\mathbb{E}[X^r] = M_X^{(r)}(0)\]

\((r)\) in this equation is the \(r\)th derivative with respect to \(t\). We calculate the derivative at \(t=0\)

  • When \(r=1\), we are taking the first derivative
  • When \(r=4\), we are taking the fourth derivative

Using the MGF to uniquely describe a probability distribution

Example 7

Let \(X \sim Poisson(\lambda)\)

  1. Find the MGF of \(X\)

  2. Find \(\mathbb{E}[X]\)

  3. Find \(Var(X)\)

Theorem

Remark: Finding the mean and variance is sometimes easier with the following trick

Theorem 8

Let \(R_X(t) = \ln[M_X(t)]\). Then,

\[\mu = \mathbb{E}[X] = R_X'(0) \text{, and}\] \[\sigma^2 = Var(X) = R_X''(0)\]

Proof.

Using \(R_X(t)\) to uniquely describe a probability distribution

Example 9

Let \(X \sim Poisson(\lambda)\).

  1. Find \(\mathbb{E}[X]\) using \(R_X(t)\)

  2. Find \(Var(X)\) using \(R_X(t)\)

Using the MGF to uniquely describe the standard normal distribution

Example 10

Let \(Z\) be a standard normal random variable, i.e. \(Z \sim N(0,1)\).

  1. Find the MGF of \(Z\)

  2. Find \(\mathbb{E}[Z]\)

  3. Find \(Var(Z)\)

Using the MGF to uniquely describe the standard normal distribution

MGFs of sums of independent RV’s

Theorem 9

If \(X\) and \(Y\) are independent RV’s with respective MGFs \(M_X(t)\) and \(M_Y(t)\), then

\[M_{X+Y}(t) = E[e^{t(X+Y)}] = E[e^{tX} e^{tY}] = E[e^{tX}]E[e^{tY}]=M_{X}(t)M_{Y}(t)\]

Main takeaways

  • MGFs are a purely mathematically definition

    • We can’t really relate it to our real world analysis

  • They are helpful mathematically because they are unique to a probability distribution

    • We can find the unique MGF from for a probability distribution
    • And we can find a distribution from an MGF

  • MGFs can sometimes make it easier to find the mean and variance of an RV

  • MGFs are most helpful when we are finding a joint distribution that is a sum or transformation of two RV’s

    • Make the calculation easier!

  • MGFs are often used to prove certain distribution are sums of other ones!

More resources