Chapter 43: Moment Generating Functions Part 1

What are moments?

Definition 1. The \(j^{th}\) moment of a r.v. \(X\) is \(\mathbb{E}[X^j]\).

Example 2. \(1^{st}-4^{th}\) moments.

What is a moment generating function (mgf)?

Definition 3. If \(X\) is a r.v., then \[M_X(t)= \mathbb{E}[e^{tX}]\] is the moment generating function (mgf) associated with \(X\).

Remarks

For a discrete r.v., the mgf of \(X\) is \[M_X(t)= \mathbb{E}[e^{tX}]=\sum_{all \ x}e^{tx}p_X(x)\]
For a continuous r.v., the mgf of \(X\) is \[M_X(t)= \mathbb{E}[e^{tX}]=\int_{-\infty}^{\infty}e^{tx}f_X(x)dx\]
The mgf \(M_X(t)\) is a function of \(t\), not of \(X\), and it might not be defined (i.e. finite) for all values of \(t\). We just need it to be defined for \(t=0\).

Example 4. What is \(M_X(t)\) for \(t=0\)?

Theorem 5. The moment generating function uniquely specifies a probability distribution.

Theorem 6. \[\mathbb{E}[X^r] = M_X^{(r)}(0)\]

Proof. Proof. ◻

Example 7. Let \(X \sim Poisson(\lambda)\).

Remark

Finding the mean and variance is sometimes easier with the following trick.

Theorem 8. Let \[R_X(t) = \ln[M_X(t)]\]

Then,

\[\mu = \mathbb{E}[X] = R_X'(0)\] and \[\sigma^2 = Var(X) = R_X''(0)\]

Proof. Proof. ◻

Example 9. Let \(X \sim Poisson(\lambda)\).

Example 10. Let \(Z\) be a standard normal random variable, i.e. \(Z \sim N(0,1)\).