Moment Generating Functions: Theory and Applications

Definition and Core Properties

The moment generating function (MGF) of a random variable X is M(t) = E[e^(tX)], defined for all t in some neighborhood of 0. For discrete X, M(t) = sum of e^(tx) * p(x). For continuous X, M(t) = integral of e^(tx) * f(x) dx. Not every distribution has an MGF (it may not converge for any t other than 0), but when it exists, it is an extraordinarily powerful tool.

The name "moment generating function" comes from its ability to generate moments through differentiation. The n-th derivative of M(t) evaluated at t = 0 gives the n-th moment: E[X^n] = M^(n)(0). In particular, E[X] = M'(0) and E[X^2] = M''(0), from which you can compute the variance as M''(0) - (M'(0))^2.

The most important property for Exam P is the uniqueness theorem: if the MGF of X exists in some neighborhood of 0, it uniquely determines the distribution of X. This means that if two random variables have the same MGF, they have the same distribution. This allows you to identify a distribution by recognizing the algebraic form of its MGF.

Computing Moments from the MGF

To find E[X] from the MGF, differentiate M(t) with respect to t and evaluate at t = 0. For example, if M(t) = exp(3(e^t - 1)), then M'(t) = 3e^t * exp(3(e^t - 1)). At t = 0: M'(0) = 3 * 1 = 3. So E[X] = 3.

For the second moment, M''(t) = (3e^t)^2 * exp(3(e^t - 1)) + 3e^t * exp(3(e^t - 1)) = exp(3(e^t - 1)) * (9e^(2t) + 3e^t). At t = 0: M''(0) = 1 * (9 + 3) = 12. So E[X^2] = 12 and Var(X) = 12 - 9 = 3. Recognizing that the MGF has the Poisson form with lambda = 3 gives the same answer immediately.

An alternative to direct differentiation is to use the cumulant generating function K(t) = ln(M(t)). The first cumulant K'(0) = E[X] and the second cumulant K''(0) = Var(X). For the Poisson example, K(t) = 3(e^t - 1), so K'(0) = 3 and K''(0) = 3, confirming E[X] = Var(X) = 3. The cumulant approach gives the variance directly without computing E[X^2] first.

MGFs of Common Distributions

Memorize these for Exam P, as identifying a distribution from its MGF is a frequently tested skill:

• Bernoulli(p): M(t) = 1 - p + p * e^t = q + p * e^t
• Binomial(n, p): M(t) = (1 - p + p * e^t)^n
• Poisson(lambda): M(t) = exp(lambda * (e^t - 1))
• Geometric(p): M(t) = p * e^t / (1 - (1-p) * e^t) for t < -ln(1-p)
• Exponential(lambda): M(t) = lambda / (lambda - t) for t < lambda
• Gamma(alpha, lambda): M(t) = (lambda / (lambda - t))^alpha for t < lambda
• Normal(mu, sigma^2): M(t) = exp(mu * t + sigma^2 * t^2 / 2)
• Uniform(a, b): M(t) = (e^(tb) - e^(ta)) / (t(b - a))

To identify a distribution from its MGF, match the algebraic form to one of these standard expressions and read off the parameters. For example, if M(t) = (5/(5 - t))^3, this matches the Gamma form with lambda = 5 and alpha = 3, so X ~ Gamma(3, 5) with mean 3/5 and variance 3/25.

MGFs of Sums of Independent Variables

The MGF of a sum of independent random variables equals the product of their individual MGFs: M_{X+Y}(t) = M_X(t) * M_Y(t). This multiplicative property makes the MGF the natural tool for determining the distribution of sums.

For example, if X ~ Poisson(3) and Y ~ Poisson(5) are independent, then M_{X+Y}(t) = exp(3(e^t - 1)) * exp(5(e^t - 1)) = exp(8(e^t - 1)). This is the MGF of a Poisson(8), confirming that the sum of independent Poisson variables is Poisson with the sum of the rates.

Similarly, for X ~ Normal(mu_1, sigma_1^2) and Y ~ Normal(mu_2, sigma_2^2) independent, M_{X+Y}(t) = exp((mu_1 + mu_2)t + (sigma_1^2 + sigma_2^2)t^2/2), which is Normal(mu_1 + mu_2, sigma_1^2 + sigma_2^2). The normal family is closed under addition of independent variables.

For Gamma variables with the same rate: if X ~ Gamma(alpha_1, lambda) and Y ~ Gamma(alpha_2, lambda), then M_{X+Y}(t) = (lambda/(lambda - t))^(alpha_1 + alpha_2), so X + Y ~ Gamma(alpha_1 + alpha_2, lambda). The rate parameters must be equal for this to work. If the rates differ, the sum is not Gamma.

MGF of Linear Transformations

If Y = aX + b, then M_Y(t) = e^(bt) * M_X(at). This follows directly from the definition: E[e^(t(aX+b))] = e^(bt) * E[e^(atX)] = e^(bt) * M_X(at). This property is useful for standardizing random variables or changing units.

For the special case of Z = (X - mu)/sigma (standardizing), M_Z(t) = e^(-mu*t/sigma) * M_X(t/sigma). If X ~ Normal(mu, sigma^2), then M_Z(t) = e^(-mu*t/sigma) * exp(mu*t/sigma + sigma^2 * t^2/(2*sigma^2)) = exp(t^2/2), which is the MGF of the standard Normal(0, 1).

When the MGF Does Not Exist

Some distributions do not have an MGF because E[e^(tX)] = infinity for all t other than 0. The Cauchy distribution and the Pareto distribution (with certain parameters) are common examples. For the Pareto with alpha <= 1, even E[X] does not exist, so the MGF certainly does not.

When the MGF does not exist, alternative tools include the characteristic function phi(t) = E[e^(itX)] (which always exists) and the probability generating function P(z) = E[z^X] (for non-negative integer-valued random variables). Exam P primarily tests MGFs, but you should be aware that not every distribution has one.

Common Pitfalls

A frequent mistake is forgetting the domain restriction on t. The exponential MGF M(t) = lambda/(lambda - t) is only valid for t < lambda. Evaluating at t = lambda or beyond gives a meaningless result. On Exam P, domain restrictions rarely cause problems, but they explain why certain manipulations work only for specific values of t.

Another error is trying to multiply MGFs of dependent random variables. The product formula M_{X+Y}(t) = M_X(t) * M_Y(t) requires independence. If X and Y are dependent, you must compute M_{X+Y}(t) = E[e^(t(X+Y))] directly from the joint distribution.

Students sometimes confuse the MGF with the probability generating function (PGF). The PGF is G(z) = E[z^X] and is related to the MGF by G(z) = M(ln(z)). They encode the same information but in different forms. Exam P typically works with MGFs.

Study Tips

For Exam P, memorize the MGFs of all major distributions and practice identifying distributions from their MGFs. Work through problems that ask you to find the distribution of a sum using the product property. Practice computing E[X] and Var(X) from an MGF using both direct differentiation and the cumulant approach. A systematic strategy for MGF problems: (1) simplify the given MGF algebraically, (2) match to a known form, (3) read off the parameters, (4) report the requested quantity (mean, variance, or probability). This approach should take about two to three minutes per problem on exam day.