Convolutions and Sums of Random Variables for Exam P

The Convolution Formula

If X and Y are independent continuous random variables, the PDF of Z = X + Y is the convolution: f_Z(z) = integral of f_X(x) * f_Y(z - x) dx. For discrete variables, the analogous formula is p_Z(z) = sum over x of p_X(x) * p_Y(z - x). The convolution formula follows from the CDF method: P(Z <= z) = P(X + Y <= z) = double integral over x + y <= z of f_X(x) * f_Y(y) dy dx, and differentiating with respect to z yields the convolution integral.

The limits of integration in the convolution formula deserve careful attention. For non-negative random variables (X >= 0 and Y >= 0), f_Z(z) = integral from 0 to z of f_X(x) * f_Y(z - x) dx, since both f_X(x) = 0 for x < 0 and f_Y(z - x) = 0 for z - x < 0 (i.e., x > z).

Example: If X and Y are independent Uniform(0, 1), the convolution gives f_Z(z) = integral from 0 to z of 1 * 1 dx = z for 0 < z < 1, and f_Z(z) = integral from z-1 to 1 of 1 * 1 dx = 2 - z for 1 < z < 2. The result is a triangular distribution on (0, 2) with mode at 1.

Discrete Convolutions

For discrete random variables, convolutions involve sums rather than integrals. If X takes values 0, 1, 2, ... and Y takes values 0, 1, 2, ..., then P(Z = z) = sum from k=0 to z of P(X = k) * P(Y = z - k).

Example: Let X and Y be independent, each taking values 0 and 1 with P(X = 0) = P(X = 1) = 0.5 and P(Y = 0) = 0.3, P(Y = 1) = 0.7. Then Z = X + Y takes values 0, 1, 2 with: P(Z = 0) = 0.5 * 0.3 = 0.15, P(Z = 1) = 0.5 * 0.7 + 0.5 * 0.3 = 0.50, P(Z = 2) = 0.5 * 0.7 = 0.35.

For Poisson random variables, the discrete convolution produces a known result: if X ~ Poisson(lambda1) and Y ~ Poisson(lambda2) are independent, then Z ~ Poisson(lambda1 + lambda2). You can verify this by computing the convolution sum, but recognizing it from the MGF is much faster.

The MGF Method for Sums

The moment generating function method is usually the fastest approach: M_{X+Y}(t) = M_X(t) * M_Y(t) for independent X and Y. If you can identify the resulting MGF as belonging to a known distribution, the problem is solved without any integration.

This method works perfectly for "closed" families, distributions that are closed under addition of independent variables:

• Normal + Normal = Normal (any parameters)
• Poisson + Poisson = Poisson (add the rates)
• Gamma + Gamma = Gamma (add the shape parameters; rate parameters must match)
• Binomial + Binomial = Binomial (add the n values; p must match)
• Negative Binomial + Negative Binomial = Negative Binomial (add the r values; p must match)

Example: If X ~ Gamma(3, 2) and Y ~ Gamma(5, 2) are independent, M_{X+Y}(t) = (2/(2-t))^3 * (2/(2-t))^5 = (2/(2-t))^8, so X + Y ~ Gamma(8, 2). The shape parameters add because the rate parameters are equal.

If the rate or probability parameters do not match, the sum does not belong to the same family. For instance, if X ~ Gamma(3, 2) and Y ~ Gamma(5, 3) are independent (different rates), X + Y is not Gamma. In such cases, you would need the convolution integral or a different approach.

The CDF Method for Sums

When the convolution integral is difficult or the variables are not independent, the CDF method provides an alternative. Find F_Z(z) = P(X + Y <= z) = double integral over the region {(x, y): x + y <= z} of f_{X,Y}(x, y) dx dy, then differentiate to get f_Z(z).

For independent variables with non-negative support, the integration region is the triangle 0 <= x, 0 <= y, x + y <= z. The CDF method is identical to the convolution in this case but may suggest a different order of integration that simplifies the calculation.

The CDF method is especially useful when X and Y are not independent. If (X, Y) has joint density f(x, y) defined on a non-rectangular region, computing P(X + Y <= z) requires integrating over the intersection of the support with the half-plane x + y <= z. Drawing the region is essential for setting up the correct limits.

Example: If f(x, y) = 2 for 0 < x < y < 1 (X and Y are dependent), find the distribution of Z = X + Y. P(Z <= z) = double integral over {0 < x < y < 1, x + y <= z} of 2 dx dy. The limits depend on whether z < 1 or 1 <= z < 2, requiring two separate calculations.

Functions Other Than Sums

The CDF method extends to any function of two random variables. For Z = X * Y, find P(XY <= z) and differentiate. For Z = X/Y, find P(X/Y <= z) = P(X <= zY) (when Y > 0) and integrate. For Z = max(X, Y), P(max(X, Y) <= z) = P(X <= z, Y <= z) = F_{X,Y}(z, z), which simplifies to F_X(z) * F_Y(z) when X and Y are independent.

Similarly, P(min(X, Y) <= z) = 1 - P(X > z, Y > z) = 1 - S_X(z) * S_Y(z) for independent X and Y. These max and min formulas are closely related to order statistics and appear frequently on Exam P.

Example: If X and Y are independent Exponential(lambda), then P(min(X, Y) > z) = e^(-lambda*z) * e^(-lambda*z) = e^(-2*lambda*z), so min(X, Y) ~ Exponential(2*lambda). This confirms the exponential minimum result from the order statistics article.

Common Pitfalls

The most common mistake with convolutions is using incorrect integration limits. For non-negative variables, the limits are 0 to z (not 0 to infinity). For variables with bounded support, the limits may change depending on the value of z, requiring piecewise computation. Always verify that both f_X(x) > 0 and f_Y(z - x) > 0 within the integration limits.

Another error is trying to multiply MGFs for dependent variables. The product formula M_{X+Y}(t) = M_X(t) * M_Y(t) requires independence. If X and Y are dependent, you must compute M_{X+Y}(t) = E[e^(t(X+Y))] from the joint distribution.

Students sometimes confuse the convolution of two distributions with their mixture. The convolution gives the distribution of the sum X + Y, while the mixture gives the distribution of a randomly selected variable (X with probability p, Y with probability 1-p). These are fundamentally different operations.

Study Tips

For Exam P, make the MGF method your first choice whenever the variables are independent and belong to a known closed family. Use the convolution integral only when the MGF method does not apply or when the problem explicitly asks for the PDF of the sum. Know the CDF method as a fallback for dependent variables or non-standard functions (products, ratios, max, min). Practice the uniform convolution example and the exponential minimum result, as these are classic Exam P problems. Convolution problems typically take three to four minutes, with the main challenge being correct integration limits.