Multivariate Normal Distribution Basics for Exam P

The Bivariate Normal Distribution

The bivariate normal distribution for (X, Y) is characterized by five parameters: mu_X, mu_Y, sigma_X^2, sigma_Y^2, and the correlation rho (where -1 < rho < 1). The joint PDF has a complex formula involving a quadratic form in the exponent, but for Exam P, the key properties matter more than the PDF itself.

The five parameters have clear interpretations: mu_X and mu_Y are the marginal means, sigma_X^2 and sigma_Y^2 are the marginal variances, and rho measures the linear dependence between X and Y. Together, these five numbers completely specify the joint distribution.

The bivariate normal is the most important joint distribution in probability and statistics. It arises naturally when two variables are influenced by many small, independent factors (by a multivariate version of the Central Limit Theorem). In actuarial applications, it can model the joint distribution of losses in two correlated lines of business.

Marginal Distributions

A fundamental property of the bivariate normal is that both marginal distributions are normal: X ~ Normal(mu_X, sigma_X^2) and Y ~ Normal(mu_Y, sigma_Y^2). The marginal distributions do not depend on the correlation rho. Knowing that (X, Y) is bivariate normal with specified parameters immediately tells you the distribution of each variable individually.

However, the converse is not true: having normal marginals does not guarantee that the joint distribution is bivariate normal. It is possible to construct joint distributions where X and Y are each marginally normal but (X, Y) is not jointly normal. On Exam P, the problem must specify that the joint distribution is bivariate normal (or multivariate normal) for you to use the special properties discussed in this article.

Linear Combinations

Any linear combination aX + bY of jointly normal variables is also normal, with mean a*mu_X + b*mu_Y and variance a^2*sigma_X^2 + b^2*sigma_Y^2 + 2ab*rho*sigma_X*sigma_Y. This property extends to any finite linear combination of jointly normal variables.

Example: If (X, Y) is bivariate normal with mu_X = 50, mu_Y = 30, sigma_X = 10, sigma_Y = 5, and rho = 0.4, find P(X + Y > 90). The sum X + Y is normal with mean 80 and variance 100 + 25 + 2(1)(1)(0.4)(10)(5) = 165. Standard deviation = sqrt(165) = 12.85. Z = (90 - 80)/12.85 = 0.778. P(X + Y > 90) = 1 - Phi(0.778) = 1 - 0.7817 = 0.218.

The difference X - Y is also normal: mean = mu_X - mu_Y = 20 and variance = sigma_X^2 + sigma_Y^2 - 2*rho*sigma_X*sigma_Y = 100 + 25 - 40 = 85. Notice the minus sign in front of the covariance term when computing the variance of a difference. This is a frequent source of errors on Exam P.

Conditional Distributions

For bivariate normal (X, Y), the conditional distribution of Y given X = x is normal: Y | X = x ~ Normal(mu_Y + rho*(sigma_Y/sigma_X)*(x - mu_X), sigma_Y^2*(1 - rho^2)). Two key features stand out: the conditional mean is a linear function of x, and the conditional variance does not depend on x.

The conditional mean mu_Y + rho*(sigma_Y/sigma_X)*(x - mu_X) can be interpreted as follows: start with the marginal mean mu_Y, then adjust by an amount proportional to how far x is from its own mean mu_X. The adjustment factor rho*(sigma_Y/sigma_X) is the regression coefficient of Y on X.

The conditional variance sigma_Y^2*(1 - rho^2) is always less than or equal to the marginal variance sigma_Y^2. The reduction factor (1 - rho^2) measures how much knowing X reduces the uncertainty about Y. When rho = 0, no reduction occurs (X provides no information about Y). When |rho| = 1, the conditional variance is 0, meaning Y is a deterministic linear function of X.

Example: If X and Y are bivariate normal with mu_X = 100, mu_Y = 80, sigma_X = 10, sigma_Y = 15, and rho = 0.6, then E[Y|X = 120] = 80 + 0.6*(15/10)*(120 - 100) = 80 + 0.9*20 = 80 + 18 = 98. Var(Y|X = 120) = 225*(1 - 0.36) = 225*0.64 = 144, so the conditional standard deviation is 12.

To find P(Y > 110 | X = 120): standardize using the conditional distribution. Z = (110 - 98)/12 = 1.0. P(Y > 110 | X = 120) = 1 - Phi(1.0) = 0.1587.

Uncorrelated Implies Independent (for Joint Normal Only)

For jointly normal random variables, uncorrelated (rho = 0) implies independent. This equivalence is unique to the normal distribution and does not hold in general. For a general joint distribution, zero correlation only means the absence of a linear relationship; the variables could still be dependent through a nonlinear relationship.

This property means that for bivariate normal variables, checking independence reduces to checking whether the correlation is zero. If rho = 0, you can factor the joint density into the product of the marginals, and all conditional distributions equal the marginal distributions.

On Exam P, be careful not to apply this result when the distribution is not specified as jointly normal. A problem that simply says "X and Y are uncorrelated" does not allow you to conclude independence unless it also states that the joint distribution is normal.

The Best Linear Predictor

The conditional mean E[Y|X = x] = mu_Y + rho*(sigma_Y/sigma_X)*(x - mu_X) is the best predictor of Y given X in the sense that it minimizes E[(Y - predictor)^2]. For jointly normal variables, this best predictor happens to be linear, which is a special property of the normal distribution.

For general (non-normal) joint distributions, the best predictor E[Y|X = x] may not be linear. However, the best linear predictor (the linear function of x that minimizes mean squared error) always has the same form as the bivariate normal conditional mean: mu_Y + rho*(sigma_Y/sigma_X)*(x - mu_X). The minimum mean squared prediction error for the best linear predictor is sigma_Y^2*(1 - rho^2), regardless of the joint distribution.

Common Pitfalls

The most common mistake is assuming bivariate normality when it is not stated. The special properties (uncorrelated implies independent, conditional distributions are normal, linear combinations are normal) require the joint distribution to be bivariate normal, not just that each marginal is normal.

Another error is forgetting the covariance term when computing the variance of a linear combination. Var(aX + bY) = a^2*Var(X) + b^2*Var(Y) + 2ab*Cov(X,Y), where Cov(X,Y) = rho*sigma_X*sigma_Y. The sign of the covariance term changes when computing the variance of a difference.

Students sometimes confuse the regression coefficient rho*(sigma_Y/sigma_X) (used in the conditional mean of Y given X) with the correlation rho itself. The regression coefficient scales the correlation by the ratio of standard deviations. Similarly, the regression coefficient of X on Y is rho*(sigma_X/sigma_Y), which is generally different from the regression coefficient of Y on X.

Study Tips

For Exam P, memorize the five parameters of the bivariate normal and the formulas for the conditional mean and conditional variance. Practice computing probabilities involving linear combinations and conditional distributions. Know that uncorrelated implies independent only for jointly normal variables. Work through problems that require computing P(aX + bY > c) and P(Y > c | X = x), as these are the most common question types. These problems typically take three to four minutes and reward students who can quickly write down the conditional mean and variance without rederiving them.