Covariance and Correlation: Measuring Dependence for Exam P

Covariance: Definition and Computation

The covariance of X and Y measures their linear association: Cov(X, Y) = E[(X - mu_X)(Y - mu_Y)] = E[XY] - E[X]*E[Y]. The shortcut formula E[XY] - E[X]*E[Y] is almost always faster to compute than the definition and is the standard approach on Exam P.

If X and Y are independent, then E[XY] = E[X]*E[Y], so Cov(X, Y) = 0. However, the converse is false: zero covariance does not imply independence. Covariance only detects linear relationships. Two variables can have a strong nonlinear relationship (for example, Y = X^2 where X is symmetric about zero) and still have zero covariance.

Positive covariance means X and Y tend to increase together, while negative covariance means one tends to decrease when the other increases. In insurance, the covariance between claim frequency and claim severity within a portfolio affects the variance of total losses.

Properties of Covariance

Covariance satisfies several algebraic properties that appear frequently on Exam P:

• Cov(X, X) = Var(X)
• Cov(X, Y) = Cov(Y, X) (symmetry)
• Cov(aX + b, cY + d) = ac * Cov(X, Y) (constants b and d do not affect covariance)
• Cov(X + Y, Z) = Cov(X, Z) + Cov(Y, Z) (linearity in each argument)
• Var(X + Y) = Var(X) + Var(Y) + 2 * Cov(X, Y)

The last property generalizes to n variables: Var(X1 + X2 + ... + Xn) = sum of Var(Xi) + 2 * sum over all pairs i < j of Cov(Xi, Xj). When all variables are pairwise uncorrelated (or independent), the cross terms vanish and variance is simply additive.

A useful special case: Var(X - Y) = Var(X) + Var(Y) - 2 * Cov(X, Y). The minus sign in X - Y produces a minus sign in front of the covariance term. Students sometimes forget this sign change, leading to errors on Exam P.

Computing Covariance from Joint Distributions

To compute Cov(X, Y) from a joint PDF f(x, y), you need three quantities: E[X], E[Y], and E[XY]. For continuous variables, E[XY] = double integral of x * y * f(x, y) dx dy. For discrete variables, E[XY] = sum over all (x, y) of x * y * p(x, y).

Example: If f(x, y) = 2 for 0 < x < y < 1, find Cov(X, Y). First, E[X] = integral from 0 to 1 of integral from x to 1 of 2x dy dx = integral from 0 to 1 of 2x(1 - x) dx = 1/3. By a similar calculation, E[Y] = 2/3. For E[XY] = integral from 0 to 1 of integral from x to 1 of 2xy dy dx = integral from 0 to 1 of x(1 - x^2) dx = 1/4. Therefore Cov(X, Y) = 1/4 - (1/3)(2/3) = 1/4 - 2/9 = 1/36.

An alternative approach uses the conditional expectation method. If you already know E[Y|X = x], then E[XY] = E[X * E[Y|X]]. This can sometimes simplify the double integral into a single integral.

Correlation Coefficient

The correlation coefficient rho(X, Y) = Cov(X, Y) / (sigma_X * sigma_Y) standardizes covariance to lie in the interval [-1, 1]. This standardization makes correlation dimensionless and allows comparison across different pairs of variables.

The extreme values rho = 1 and rho = -1 indicate a perfect linear relationship: Y = aX + b with a > 0 when rho = 1, and a < 0 when rho = -1. Any other value of rho between -1 and 1 indicates an imperfect linear relationship or no linear relationship at all (when rho = 0).

On Exam P, you might be given a joint distribution and asked to compute the correlation. The systematic steps are: (1) find E[X], E[Y], E[X^2], E[Y^2], and E[XY]; (2) compute Var(X) = E[X^2] - (E[X])^2 and Var(Y) = E[Y^2] - (E[Y])^2; (3) compute Cov(X, Y) = E[XY] - E[X]*E[Y]; (4) divide by the product of the standard deviations.

Continuing the example above: Var(X) = E[X^2] - (1/3)^2. E[X^2] = integral from 0 to 1 of integral from x to 1 of 2x^2 dy dx = integral from 0 to 1 of 2x^2(1 - x) dx = 1/6. So Var(X) = 1/6 - 1/9 = 1/18. Similarly, Var(Y) = 1/18. The correlation is rho = (1/36) / sqrt(1/18 * 1/18) = (1/36) / (1/18) = 1/2.

Covariance with Indicator Variables

For indicator random variables I_A and I_B (which take value 1 when their respective events occur and 0 otherwise), the covariance has a particularly clean form: Cov(I_A, I_B) = E[I_A * I_B] - E[I_A] * E[I_B] = P(A and B) - P(A) * P(B).

This formula is useful for computing the variance of a count. If N = I_1 + I_2 + ... + I_n counts how many of n events occur, then Var(N) = sum of Var(I_i) + 2 * sum over i < j of Cov(I_i, I_j). Each Var(I_i) = P(A_i)(1 - P(A_i)), and each covariance is P(A_i and A_j) - P(A_i) * P(A_j).

Example: Draw 5 cards from a standard deck. Let N be the number of aces drawn. Each I_i indicates whether the i-th card is an ace. By symmetry, each P(A_i) = 4/52 = 1/13 and each P(A_i and A_j) = (4/52)(3/51) = 12/2652 = 1/221. There are C(5, 2) = 10 pairs. So Var(N) = 5 * (1/13)(12/13) + 2 * 10 * (1/221 - 1/169). This indicator approach handles sampling without replacement elegantly.

Uncorrelated vs. Independent

A critical distinction for Exam P: uncorrelated (rho = 0) is strictly weaker than independent. Independence implies zero correlation, but zero correlation does not imply independence. The one major exception is the jointly normal distribution, where uncorrelated and independent are equivalent.

Classic counterexample: Let X ~ Uniform(-1, 1) and Y = X^2. Then E[XY] = E[X^3] = 0 (by symmetry) and E[X] * E[Y] = 0 * E[X^2] = 0, so Cov(X, Y) = 0. Yet X and Y are clearly dependent because Y is completely determined by X. This example demonstrates that covariance and correlation miss nonlinear dependencies.

On Exam P, when a problem states that two variables are uncorrelated, you can only conclude Var(X + Y) = Var(X) + Var(Y). You cannot factor the joint density or treat the variables as independent in any other way, unless the problem also specifies a jointly normal distribution.

Study Tips

For Exam P, practice computing covariance using the shortcut formula E[XY] - E[X]*E[Y] from both joint tables (discrete) and joint densities (continuous). Memorize the variance of a sum formula and its generalization to n variables. Know the indicator variable covariance formula for counting problems. Be prepared to distinguish between uncorrelated and independent in conceptual questions. When computing correlation, organize your work by listing all five expectations (E[X], E[Y], E[XY], E[X^2], E[Y^2]) before plugging into formulas. This systematic approach minimizes errors and should take about four minutes per problem on exam day.