Law of Total Expectation

The law of total expectation (also called the tower property, iterated expectation, or double expectation formula) states E[X] = E[E[X|Y]]. Here, E[X|Y] is itself a random variable (a function of the random variable Y), and taking its expectation over Y gives the unconditional mean E[X].

When Y is discrete with values y1, y2, ..., the formula becomes E[X] = sum of E[X|Y = yi] * P(Y = yi). When Y is continuous with PDF g(y), it becomes E[X] = integral of E[X|Y = y] * g(y) dy. In both cases, you are computing a weighted average of conditional means.

Example: Suppose the number of claims N ~ Poisson(Lambda) where Lambda ~ Gamma(2, 1). Then E[N] = E[E[N|Lambda]] = E[Lambda] = 2. Without the tower property, you would need to find the marginal distribution of N (which is Negative Binomial) and compute its mean directly. The tower property bypasses this entirely.

The law of total expectation extends to conditional expectations: E[X|Z] = E[E[X|Y, Z] | Z]. This "tower" can be stacked to any height. For Exam P, the two-level version E[X] = E[E[X|Y]] is sufficient for nearly all problems.

Law of Total Variance

The law of total variance (also called Eve's law or the variance decomposition formula) states: Var(X) = E[Var(X|Y)] + Var(E[X|Y]). This decomposes the total variance into two non-negative components:

E[Var(X|Y)]: the expected value of the within-group variance (how much X varies within each level of Y, on average)
Var(E[X|Y]): the variance of the group means (how much the conditional mean of X varies across levels of Y)

Both terms are always non-negative, so Var(X) >= E[Var(X|Y)] and Var(X) >= Var(E[X|Y]). The total variance is at least as large as either component alone.

Continuing the Poisson-Gamma example: since N | Lambda ~ Poisson(Lambda), we have E[N|Lambda] = Lambda and Var(N|Lambda) = Lambda. Therefore:

E[Var(N|Lambda)] = E[Lambda] = 2
Var(E[N|Lambda]) = Var(Lambda) = 2 (since Lambda ~ Gamma(2, 1) has variance 2)
Var(N) = 2 + 2 = 4

Notice that Var(N) = 4 > E[N] = 2, which means the marginal distribution of N has more variance than a Poisson with the same mean. This overdispersion is characteristic of Poisson mixtures and is a key concept in actuarial modeling.

Application Strategy for Exam P

On Exam P, whenever you see a hierarchical model (X depends on Theta, and Theta has its own distribution), immediately apply the total expectation and total variance formulas. Follow this systematic approach:

Step 1: Identify the conditional distribution of X given Theta. Write down E[X|Theta] and Var(X|Theta) as functions of Theta.
Step 2: Compute E[X] = E[E[X|Theta]] by taking the expectation of the function from Step 1 over the distribution of Theta.
Step 3: Compute E[Var(X|Theta)] by taking the expectation of Var(X|Theta) over Theta.
Step 4: Compute Var(E[X|Theta]) by finding the variance of the function E[X|Theta] over the distribution of Theta.
Step 5: Add the results from Steps 3 and 4 to get Var(X).

Worked Example: Binomial-Uniform Model

Suppose X | P ~ Binomial(10, P) where P ~ Uniform(0, 1). Find E[X] and Var(X).

Step 1: E[X|P] = 10P and Var(X|P) = 10P(1-P).

Step 2: E[X] = E[10P] = 10 * E[P] = 10 * 0.5 = 5.

Step 3: E[Var(X|P)] = E[10P(1-P)] = 10 * (E[P] - E[P^2]) = 10 * (0.5 - 1/3) = 10 * 1/6 = 5/3.

Step 4: Var(E[X|P]) = Var(10P) = 100 * Var(P) = 100 * 1/12 = 25/3.

Step 5: Var(X) = 5/3 + 25/3 = 30/3 = 10.

In Step 3, we used E[P^2] = Var(P) + (E[P])^2 = 1/12 + 1/4 = 1/3 for the Uniform(0, 1) distribution.

Common Conjugate Pairs

Several conditional-marginal pairings appear repeatedly on Exam P. Knowing the conditional moments for each pairing speeds up the application of the total expectation and variance formulas:

Poisson(Lambda) with Lambda ~ Gamma(alpha, beta): E[X|Lambda] = Lambda, Var(X|Lambda) = Lambda. Marginal: Negative Binomial.
Binomial(n, P) with P ~ Beta(a, b): E[X|P] = nP, Var(X|P) = nP(1-P). Marginal: Beta-Binomial.
Normal(Theta, sigma^2) with Theta ~ Normal(mu, tau^2): E[X|Theta] = Theta, Var(X|Theta) = sigma^2. Marginal: Normal(mu, sigma^2 + tau^2).
Exponential(1/Theta) with Theta ~ Gamma: E[X|Theta] = Theta, Var(X|Theta) = Theta^2.

For each pairing, the conditional moments are simple functions of the mixing parameter, and the formulas can be applied quickly once you recognize the structure.

Multiple Levels of Conditioning

Some Exam P problems involve three or more levels of hierarchy. For example, N | Lambda ~ Poisson(Lambda), Lambda | Alpha ~ Gamma(Alpha, 1), and Alpha ~ some distribution. In such cases, apply the tower property iteratively: E[N] = E[E[E[N|Lambda, Alpha] | Alpha]] = E[E[Lambda|Alpha]] = E[Alpha].

For variance with multiple levels, the formula extends: Var(N) = E[E[Var(N|Lambda, Alpha)|Alpha]] + E[Var(E[N|Lambda, Alpha]|Alpha)] + Var(E[E[N|Lambda, Alpha]|Alpha]). While this multi-level decomposition is rarely tested in its full generality on Exam P, the two-level version appears frequently.

Common Pitfalls

The most frequent error is confusing E[Var(X|Y)] with Var(E[X|Y]). These are two different quantities that play different roles in the decomposition. A helpful mnemonic: "EVVE" stands for E[V] + V[E], where V means Var. The first term averages the conditional variance, and the second term measures the variability of the conditional mean.

A third pitfall is applying the formulas when the conditional distribution is not well-defined. The formulas require that E[X|Y] and Var(X|Y) exist for all (or almost all) values of Y. If the conditional distribution does not have a finite mean or variance for some values of Y, the formulas may not apply.

Study Tips

For Exam P, memorize both formulas (total expectation and total variance) and practice the five-step strategy until it becomes automatic. Work through at least eight problems with different conjugate pairs. For each problem, clearly write out E[X|Theta] and Var(X|Theta) as functions of Theta before computing anything else. Check your answers by verifying that Var(X) > 0 and that it is consistent with the marginal distribution when you know what that distribution is. These problems typically take four to five minutes and are among the most frequently tested topics on Exam P.

Double Expectation and Total Variance Formulas