Central Limit Theorem: Statement, Proof Sketch, and Applications

Statement of the CLT

Let X1, X2, ..., Xn be independent and identically distributed random variables with mean mu and finite variance sigma^2. The Central Limit Theorem (CLT) states that the standardized sample mean (X_bar - mu) / (sigma / sqrt(n)) converges in distribution to a standard normal N(0, 1) as n approaches infinity. Equivalently, for large n, X_bar is approximately Normal(mu, sigma^2/n).

The CLT also applies to sums: S_n = X1 + ... + Xn is approximately Normal(n*mu, n*sigma^2). The two forms are equivalent since X_bar = S_n/n, and a linear transformation of a normal variable is normal. Either form may appear on Exam P, so be comfortable with both.

The remarkable power of the CLT is that it works regardless of the original distribution of the Xi. Whether the individual variables are exponential, uniform, Poisson, or any other distribution with finite variance, the sum (or average) of enough of them will be approximately normal. This universality makes the CLT one of the most important theorems in probability and statistics.

Proof Sketch Using MGFs

While a full proof of the CLT is beyond the scope of Exam P, understanding the idea behind the proof deepens your intuition. The proof uses moment generating functions. Let Z_n = (S_n - n*mu) / (sigma*sqrt(n)) be the standardized sum. The MGF of Z_n is M_{Z_n}(t) = [M_{(X-mu)/(sigma)}(t/sqrt(n))]^n.

Expanding the inner MGF in a Taylor series around t = 0: M_{(X-mu)/sigma}(s) = 1 + s^2/2 + higher order terms (since the standardized variable has mean 0 and variance 1). Setting s = t/sqrt(n): M_{Z_n}(t) = [1 + t^2/(2n) + ...]^n. As n approaches infinity, this converges to e^(t^2/2), which is the MGF of N(0, 1). By the uniqueness of MGFs, Z_n converges in distribution to a standard normal.

The key insight is that the higher-order terms become negligible as n grows, and only the first two moments (mean and variance) of the original distribution affect the limiting distribution. This explains why the CLT requires only finite mean and variance, not knowledge of the entire distribution.

Applying the CLT on Exam P

The typical Exam P CLT problem specifies a distribution (not necessarily normal), gives n (usually n >= 30), and asks for an approximate probability involving the sample mean or sum. Follow these steps systematically:

• Step 1: Find mu = E[Xi] and sigma^2 = Var(Xi) for the individual random variable.
• Step 2: Determine whether the question asks about the sum S_n or the average X_bar.
• Step 3: Compute the mean and variance of S_n (which are n*mu and n*sigma^2) or X_bar (which are mu and sigma^2/n).
• Step 4: Standardize by computing Z = (value - mean) / standard deviation.
• Step 5: Use the standard normal table to find the probability.

Example: Claim sizes have mean 500 and standard deviation 200. For 100 independent claims, find P(total > 52,000). By the CLT, S ~ Normal(50000, 100 * 40000) = Normal(50000, 4000000). The standard deviation of S is sqrt(4000000) = 2000. Z = (52000 - 50000)/2000 = 1.0. P(S > 52000) = 1 - Phi(1.0) = 1 - 0.8413 = 0.1587.

The Continuity Correction

When the underlying distribution is discrete, the CLT approximation can be improved by applying a continuity correction. The idea is that the integer value k in the discrete distribution occupies the interval (k - 0.5, k + 0.5) on the continuous normal scale.

For a discrete sum S, the corrections are:

• P(S >= k) is approximated by P(Z >= (k - 0.5 - mu_S) / sigma_S)
• P(S <= k) is approximated by P(Z <= (k + 0.5 - mu_S) / sigma_S)
• P(S = k) is approximated by P((k - 0.5 - mu_S)/sigma_S < Z < (k + 0.5 - mu_S)/sigma_S)

Example: Roll a fair die 100 times. Find P(sum >= 370). Each roll has mu = 3.5 and sigma^2 = 35/12. The sum has mean 350 and variance 100 * 35/12 = 2916.67, so sigma_S = 17.11. With continuity correction: Z = (369.5 - 350)/17.11 = 1.14. P(S >= 370) = P(Z >= 1.14) = 1 - 0.8729 = 0.1271. Without the correction, Z = (370 - 350)/17.11 = 1.17, giving 0.1210. The correction matters more for smaller n.

How Large Must n Be?

The CLT does not specify exactly how large n must be for the normal approximation to be adequate. The common guideline is n >= 30, but this depends on the shape of the underlying distribution. Symmetric distributions (like the uniform) converge quickly and may give good approximations with n as small as 10. Highly skewed distributions (like the exponential or Pareto) may need n >= 50 or more.

For the binomial distribution, the traditional rule of thumb is np >= 5 and n(1-p) >= 5. For the Poisson distribution, lambda >= 10 is typically sufficient. These distribution-specific guidelines reflect the fact that skewness (which equals 1/sqrt(lambda) for the Poisson) determines the rate of convergence.

On Exam P, when a problem states "use the normal approximation" or gives a large n, you should apply the CLT without worrying about whether n is large enough. The problem is telling you to use the approximation.

CLT for Non-Identical Distributions

A generalized version of the CLT (the Lindeberg-Feller CLT) applies to independent but not necessarily identically distributed random variables, provided certain regularity conditions hold. While this generalization is not directly tested on Exam P, it justifies using the normal approximation in scenarios where risks in a portfolio have slightly different distributions.

In practice, if X1, X2, ..., Xn are independent with means mu_i and variances sigma_i^2, then S_n = X1 + ... + Xn is approximately Normal(sum of mu_i, sum of sigma_i^2) for large n, provided no single variable dominates the sum. This is relevant in insurance, where policies in a portfolio may have different characteristics.

Common Pitfalls

A frequent error is confusing the variance of the sum with the variance of the average. Var(S_n) = n * sigma^2 (grows with n), while Var(X_bar) = sigma^2/n (shrinks with n). The standard deviation of S_n is sigma*sqrt(n), while the standard deviation of X_bar is sigma/sqrt(n). Mixing these up leads to Z-scores that are off by a factor of n.

Another mistake is applying the CLT when the variance is infinite. Distributions like the Cauchy and certain Pareto distributions do not have finite variance, so the CLT does not apply. If a problem specifies a distribution with infinite variance and asks about the sum, a different limit theorem (the stable distribution limit) applies, but this is not tested on Exam P.

Students also sometimes forget to check whether the problem asks for the probability involving the sum or the average. Reading the problem carefully before setting up the calculation avoids this error.

Study Tips

For Exam P, practice the five-step CLT procedure until it becomes automatic. Work through examples with different underlying distributions (uniform, exponential, Poisson, binomial) to build confidence. Memorize the key z-values: z_0.90 = 1.282, z_0.95 = 1.645, z_0.975 = 1.960, z_0.99 = 2.326. Practice both the sum version and the average version of the CLT. Know when to apply the continuity correction (discrete underlying distribution) and when to skip it (continuous underlying distribution). CLT problems should take about three minutes on exam day.