Advanced Simulation: Importance Sampling and Variance Reduction
Apply importance sampling and advanced variance reduction techniques for Exam MAS-II.
Why Variance Reduction Matters
Monte Carlo simulation estimates quantities by generating random samples and computing averages. The precision of these estimates depends on the number of samples: the standard error decreases as 1/sqrt(n), meaning that reducing the error by a factor of 10 requires 100 times as many samples. For complex actuarial models where each simulation run is computationally expensive, this brute-force approach can be prohibitively slow. Variance reduction techniques improve the precision of Monte Carlo estimates without increasing the number of samples, effectively making each sample more informative.
Variance reduction is especially important for estimating tail probabilities and risk measures. Estimating the probability of ruin, the VaR at the 99.9th percentile, or the expected shortfall of an extreme layer requires many samples in the tail of the distribution. With crude Monte Carlo, most samples fall in the body of the distribution and provide no information about the tail. Variance reduction techniques redirect computational effort to the regions of the distribution that matter most for the quantity being estimated.
Importance Sampling
Importance sampling changes the sampling distribution to concentrate samples where the integrand contributes most. To estimate E_f[h(X)] = integral of h(x)*f(x)dx, draw samples X_1, ..., X_n from an alternative distribution g (the importance distribution) and compute the weighted average (1/n) * sum of h(X_i) * f(X_i) / g(X_i). The ratio w(x) = f(x)/g(x) is the likelihood ratio or importance weight. The estimator is unbiased as long as g(x) > 0 wherever h(x)*f(x) is non-zero.
The optimal importance distribution is g*(x) proportional to |h(x)| * f(x), which concentrates sampling where the product of the function value and the original density is largest. In practice, this optimal distribution is rarely available (since computing it requires knowing the integral we are trying to estimate), but approximations can still provide dramatic variance reduction. For rare event estimation, a common approach is exponential tilting: shifting the mean of the sampling distribution toward the rare event region.
Importance Sampling for Rare Events
Consider estimating the probability of ruin for an insurance portfolio, P(S > c) where S is the aggregate loss and c is a large threshold. Under the original distribution, very few simulated values of S exceed c, producing an estimate with high relative error. Importance sampling addresses this by sampling from a distribution under which S exceeds c more frequently, then correcting with likelihood ratios.
For a compound Poisson model with Poisson rate lambda and severity distribution F_X, exponential tilting increases the Poisson rate and shifts the severity distribution to generate more large losses. The tilting parameter theta is chosen to maximize the probability of the rare event under the tilted distribution. The likelihood ratio for each simulated path corrects for the distributional change, producing an unbiased estimate of the original probability. This technique can reduce the number of samples needed by factors of thousands for very rare events.
Stratified Sampling
Stratified sampling divides the sample space into non-overlapping strata and samples from each stratum separately. If the strata are chosen so that the quantity of interest is relatively homogeneous within each stratum, the variance of the overall estimate is reduced compared to simple random sampling. The variance reduction is guaranteed: stratified sampling can never increase variance compared to simple random sampling (assuming the stratum means are estimated without bias).
Proportional allocation assigns samples to each stratum in proportion to the stratum's probability. Optimal allocation assigns samples in proportion to the stratum probability times the stratum standard deviation, putting more samples in strata where the quantity of interest is more variable. In actuarial applications, strata might be defined by loss ranges (small, medium, large), risk classes, or quantiles of a key risk driver.
Latin Hypercube Sampling
Latin hypercube sampling (LHS) extends the stratification concept to multiple dimensions. For d input variables each with n samples, LHS ensures that each variable's marginal distribution is uniformly represented by dividing its range into n equal-probability intervals and placing exactly one sample in each interval. The samples are then randomly paired across dimensions. This produces a space-filling design that avoids the clumping and gaps that can occur with simple random sampling in high dimensions.
LHS is particularly useful in DFA and capital modeling where many correlated risk factors (investment returns, claim frequencies, catastrophe counts) are simulated simultaneously. By ensuring that each risk factor's marginal distribution is well-represented, LHS reduces the variance of estimates that depend on individual risk factors while maintaining the specified correlation structure between factors.
Control Variates
The control variate method exploits a correlated random variable Y with known expectation E[Y] to reduce the variance of the estimator. The adjusted estimator is theta_hat_CV = theta_hat minus c * (Y_bar minus E[Y]), where theta_hat is the crude Monte Carlo estimate, Y_bar is the sample mean of Y, and c is a coefficient chosen to minimize variance (typically c = Cov(theta_hat, Y) / Var(Y)). The variance reduction factor is 1 minus rho^2, where rho is the correlation between the estimand and the control variate.
In actuarial applications, control variates often arise from simpler models with known solutions. For example, when simulating a complex aggregate loss model, a simpler compound Poisson model with known mean might serve as a control variate. The simulation estimates both the complex model result and the simple model result on each path. Since the simple model's true mean is known analytically, the difference between the simulated and true means of the simple model provides information about the simulation error in the complex model.
Antithetic Variates and Conditional Monte Carlo
Antithetic variates pair each uniform random number U with its complement 1 minus U, generating negatively correlated pairs of samples. If h(X) is monotone in X, the pair (h(F^{-1}(U)), h(F^{-1}(1-U))) has negative covariance, so their average has lower variance than two independent samples. The variance reduction is free: it requires no additional model evaluations, only using paired uniform inputs instead of independent ones.
Conditional Monte Carlo reduces variance by analytically computing the conditional expectation over one source of randomness. To estimate E[h(X,Y)], if E[h(X,Y)|Y] can be computed analytically as a function of Y, then the estimator E[h(X,Y)|Y_i] for simulated values Y_i has lower variance than h(X_i, Y_i) because it has removed the variability due to X. In actuarial models, this technique applies when one component of the model has a tractable conditional distribution (for example, the severity distribution conditional on a simulated frequency).
Common Pitfalls
A critical pitfall in importance sampling is choosing a poor importance distribution. If the importance distribution does not adequately cover the region of interest, the likelihood ratios can become extreme, producing a few very large weights that dominate the estimate. This results in an estimator with high or even infinite variance, worse than crude Monte Carlo. Always check the distribution of importance weights: they should be relatively stable, without extreme outliers.
For control variates, the variance reduction depends on the correlation between the estimand and the control variable. If the correlation is weak, the variance reduction is negligible and the computational overhead of evaluating the control variate is wasted. Choose control variates that are strongly correlated with the target quantity. When combining multiple variance reduction techniques, test each one individually and in combination, as some techniques may interact in unexpected ways.