Bayesian Statistics Fundamentals for Exam SRM

The Bayesian Framework

Bayesian statistics takes a fundamentally different approach to inference compared to the frequentist methods (MLE, hypothesis testing) covered in other posts. In the Bayesian framework, parameters are treated as random variables with probability distributions, rather than as fixed but unknown constants. This allows us to make direct probability statements about parameters: "there is a 95% probability that the claim rate is between 0.03 and 0.07," rather than the indirect frequentist interpretation of confidence intervals.

The framework has three components. The prior distribution p(theta) encodes what we believe about the parameter before observing data. This can come from expert judgment, historical data, or a desire for mathematical convenience. The likelihood function L(data | theta) = p(data | theta) gives the probability of observing the data for each possible parameter value. Bayes' theorem combines these to produce the posterior distribution: p(theta | data) = p(data | theta) * p(theta) / p(data), where p(data) = integral of p(data | theta) * p(theta) d_theta is the normalizing constant.

The key relationship is: posterior is proportional to likelihood times prior. The normalizing constant p(data) ensures the posterior integrates to 1, but for identifying the distributional form of the posterior, it can be treated as a constant. You often write p(theta | data) proportional to p(data | theta) * p(theta) and recognize the distributional form from the kernel.

Prior Distributions

Choosing a prior is both the strength and the controversy of Bayesian statistics. Informative priors incorporate genuine knowledge about the parameter. For example, an actuary might use historical data from similar portfolios to form a prior for the claim frequency of a new line of business. This is especially valuable when the current dataset is small.

Non-informative (vague) priors try to let the data dominate by assigning roughly equal probability to all parameter values. Common choices include uniform priors, Jeffreys priors (which are invariant to reparameterization), and improper priors (which do not integrate to a finite value but can still produce valid posteriors). With a non-informative prior and large samples, the posterior is approximately the same as the MLE-based normal approximation, so Bayesian and frequentist results converge.

The choice of prior matters most when the sample size is small. As the sample size grows, the likelihood dominates and the posterior becomes less sensitive to the prior. This "washing out" of the prior is a reassuring property: with enough data, reasonable people with different priors will reach similar conclusions.

Conjugate Priors

A prior is called conjugate to a likelihood if the posterior distribution belongs to the same family as the prior. Conjugate priors make computation tractable because the posterior parameters can be derived analytically, without numerical integration.

The essential conjugate pairs for Exam SRM are as follows. Beta prior with Binomial likelihood: if theta ~ Beta(a, b) and X | theta ~ Binomial(n, theta), then theta | X ~ Beta(a + x, b + n - x). The posterior mean is (a + x) / (a + b + n).

Gamma prior with Poisson likelihood: if lambda ~ Gamma(alpha, beta) and X_1, ..., X_n | lambda ~ Poisson(lambda), then lambda | data ~ Gamma(alpha + sum(x_i), beta + n). The posterior mean is (alpha + sum(x_i)) / (beta + n).

Normal prior with Normal likelihood: if mu ~ Normal(mu_0, sigma_0^2) and X_1, ..., X_n | mu ~ Normal(mu, sigma^2) with sigma^2 known, then mu | data ~ Normal(mu_n, sigma_n^2), where mu_n = (mu_0/sigma_0^2 + n*x_bar/sigma^2) / (1/sigma_0^2 + n/sigma^2) and sigma_n^2 = 1 / (1/sigma_0^2 + n/sigma^2).

Worked Example: Beta-Binomial

An actuary believes the claim probability for a new product is around 0.05, with moderate uncertainty. She models her prior as theta ~ Beta(5, 95), which has mean 5/100 = 0.05 and standard deviation sqrt(5*95/100^2/101) = 0.022.

In the first year, 200 policies are sold and 14 claims are observed. The posterior is theta | data ~ Beta(5 + 14, 95 + 200 - 14) = Beta(19, 281). The posterior mean is 19/300 = 0.0633, which lies between the prior mean (0.05) and the sample proportion (14/200 = 0.07). The posterior gives more weight to the data because n = 200 is much larger than the "effective sample size" of the prior (a + b = 100).

The posterior standard deviation is sqrt(19*281/300^2/301) = 0.014, which is smaller than both the prior standard deviation (0.022) and the standard error of the sample proportion (sqrt(0.07*0.93/200) = 0.018). The posterior combines information from both sources, producing a more precise estimate than either alone.

Point Estimation and Credible Intervals

Bayesian inference produces the entire posterior distribution, from which various summaries can be extracted. The posterior mean minimizes expected squared error loss: E[theta | data]. The posterior median minimizes expected absolute error loss. The posterior mode (also called the MAP estimate, for Maximum A Posteriori) is the single most probable value. For symmetric posteriors (like the normal), all three coincide.

A 95% credible interval [L, U] is an interval such that P(L < theta < U | data) = 0.95. The highest posterior density (HPD) interval is the shortest such interval. For the Beta(19, 281) posterior above, a 95% credible interval might be approximately (0.039, 0.092).

The interpretation of a credible interval is direct: "given our prior and the observed data, there is a 95% probability that theta lies in this interval." This is fundamentally different from a frequentist confidence interval, which says "if we repeated the experiment many times, 95% of the resulting intervals would contain the true theta." The Bayesian interpretation is more intuitive and is often what practitioners actually want to know.

Bayesian vs. Frequentist Inference

The two approaches differ philosophically and practically. Frequentist inference treats parameters as fixed and makes probability statements about data (p-values, confidence intervals). Bayesian inference treats parameters as random and makes probability statements about parameters (posterior distributions, credible intervals). Frequentist methods do not require a prior; Bayesian methods do.

In actuarial science, Bayesian methods are the foundation of credibility theory, which blends individual risk experience with class-level data, exactly as the posterior blends the likelihood with the prior. The Buhlmann credibility formula Z * x_bar + (1 - Z) * mu_0 is the posterior mean under a normal-normal conjugate model.

For small samples, the prior can meaningfully improve estimates by incorporating external information. For large samples, the prior becomes irrelevant and both approaches give similar results. The Bayesian framework is especially valuable in insurance when data for individual risks is limited but historical or industry-wide information is available.

Study Tips for Exam SRM

For Exam SRM, focus on these core skills. First, be able to compute the posterior distribution for conjugate models (Beta-Binomial, Gamma-Poisson, Normal-Normal). Second, calculate and interpret the posterior mean as a weighted average of the prior mean and the data. Third, understand the difference between credible intervals and confidence intervals. Fourth, know that with large samples or non-informative priors, Bayesian and frequentist results converge. Practice problems that give a prior and data and ask for the posterior parameters, the posterior mean, and a credible interval. Also be prepared for conceptual questions comparing the Bayesian and frequentist approaches to a given inference problem.