Bayesian Analysis: Prior, Posterior, and Predictive Distributions
Apply Bayesian statistical methods with prior, posterior, and predictive distributions for Exam MAS-I.
Bayesian Framework
Bayesian analysis provides a principled framework for updating beliefs about unknown parameters as data is observed. The approach begins with a prior distribution pi(theta) that encodes what is known about the parameter before seeing data. After observing data x, the prior is updated to a posterior distribution using Bayes' theorem: pi(theta|x) = L(theta|x) * pi(theta) / f(x), where L(theta|x) is the likelihood function and f(x) = integral of L(theta|x) * pi(theta) dtheta is the marginal likelihood (normalizing constant).
In practice, the normalizing constant f(x) is often difficult to compute, so we use the proportional form: pi(theta|x) is proportional to L(theta|x) * pi(theta). The posterior distribution is then identified by recognizing its functional form in theta. The posterior summarizes all available information about theta, combining prior knowledge with evidence from the data.
Conjugate Priors
A conjugate prior is a prior distribution that, when combined with a particular likelihood, produces a posterior distribution in the same family as the prior. Conjugacy simplifies Bayesian analysis enormously because the posterior parameters can be computed by simple formulas rather than numerical integration. The most important conjugate pairs for Exam MAS-I are listed below.
Beta-Binomial: If X|p ~ Binomial(n, p) and p ~ Beta(alpha, beta), then p|x ~ Beta(alpha + x, beta + n - x). The posterior mean is (alpha + x) / (alpha + beta + n). Gamma-Poisson: If X|lambda ~ Poisson(lambda) and lambda ~ Gamma(alpha, beta), then lambda|x ~ Gamma(alpha + sum(x_i), beta + n). Normal-Normal: If X|mu ~ N(mu, sigma^2) (sigma^2 known) and mu ~ N(mu_0, sigma_0^2), then mu|x ~ N(posterior_mean, posterior_variance), where the posterior mean is a precision-weighted average of the prior mean and the sample mean.
Posterior Mean as Credibility Estimate
A beautiful connection between Bayesian and credibility theory emerges from conjugate analysis. The posterior mean is always a weighted average of the prior mean and the data mean, with weights determined by the relative precision of prior information versus data. For the Normal-Normal model, the posterior mean is Z * x-bar + (1-Z) * mu_0, where Z = n*tau^2 / (n*tau^2 + sigma^2), with tau^2 = sigma_0^2 being the prior variance and sigma^2 the observation variance.
This is exactly the Buhlmann credibility formula. The credibility weight Z increases with the sample size n and with the prior variance tau^2 (more heterogeneity means individual data is more informative). It decreases with the observation variance sigma^2 (noisier data is less informative). This connection shows that credibility theory, widely used in actuarial pricing, is a special case of Bayesian estimation.
Prior and Posterior Predictive Distributions
The prior predictive distribution gives the marginal distribution of data before any observations are made: f(x) = integral of f(x|theta) * pi(theta) dtheta. This integrates out the unknown parameter, producing a distribution that accounts for parameter uncertainty. For example, if X|lambda ~ Poisson(lambda) and lambda ~ Gamma(alpha, beta), the prior predictive is Negative Binomial with parameters (alpha, beta/(1+beta)).
The posterior predictive distribution predicts future observations after data has been observed: f(x_new|x) = integral of f(x_new|theta) * pi(theta|x) dtheta. This uses the posterior (updated) distribution of theta rather than the prior. For the Poisson-Gamma model, the posterior predictive is also Negative Binomial but with updated parameters reflecting the observed data. The posterior predictive incorporates both the randomness of future observations and the remaining uncertainty about the parameter.
Bayesian Credible Intervals
A (1-alpha) Bayesian credible interval is a region C such that P(theta in C | x) = 1 - alpha. Unlike frequentist confidence intervals, credible intervals have a direct probability interpretation: there is a (1-alpha) probability (given the data and prior) that theta lies in the interval. The highest posterior density (HPD) interval is the shortest credible interval, formed by including all theta values where the posterior density exceeds a certain threshold.
For symmetric posterior distributions (like the Normal), the HPD interval is symmetric around the posterior mean. For skewed posteriors (like the Gamma), the HPD interval is asymmetric, placing more probability mass near the mode. In practice, equal-tail credible intervals (cutting off alpha/2 from each tail) are simpler to compute and are used when HPD computation is impractical.
Loss Functions and Bayesian Point Estimates
Different loss functions lead to different Bayesian point estimates. Squared error loss L(theta, d) = (theta - d)^2 yields the posterior mean as the optimal estimate. Absolute error loss L(theta, d) = |theta - d| yields the posterior median. Zero-one loss (for discrete parameters) yields the posterior mode. The choice of loss function should reflect the practical consequences of estimation errors in the specific application.
In actuarial applications, squared error loss is most common, making the posterior mean the standard Bayesian estimate. However, when the consequence of underestimation differs from overestimation (for example, in reserving where underestimation creates solvency risk), asymmetric loss functions may be more appropriate.
Common Pitfalls
A frequent mistake is forgetting to update the prior parameters correctly. For the Beta-Binomial conjugate pair, the posterior parameters are alpha + x (successes) and beta + n - x (failures), not alpha + n and beta + x. Another common error is confusing the prior predictive with the posterior predictive. The prior predictive uses the prior distribution of theta; the posterior predictive uses the posterior distribution.
Students also sometimes compute the posterior distribution without recognizing it as a known family. After multiplying the likelihood by the prior, carefully examine the functional form in theta to identify the posterior family and its parameters.
Study Tips
Exam MAS-I tests conjugate analysis, posterior computation, predictive distribution derivation, and the connection between Bayesian estimation and credibility theory. Memorize the key conjugate pairs and their update formulas. Practice deriving posterior distributions from scratch by multiplying the likelihood and prior, then identifying the posterior family. Work through problems that require computing posterior means, credible intervals, and posterior predictive distributions. Understand the conceptual relationship between Bayesian updating and credibility weighting, as this connection is tested frequently.