Maximum Likelihood Estimation for Actuaries
Master maximum likelihood estimation principles and methods for Exam SRM.
The Likelihood Function
Maximum likelihood estimation (MLE) is the most widely used method for estimating parameters of statistical models. The central idea is simple: choose the parameter values that make the observed data most probable.
Given observed data x_1, x_2, ..., x_n from a distribution with density (or mass function) f(x | theta), the likelihood function is L(theta) = product_{i=1}^{n} f(x_i | theta). The likelihood function is a function of the parameter theta, with the data held fixed. It measures how "likely" each possible parameter value is, given the data we observed.
The maximum likelihood estimator (MLE) is the value theta_hat that maximizes L(theta). Because products are difficult to work with, we almost always work with the log-likelihood instead: l(theta) = ln(L(theta)) = sum_{i=1}^{n} ln(f(x_i | theta)). Since the logarithm is a monotonically increasing function, the value that maximizes l(theta) also maximizes L(theta).
To find the MLE analytically: take the derivative of l(theta) with respect to theta (this derivative is called the score function), set it equal to zero, and solve for theta. Verify that the solution is a maximum (not a minimum or saddle point) by checking that the second derivative is negative at that point.
Worked Examples
For the Poisson distribution with parameter lambda: f(x | lambda) = e^(-lambda) * lambda^x / x!. The log-likelihood is l(lambda) = sum[-lambda + x_i * ln(lambda) - ln(x_i!)] = -n*lambda + (sum x_i) * ln(lambda) - sum ln(x_i!). Setting dl/d_lambda = -n + (sum x_i) / lambda = 0 gives lambda_hat = (sum x_i) / n = x_bar, the sample mean.
For the exponential distribution with rate parameter theta: f(x | theta) = theta * e^(-theta*x). The log-likelihood is l(theta) = n * ln(theta) - theta * sum(x_i). Setting dl/d_theta = n/theta - sum(x_i) = 0 gives theta_hat = n / sum(x_i) = 1 / x_bar.
For the normal distribution with unknown mean mu and known variance sigma^2: l(mu) = -(n/2) * ln(2*pi*sigma^2) - sum(x_i - mu)^2 / (2*sigma^2). Setting dl/d_mu = sum(x_i - mu) / sigma^2 = 0 gives mu_hat = x_bar. If both mu and sigma^2 are unknown, the MLEs are mu_hat = x_bar and sigma^2_hat = sum(x_i - x_bar)^2 / n. Note that the MLE of the variance uses n in the denominator (not n-1), making it biased but asymptotically correct.
For the binomial distribution with n trials and success probability p: l(p) = x * ln(p) + (n - x) * ln(1 - p) + ln(C(n,x)). Setting dl/dp = x/p - (n-x)/(1-p) = 0 gives p_hat = x/n, the sample proportion.
Properties of MLEs
Under regularity conditions (the parameter space is open, the log-likelihood is twice differentiable, etc.), MLEs possess several desirable large-sample properties that make them the default estimation method in statistics.
Consistency: as the sample size n grows, the MLE converges in probability to the true parameter value. That is, theta_hat approaches theta_true as n approaches infinity.
Asymptotic normality: for large n, the MLE is approximately normally distributed: theta_hat ~ Normal(theta, 1/(n * I(theta))), where I(theta) is the Fisher information for a single observation. This result enables the construction of confidence intervals and hypothesis tests.
Asymptotic efficiency: the MLE achieves the Cramer-Rao lower bound asymptotically, meaning no other consistent estimator has smaller asymptotic variance. In other words, the MLE extracts the maximum possible information from the data.
Invariance property: if theta_hat is the MLE of theta, then g(theta_hat) is the MLE of g(theta) for any function g. For example, if lambda_hat is the MLE of a Poisson rate, then exp(-lambda_hat) is the MLE of the zero-claim probability P(X = 0) = e^(-lambda).
Fisher Information and Standard Errors
The Fisher information quantifies the amount of information that the data carries about the parameter. For a single observation, the Fisher information is I(theta) = -E[d^2 ln(f(X | theta)) / d_theta^2] = Var(d ln(f(X | theta)) / d_theta). For n independent observations, the total Fisher information is n * I(theta).
The approximate standard error of the MLE is SE(theta_hat) = 1 / sqrt(n * I(theta_hat)), where we evaluate the Fisher information at the MLE. This gives the approximate confidence interval: theta_hat +/- z_{alpha/2} * SE(theta_hat).
For the Poisson example with lambda_hat = x_bar: I(lambda) = 1/lambda, so SE(lambda_hat) = sqrt(lambda_hat / n) = sqrt(x_bar / n). A 95% confidence interval is x_bar +/- 1.96 * sqrt(x_bar / n).
In practice, the observed Fisher information (the negative of the actual second derivative of the log-likelihood at theta_hat, rather than its expected value) is often used instead. For many common distributions, the observed and expected Fisher information are equivalent.
Numerical Optimization
Many models do not have closed-form MLEs. GLMs, logistic regression, and complex loss distribution models require numerical optimization. The Newton-Raphson method iteratively updates the estimate: theta_{k+1} = theta_k - [l''(theta_k)]^(-1) * l'(theta_k), using the score function and the Hessian. For GLMs, the iteratively reweighted least squares (IRLS) algorithm is a specialized version of Newton-Raphson that is particularly efficient.
Convergence issues can arise with poor starting values, flat likelihoods, or boundary parameters. Always check that the algorithm converged and that the solution is a global maximum (not a local one).
Censored and Truncated Data
In insurance, data is often incomplete. Right censoring occurs when the observation period ends before the event of interest (e.g., a policy lapses before a claim occurs). Truncation occurs when only observations meeting certain criteria are included (e.g., only claims exceeding a deductible are recorded).
For right-censored data, the likelihood contribution of a censored observation at time c is the survival function S(c | theta) = 1 - F(c | theta), rather than the density f(c | theta). The full likelihood is L(theta) = product_{uncensored} f(x_i | theta) * product_{censored} S(c_j | theta). For left-truncated data at threshold d, the likelihood contribution is f(x | theta) / S(d | theta), accounting for the fact that only observations exceeding d are observable.
Study Tips for Exam SRM
For Exam SRM, practice deriving MLEs for the four common distributions (normal, Poisson, exponential, binomial) until you can do them quickly. Know how to compute the Fisher information and construct approximate confidence intervals using the asymptotic normal result. Understand the invariance property and how to apply it. Be familiar with the likelihood ratio test as a tool for comparing nested models. Also understand that for GLMs and logistic regression, MLE is the estimation method, even though it requires numerical optimization rather than a closed-form solution.