Survival Analysis: Kaplan-Meier and Cox Regression

What Is Survival Analysis?

Survival analysis studies the time until an event of interest occurs. In actuarial science, the "event" could be death, the filing of a claim, policy lapse, disability onset, or any other transition. The defining feature that sets survival analysis apart from ordinary regression is the presence of censoring: for some subjects, the event has not yet occurred by the end of the study, so the exact event time is unknown. Special statistical methods are needed to extract valid information from censored data.

The survival function S(t) = P(T > t) gives the probability that the event time T exceeds t. It starts at S(0) = 1 (everyone is alive/active at time 0) and decreases toward 0. The hazard function h(t) = f(t) / S(t) gives the instantaneous rate of the event at time t, conditional on survival up to t. The cumulative hazard H(t) = integral from 0 to t of h(u) du is related to the survival function by S(t) = exp(-H(t)).

Types of Censoring

Right censoring is the most common type in actuarial data. It occurs when a subject's observation ends before the event happens. Examples include: a policyholder is still active at the study cutoff date, a participant drops out of a clinical trial, or a policy is cancelled for non-payment before a claim occurs. For each subject, we observe either the event time T (if the event occurs) or the censoring time C (if censored), whichever comes first: Y = min(T, C). We also observe the indicator delta = 1 if T <= C (event observed) and delta = 0 if T > C (censored).

Left censoring occurs when the event has already happened before the observation begins (the event time is known only to be less than some value). Interval censoring occurs when the event is known to have occurred within a time interval but the exact time is unknown (e.g., a disease detected at an annual checkup could have started anytime since the last checkup).

The key assumption for valid survival analysis is that censoring is non-informative: the probability of being censored does not depend on the (unobserved) event time. If sicker patients are more likely to drop out of a study, the censoring is informative and standard methods produce biased results.

Kaplan-Meier Estimator

The Kaplan-Meier (KM) estimator, also called the product-limit estimator, provides a nonparametric estimate of the survival function. At each observed event time t_i, compute the conditional probability of surviving past t_i given survival up to that point: 1 - d_i/n_i, where d_i is the number of events at t_i and n_i is the number at risk (alive and uncensored) just before t_i.

The KM estimator is the product of these conditional survival probabilities: S_hat(t) = product over all t_i <= t of (1 - d_i / n_i). The result is a step function that drops at each observed event time. Censored observations reduce the risk set n_i at subsequent times but do not cause a step in the survival curve.

A worked example: 10 subjects enter a study. Events occur at times 3, 5, 7, 7, 12. Censorings occur at times 4, 6, 8, 10, 15. At time 3: n = 10, d = 1, S_hat(3) = 1 * (1 - 1/10) = 0.900. Before time 5: one censoring at time 4 reduces the risk set. At time 5: n = 8, d = 1, S_hat(5) = 0.900 * (1 - 1/8) = 0.788. Before time 7: one censoring at time 6. At time 7: n = 6, d = 2, S_hat(7) = 0.788 * (1 - 2/6) = 0.525. And so on.

The Greenwood formula provides the variance of the KM estimator: Var(S_hat(t)) = S_hat(t)^2 * sum over t_i <= t of d_i / (n_i * (n_i - d_i)). This enables the construction of confidence intervals for the survival function at any time point. A 95% confidence interval is S_hat(t) +/- 1.96 * sqrt(Var(S_hat(t))), though log-transformed intervals are preferred to avoid values outside [0, 1].

The Log-Rank Test

The log-rank test compares survival curves between two or more groups (e.g., treatment vs. control, or male vs. female policyholders). The null hypothesis is that the survival functions are the same across groups: H0: S_1(t) = S_2(t) for all t.

At each event time, the test compares the observed number of events in each group to the expected number under H0. If there are d events among n subjects at risk (n_1 in group 1, n_2 in group 2), the expected events in group 1 under H0 are e_1 = d * n_1 / n. The test statistic is chi-squared = (sum(O_1 - E_1))^2 / sum(V_1), where the sum runs over all event times and V_1 is the variance of the expected count. Under H0, this follows a chi-squared distribution with 1 degree of freedom (for two groups).

The log-rank test is most powerful when the proportional hazards assumption holds (the hazard ratio between groups is constant over time). It has less power to detect differences when survival curves cross.

Cox Proportional Hazards Model

The Cox model is the most widely used regression model for survival data. It relates the hazard function to predictors through: h(t | X) = h_0(t) * exp(beta_1 * X_1 + beta_2 * X_2 + ... + beta_p * X_p). Here h_0(t) is the baseline hazard function (the hazard when all predictors equal zero), and exp(beta' X) is the relative risk multiplier.

The model is semi-parametric: the regression coefficients beta are estimated without specifying the baseline hazard h_0(t). This is achieved through the partial likelihood, which depends only on the order of events and the predictor values, not on the times between events. The partial likelihood at each event time t_i is exp(beta' X_i) / sum_{j in risk set at t_i} exp(beta' X_j), and the total partial likelihood is the product over all event times.

The interpretation of coefficients is through hazard ratios. exp(beta_j) is the hazard ratio for a one-unit increase in X_j, holding all other predictors constant. A hazard ratio of 1.5 means the hazard (instantaneous event rate) is 50% higher. A hazard ratio of 0.7 means the hazard is 30% lower. Hazard ratios are the survival analysis equivalent of odds ratios in logistic regression.

The proportional hazards assumption states that the hazard ratio between any two individuals is constant over time. This means the predictor effects do not change with time. If a drug reduces the hazard by 30%, this reduction applies equally at 1 month, 6 months, and 5 years. This assumption can be checked by plotting scaled Schoenfeld residuals against time; a nonzero slope suggests non-proportionality.

Actuarial Applications

Survival analysis is central to life insurance and pension actuarial work. Mortality studies use Kaplan-Meier curves and Cox models to estimate life expectancy and identify risk factors for mortality. Policy persistency analysis models the time until policy lapse, helping insurers understand retention and plan for reserves.

In health insurance, survival analysis models the duration of disability claims or the time until recovery. In property and casualty insurance, it can model the time from accident to claim report (reporting delay) or from report to settlement (settlement delay). These time-to-event analyses inform reserve adequacy and claims management decisions.

Study Tips for Exam SRM

For Exam SRM, focus on these essential skills. First, understand censoring and why ignoring it produces biased results. Second, be able to compute the Kaplan-Meier estimator step by step for a small dataset, correctly handling censored observations in the risk set. Third, know the Greenwood formula for variance. Fourth, understand the log-rank test for comparing groups. Fifth, interpret Cox model output: hazard ratios, their confidence intervals, and the proportional hazards assumption. Practice problems that present a survival dataset and ask you to compute the KM estimate at specific times, or that give Cox regression output and ask you to interpret the hazard ratios in actuarial context.