Markov Chain Models for Multi-State Insurance

Continuous-Time Markov Chains

Multi-state models generalize traditional life contingencies by allowing multiple living states plus one or more absorbing states (typically death). Instead of the simple alive/dead framework of basic life contingencies, a multi-state model can represent states such as healthy, disabled, critically ill, and dead. The model is defined by transition intensities mu_x^{ij}, which give the instantaneous rate of moving from state i to state j at age x. These transition intensities play the same role as the force of mortality in traditional models, but now there is one for each possible transition between states.

The transition probability t_p_x^{ij} is the probability of being in state j at age x+t, given the individual is in state i at age x. For a Markov model, these transition probabilities depend only on the current state and age, not on the history of states visited. This memoryless property greatly simplifies calculations and is the defining characteristic of a Markov chain.

Kolmogorov Equations

The transition probabilities satisfy two systems of differential equations. Kolmogorov's forward equations describe how transition probabilities evolve as the ending time moves forward: (d/dt) t_p_x^{ij} = sum over k of t_p_x^{ik} * mu_{x+t}^{kj} minus t_p_x^{ij} * sum over k (k not equal to j) of mu_{x+t}^{jk}. The first term represents inflow into state j from other states, and the second represents outflow from state j to other states.

Kolmogorov's backward equations describe how transition probabilities change as the starting time moves backward: (d/ds) (t-s)_p_{x+s}^{ij} evaluated differently, or equivalently, the backward equations express t_p_x^{ij} in terms of what happens at the start of the interval. For a two-state model (alive and dead), the forward equation reduces to the familiar relationship (d/dt) t_p_x = minus t_p_x * mu_{x+t}, recovering classical survival analysis.

Insurance and Annuity Valuation

Multi-state annuity values sum discounted sojourn probabilities. The annuity payable continuously while in state i is a-bar_x^{ii} = integral from 0 to infinity of v^t * t_p_x^{ii} dt. This is the present value of a stream of payments at rate 1 per year, conditional on remaining in state i. For disability insurance, the annuity while healthy (state 0) is a-bar_x^{00}, and the annuity while disabled (state 1) is a-bar_x^{11}.

Insurance values for transitions between states represent benefits paid upon changing state. The expected present value of a benefit of 1 paid upon each transition from any state to state j is A-bar_x^{ij} = integral from 0 to infinity of v^t * (sum over k not equal to j of t_p_x^{ik} * mu_{x+t}^{kj}) dt. The integrand captures the probability of being in some state k at time t (t_p_x^{ik}) multiplied by the instantaneous transition rate from k to j (mu_{x+t}^{kj}), summed over all possible source states.

Thiele's Equations for Multi-State Models

Thiele's differential equation extends to multi-state models with one equation for each state. If the individual is currently in state i, the reserve satisfies: (d/dt) t_V^{(i)} = delta * t_V^{(i)} + P^{(i)}_t minus b^{(i)}_t minus sum over j (j not equal to i) of mu_{x+t}^{ij} * (S^{ij}_t + t_V^{(j)} minus t_V^{(i)}). Here delta is the force of interest, P^{(i)}_t is the premium rate while in state i, b^{(i)}_t is the benefit rate while in state i, S^{ij}_t is the lump sum benefit on transition from i to j, and t_V^{(j)} is the reserve in the destination state.

The last term captures the expected cost of transitions: each possible transition from state i to state j costs a lump sum S^{ij} plus the change in reserve (t_V^{(j)} minus t_V^{(i)}). This system of coupled differential equations must be solved simultaneously, typically using numerical methods such as Euler's method or Runge-Kutta methods, working backward from the policy's termination.

The Three-State Disability Model

The most commonly tested multi-state model on Exam LTAM is the three-state disability model with states: 0 (active/healthy), 1 (disabled), and 2 (dead). The transition intensities are sigma_x (0 to 1, becoming disabled), rho_x (1 to 0, recovery), mu_x^{02} (healthy to dead), and mu_x^{12} (disabled to dead). There is no direct transition from dead to any other state, making state 2 absorbing.

For this model, the premium equation sets P * a-bar_x^{00} = b * a-bar_x^{01} + c * A-bar_x^{02} + d * A-bar_x^{12}, where the right side includes the cost of disability benefits (annuity while disabled), death benefit while healthy, and death benefit while disabled. The Thiele equations for this model consist of two coupled equations (one for the active reserve and one for the disabled reserve), since no reserve is needed in the dead state.

Common Pitfalls

Students frequently confuse the sojourn probability t_p_x^{ii} (remaining continuously in state i) with the occupancy probability t_p_x^{ii*} (being in state i at time t, possibly having left and returned). In a Markov model with recovery (like the disability model), these are different because an individual can leave state 0, enter state 1, recover back to state 0, and be counted in the occupancy probability but not the sojourn probability. Exam LTAM typically uses occupancy probabilities for annuity calculations unless otherwise specified.

Another pitfall is setting up Thiele's equations with the wrong sign convention. The derivative (d/dt) t_V^{(i)} represents the rate of change of the reserve as duration increases. Premiums add to the reserve, benefits reduce it, interest increases it, and expected transition costs reduce it. Getting any sign wrong throws off the entire numerical solution.

Study Tips

Exam LTAM tests multi-state models extensively, particularly the three-state disability model. Practice setting up Kolmogorov equations for small models (two and three states), computing transition probabilities numerically using Euler's method, and deriving premium and reserve formulas from the general multi-state framework. Be comfortable with Thiele's equations for two coupled states and practice the numerical backward recursion. Draw the state diagram for each problem, labeling all transition intensities, before starting any calculation. This visual aid prevents errors in setting up the equations.