Goodness-of-Fit Tests for Loss Distributions

Why Goodness-of-Fit Matters

After fitting a parametric distribution to loss data, you must verify the model fits adequately. Goodness-of-fit tests compare observed data to the fitted model\'s predictions. Common tests include chi-squared, Kolmogorov-Smirnov (KS), and Anderson-Darling (AD). The choice of test depends on data characteristics, sample size, and what aspects of fit matter most. A poor fit may indicate the wrong distributional family, inadequate parameter estimation, or structural issues in the data.

In actuarial practice, the consequences of a poor distributional fit can be severe. If the fitted distribution understates the probability of large losses, reserves will be inadequate and the insurer may face solvency concerns. If it overstates loss probabilities, premiums will be uncompetitively high. Goodness-of-fit testing provides an objective check on model quality before the model is used for pricing or reserving decisions.

The Chi-Squared Test

The chi-squared test groups data into c intervals and compares observed counts O_j to expected counts E_j under the fitted model. The test statistic is Q = sum from j=1 to c of (O_j - E_j)^2 / E_j. Under the null hypothesis that the data comes from the fitted distribution, Q follows approximately a chi-squared distribution with c - 1 - p degrees of freedom, where p is the number of parameters estimated from the data.

For the chi-squared approximation to be valid, each expected count E_j should be at least 5. If some intervals have expected counts below 5, they should be combined with adjacent intervals. This requirement means that sample sizes need to be reasonably large for the chi-squared test to be applicable.

For example, suppose you fit an exponential distribution to 200 loss observations and group them into 5 intervals. You estimate one parameter (theta), so degrees of freedom = 5 - 1 - 1 = 3. If the computed test statistic is Q = 4.2, you compare this to the chi-squared critical value with 3 degrees of freedom at your chosen significance level (for example, 7.815 at the 5% level). Since 4.2 < 7.815, you would not reject the null hypothesis, suggesting the exponential model fits adequately.

The Kolmogorov-Smirnov Test

The KS test uses the maximum absolute difference between the empirical CDF F_n(x) and the fitted CDF F(x). The test statistic is D = max over all x of |F_n(x) - F(x)|. In practice, D is computed at each data point: D = max over i of max(|F(x_i) - (i-1)/n|, |F(x_i) - i/n|), where x_1 <= x_2 <= ... <= x_n are the ordered observations.

The KS test works with ungrouped (individual) data and does not require any arbitrary grouping decisions. However, it is most sensitive to differences near the median of the distribution, where the empirical CDF is changing most rapidly. It is less sensitive to differences in the tails. Critical values depend on the sample size n and must be adjusted when parameters are estimated from the data (the standard KS table assumes the distribution is fully specified, not fitted).

The Anderson-Darling Test

The Anderson-Darling test addresses the KS test\'s weakness in the tails by applying a weight function that emphasizes tail deviations. The test statistic is A^2 = -n - (1/n) * sum from i=1 to n of (2i-1) * [ln(F(x_i)) + ln(1 - F(x_{n+1-i}))]. This weighting function 1/(F(x)(1-F(x))) gives more importance to discrepancies where F(x) is near 0 or 1, which correspond to the tails of the distribution.

For insurance applications, the Anderson-Darling test is often preferred because accurate tail modeling is critical. A model that fits the body of the loss distribution well but underestimates the right tail could lead to significant underreserving. The AD test is more likely to detect such problems than either the chi-squared or KS tests.

Comparing the Three Tests

Each test has distinct strengths. The chi-squared test is versatile and works with both grouped and individual data (after grouping), but results depend on the choice of intervals. The KS test requires no grouping and is straightforward to compute, but is insensitive to tail discrepancies. The AD test provides the best tail sensitivity but requires individual (ungrouped) data and has more complex critical values.

In terms of data requirements, the chi-squared test can handle grouped data where individual observations are not available (common in insurance data sets reported by interval). The KS and AD tests require individual observations. If parameters are estimated from the data, critical values for all three tests must be adjusted, though this adjustment is most commonly discussed for the chi-squared test (subtracting p from degrees of freedom).

Common Pitfalls

Students frequently forget to adjust degrees of freedom for estimated parameters in the chi-squared test. If you estimate two parameters from the data and have 6 intervals, degrees of freedom = 6 - 1 - 2 = 3, not 5. Another common error is applying the KS test and expecting it to catch tail misfit. If tail accuracy matters, use Anderson-Darling instead.

Be careful about the distinction between "the data fits the model" and "the model is correct." A goodness-of-fit test that fails to reject the null hypothesis does not prove the model is right. It only means the data does not provide strong evidence against the model. Multiple models may pass the same goodness-of-fit test, and other criteria (like parsimony, interpretability, and out-of-sample performance) should inform the final choice.

Study Tips

For Exam STAM, know the test statistics, critical values, and when each test is most appropriate. Practice computing all three test statistics from a small data set. Memorize the chi-squared degrees of freedom formula (c - 1 - p), know how to set up the KS computation by stepping through ordered observations, and understand the conceptual basis for Anderson-Darling\'s tail weighting. Be prepared to explain which test is most appropriate for a given situation and to interpret test results in the context of model selection.