Residual Analysis and Model Diagnostics for Exam SRM

Types of Residuals

The ordinary residual for observation i is e_i = Y_i - Y_hat_i. Residuals estimate the true errors epsilon_i but have different properties: they sum to zero, are correlated with each other (even when true errors are independent), and have unequal variances. Because of these properties, raw residuals are not ideal for diagnostics.

Standardized residuals divide by the estimated standard deviation: r_i = e_i / (s * sqrt(1 - h_ii)), where s = sqrt(MSE) and h_ii is the leverage. Studentized (externally studentized) residuals use a leave-one-out variance estimate: t_i = e_i / (s_(i) * sqrt(1 - h_ii)), where s_(i) is the root MSE computed without observation i. Studentized residuals follow a t-distribution with n-p-2 degrees of freedom, making them better for formal outlier testing.

A studentized residual exceeding 2 or 3 in absolute value suggests a potential outlier. On Exam SRM, know the difference between standardized and studentized residuals and when each is appropriate.

Residual Plots for Checking Assumptions

Residuals vs. fitted values is the most important diagnostic plot. Look for three things: (1) non-linearity, indicated by a systematic curve in the residual pattern, suggesting the true relationship is not linear; (2) non-constant variance (heteroscedasticity), indicated by a funnel or fan shape where residual spread increases or decreases with fitted values; (3) outliers, visible as points far from the main cluster. A good model shows random scatter centered around zero with constant spread.

Normal Q-Q plot: plot the ordered residuals against theoretical normal quantiles. If the residuals are approximately normal, the points fall close to a straight line. Deviations at the tails indicate heavy tails (points curve away from the line at both ends) or light tails (points curve toward the line). Skewness shows as a systematic departure from the line on one side.

Residuals vs. each predictor: plot residuals against each X variable to check for nonlinear relationships that the model fails to capture. A curved pattern suggests adding a polynomial term or applying a transformation. Also check for predictor-specific heteroscedasticity.

Detecting Non-Linearity and Heteroscedasticity

When residual plots suggest non-linearity, consider adding polynomial terms (X^2, X^3), using a different functional form (log, square root), or switching to a non-linear model. In actuarial applications, claim severity often requires a log transformation because the distribution is right-skewed and the variance increases with the mean.

Heteroscedasticity (non-constant variance) violates the regression assumptions and causes standard errors to be incorrect, though the coefficient estimates remain unbiased. Solutions include: transforming Y (e.g., log(Y)), using weighted least squares (WLS) where each observation is weighted inversely proportional to its variance, or using heteroscedasticity-consistent standard errors (robust standard errors).

Formal tests for heteroscedasticity include the Breusch-Pagan test (regress squared residuals on the predictors and test for significance) and White's test. On Exam SRM, you are more likely to identify heteroscedasticity from a plot than to perform a formal test, but understanding both approaches is important.

Leverage and the Hat Matrix

Leverage measures how far an observation's predictor values are from the center of the predictor space. The hat matrix H = X(X'X)^(-1)X' maps observed Y values to fitted values: Y_hat = H*Y. The leverage for observation i is h_ii, the i-th diagonal element of H. Properties of leverage: 0 <= h_ii <= 1, and the sum of all leverages equals p+1 (the number of parameters).

The average leverage is (p+1)/n. A common rule of thumb flags observations with h_ii > 2*(p+1)/n as high leverage points. High leverage means the observation has unusual predictor values and could potentially have a large influence on the regression fit. However, high leverage alone does not mean the observation actually does influence the fit; that depends also on whether its Y value is unusual.

Influential Observations and Cook's Distance

An influential observation substantially changes the regression results when removed. Influence requires both high leverage (unusual X values) and a large residual (unusual Y value given its X values). A high-leverage point whose Y value falls near the regression line has little influence; it conforms to the pattern established by other observations.

Cook's distance D_i measures the overall influence of observation i on all fitted values: D_i = (1/p) * (e_i^2 / MSE) * (h_ii / (1-h_ii)^2). Equivalently, D_i captures the squared distance between the fitted values from the full model and the fitted values from the model with observation i removed. Common thresholds are D_i > 1 or D_i > 4/n.

DFFITS measures the influence of observation i on its own fitted value: DFFITS_i = t_i * sqrt(h_ii / (1-h_ii)). The threshold is |DFFITS_i| > 2*sqrt((p+1)/n). DFBETAS measures the influence on each individual coefficient. On Exam SRM, Cook's distance is the most commonly tested influence measure.

Addressing Problematic Observations

When diagnostics reveal outliers or influential observations, several approaches are available. First, check for data errors (transcription mistakes, measurement errors, or data entry problems). If the observation is genuinely unusual, consider whether it represents a different population that should be modeled separately. Robust regression methods (such as least absolute deviations or M-estimation) are less sensitive to outliers than ordinary least squares.

Removing observations should be done cautiously and reported transparently. Fit the model with and without the observation and compare results. If removing one or two observations dramatically changes the conclusions, the results are not robust and the model should be reconsidered.

Study Tips and Common Pitfalls

The most common mistake on Exam SRM is confusing leverage with influence. High leverage means unusual X values; influence requires both high leverage and a large residual. A point can have high leverage but low influence, or moderate leverage with a large residual that creates moderate influence.

Practice interpreting residual plots: be able to identify non-linearity, heteroscedasticity, non-normality, and outliers from visual patterns. Know the formulas for Cook's distance and leverage, and be able to compute them from given quantities. Work through at least 5 problems involving diagnostic interpretation and influential observation analysis.