Predictive Modeling Ethics and Fairness in Insurance
Ethical considerations in actuarial predictive modeling, including bias detection and fairness metrics for MAS-II.
Why Fairness Matters
Predictive models can perpetuate or amplify existing biases in insurance pricing. Even when protected characteristics like race or gender are excluded, proxy variables such as zip code or credit score may produce disparate impacts. Regulators increasingly require insurers to demonstrate that models do not unfairly discriminate, making this a growing exam topic.
The issue arises because machine learning algorithms optimize for predictive accuracy, not for social fairness. An algorithm trained on historical data will learn any patterns present in that data, including patterns that reflect past discrimination. For example, if a geographic area has historically been underserved by insurance, leading to fewer claims being reported, a model might learn that the area is low risk when in fact the low claim volume reflects barriers to filing rather than genuinely lower risk. Understanding these dynamics is increasingly important for the MAS-II exam.
Key Fairness Concepts
Before diving into specific metrics, it helps to understand the underlying concepts. Direct discrimination occurs when a protected characteristic (such as race or gender) is used explicitly in the model. Indirect discrimination occurs when a facially neutral variable serves as a proxy for a protected characteristic. Disparate impact refers to a policy or model that affects protected groups differently, even if the intent is not discriminatory. The legal and regulatory frameworks around these concepts vary by jurisdiction, but the actuarial principles are consistent across the exam syllabus.
Fairness Metrics and Mitigation
Key fairness metrics include demographic parity, equalized odds, and calibration across groups. These metrics often conflict with each other, so actuaries must make principled trade-offs. Mitigation strategies include pre-processing (reweighting training data), in-processing (adding fairness constraints to the objective function), and post-processing (adjusting model outputs). For MAS-II, understand how to measure disparate impact ratios, explain the tension between actuarial accuracy and social fairness, and describe regulatory frameworks that govern the use of predictive models in insurance pricing.
Fairness Metrics in Detail
Demographic parity requires that the model's positive prediction rate is equal across groups. In insurance pricing, this would mean that the average predicted loss cost is the same for all demographic groups. This metric is easy to compute but often conflicts with actuarial accuracy, because if different groups genuinely have different risk profiles, forcing equal predictions reduces the model's discriminatory power in the actuarial sense of distinguishing risk levels.
Equalized odds requires that the model's true positive rate and false positive rate are equal across groups. In a classification context (such as fraud detection), this means the model is equally accurate for all groups. A model that detects fraud well for one demographic group but poorly for another fails this criterion.
Calibration across groups requires that when the model predicts a certain probability, the actual outcome frequency is the same regardless of group membership. For example, if the model predicts a 5% claim probability for both Group A and Group B applicants, then approximately 5% of each group should actually file claims. A model can be well-calibrated overall but poorly calibrated within subgroups.
The impossibility theorem states that except in trivial cases, a model cannot simultaneously satisfy demographic parity, equalized odds, and calibration across groups. This means actuaries must choose which fairness criteria to prioritize, which involves value judgments beyond pure statistics.
Proxy Variables and Detection
Even when protected characteristics are excluded from a model, other variables may serve as proxies. Zip code is strongly correlated with race in many areas. Credit score has been shown to have disparate impacts across racial groups. Vehicle type may correlate with income level. Detecting proxy effects requires analyzing model predictions across protected groups, even if those groups are not used as model inputs.
The disparate impact ratio is a common detection tool. Calculate the ratio of the average predicted value for the disadvantaged group to the average predicted value for the advantaged group. Under the four-fifths rule commonly used in employment discrimination cases, a ratio below 0.8 suggests disparate impact. While this rule is not universally applied in insurance, it provides a useful benchmark. On the exam, you may be asked to compute a disparate impact ratio from model output data and interpret whether the result indicates potential unfair discrimination.
Mitigation Strategies
Pre-processing approaches modify the training data before model fitting. Reweighting assigns higher weight to underrepresented groups so that the training data better reflects a balanced population. Resampling techniques such as oversampling the minority group or undersampling the majority group achieve similar effects. These methods are straightforward but may reduce overall model accuracy.
In-processing approaches modify the model training itself. Constrained optimization adds fairness constraints to the loss function, so the algorithm seeks the most accurate model that also satisfies a fairness criterion. For example, you might minimize the Poisson deviance subject to the constraint that the disparate impact ratio exceeds 0.8. This approach allows explicit control over the accuracy and fairness trade-off.
Post-processing approaches adjust model outputs after fitting. Group-specific calibration adjusts predictions so that fairness criteria are met in the final output. Threshold adjustment sets different classification thresholds for different groups to equalize error rates. These methods are simple to implement but may be viewed as less transparent since the underlying model remains unchanged.
Common Pitfalls
Students often assume that removing protected characteristics from the model is sufficient to ensure fairness. This is incorrect because proxy variables can reproduce the same disparate impacts. Another common mistake is treating fairness as a purely statistical problem. The choice of which fairness metric to prioritize involves ethical, legal, and business considerations that go beyond the data. The exam may present scenarios where different stakeholders prefer different fairness criteria and ask you to evaluate the trade-offs.
Study Tips
For MAS-II, practice computing disparate impact ratios and interpreting them in context. Understand the definitions of demographic parity, equalized odds, and calibration well enough to explain them without notes. Be ready to describe at least one pre-processing, one in-processing, and one post-processing mitigation strategy. Review the regulatory landscape, including how state insurance departments evaluate predictive models for unfair discrimination. Finally, understand the impossibility theorem and be prepared to explain why perfect fairness across all metrics simultaneously is generally not achievable.