Support Vector Machines for Insurance Classification
Learn SVM fundamentals and their application to insurance classification on Exam SRM.
Maximal Margin Classifier
The maximal margin classifier finds the hyperplane that separates two classes with the largest margin. The margin is the perpendicular distance from the hyperplane to the nearest data points on either side. A hyperplane in p-dimensional space is defined by w'x + b = 0, where w is the normal vector and b is the offset. An observation x is classified based on the sign of w'x + b.
The data points closest to the hyperplane are called support vectors. A remarkable property of this classifier is that the decision boundary depends only on these support vectors, not on other observations far from the boundary. Moving or removing a non-support-vector observation does not change the classifier at all.
For data that is not linearly separable (no hyperplane perfectly separates the two classes), the maximal margin classifier does not exist. This motivates the soft margin approach, which allows some degree of misclassification in exchange for a wider margin.
Support Vector Classifier (Soft Margin)
The support vector classifier relaxes the requirement for perfect separation by introducing slack variables xi_i >= 0 for each observation. The optimization minimizes (1/2)||w||^2 + C * sum(xi_i), subject to y_i(w'x_i + b) >= 1 - xi_i. When xi_i = 0, the observation is correctly classified and outside the margin. When 0 < xi_i < 1, the observation is correctly classified but inside the margin. When xi_i > 1, the observation is misclassified.
The cost parameter C controls the trade-off between margin width and classification accuracy. Large C penalizes misclassifications heavily, resulting in a narrow margin, low bias, and high variance. Small C allows more misclassifications, producing a wide margin, higher bias, and lower variance. This bias-variance trade-off makes C a key tuning parameter, typically chosen by cross-validation.
The total slack sum(xi_i) is bounded above by the number of training observations that are misclassified or inside the margin. This means C effectively limits how many observations can violate the margin, with larger C being more restrictive.
The Kernel Trick
For non-linear decision boundaries, SVMs use the kernel trick: replace the dot product x_i'x_j in the optimization with a kernel function K(x_i, x_j) that implicitly maps data to a higher-dimensional feature space. The classifier then finds a linear boundary in this high-dimensional space, which corresponds to a non-linear boundary in the original space.
Common kernels include: the polynomial kernel K(x, x') = (1 + x'x')^d, which captures interactions up to degree d; and the radial basis function (RBF) kernel K(x, x') = exp(-gamma * ||x - x'||^2), which can model highly non-linear and even non-contiguous boundaries. The linear kernel K(x, x') = x'x' is simply the support vector classifier without any non-linear mapping.
The parameter gamma in the RBF kernel controls how quickly the similarity between points decays with distance. Large gamma means only very close points influence each other (high variance, flexible boundary); small gamma means distant points also have influence (high bias, smooth boundary). Both C and gamma must be tuned for the RBF kernel.
SVMs for Multi-Class Classification
SVMs are inherently binary classifiers. For multi-class problems (more than two classes), two common strategies are used. One-vs-one (OVO) builds a classifier for every pair of classes, producing K*(K-1)/2 classifiers for K classes, and assigns each test observation to the class that wins the most pairwise comparisons. One-vs-all (OVA) builds K classifiers, each separating one class from all others, and assigns the test observation to the class whose classifier has the highest decision function value.
In actuarial applications with multiple risk categories (e.g., low, medium, high, and very high risk), multi-class SVMs can classify policyholders. However, the loss of interpretability compared to multinomial logistic regression or ordinal regression is a practical concern.
Advantages and Limitations
SVMs have several strengths: they work well in high-dimensional spaces (even when the number of features exceeds the number of observations), they are robust to outliers (the margin is determined by only a few support vectors), and kernel methods can capture complex non-linear relationships. The resulting classifier is also unique (the optimization has a single global solution).
Limitations include: SVMs do not directly produce probability estimates (though Platt scaling can approximate probabilities), they are computationally expensive for very large datasets, and they are less interpretable than logistic regression, decision trees, or GLMs. Feature scaling (standardization) is important because the margin calculation is sensitive to the scale of predictors.
In the insurance context, SVMs are used for fraud detection, claim classification, and risk segmentation. However, actuaries often prefer GLMs or tree-based methods for ratemaking because regulatory requirements demand model interpretability and transparency.
Insurance Applications and Exam SRM Focus
For Exam SRM, the focus is on conceptual understanding rather than mathematical details of the optimization. You should be able to explain what the margin represents, how the cost parameter C affects the bias-variance trade-off, what the kernel trick accomplishes, and when SVMs are preferred over other classifiers. Understanding the role of support vectors and why only they determine the decision boundary is essential.
Typical exam questions might ask you to compare SVMs with logistic regression or decision trees, to explain the effect of increasing or decreasing C, or to describe why the RBF kernel can overfit with large gamma values. Know that SVMs maximize the margin (a geometric concept), while logistic regression maximizes the likelihood (a probabilistic concept).
Study Tips and Common Pitfalls
The most common mistake is confusing the cost parameter C with the kernel parameters (gamma for RBF, degree d for polynomial). C controls how much misclassification is tolerated; kernel parameters control the flexibility of the decision boundary shape. Both affect the bias-variance trade-off, but they do so through different mechanisms.
Another pitfall is forgetting to standardize features before fitting an SVM. Since the margin is based on Euclidean distance, features with larger scales will dominate the distance calculation. Always standardize (mean 0, variance 1) all predictors when using SVMs. Practice at least 4 conceptual problems comparing SVMs with other classifiers and interpreting the effects of tuning parameters.