Dimension Reduction Techniques Beyond PCA
Explore dimension reduction methods including PCA extensions and alternatives for Exam MAS-I.
Why Dimension Reduction Matters
Modern actuarial data sets often include many variables. Telematics data from auto insurance may include dozens of driving behavior metrics. Health insurance models may incorporate hundreds of diagnosis codes. When the number of variables is large relative to the number of observations, standard regression methods suffer from overfitting, multicollinearity, and computational challenges. Dimension reduction techniques address these problems by replacing the original high-dimensional data with a smaller number of informative features.
Dimension reduction serves two purposes. First, it improves model performance by removing noise and redundancy from the data. Second, it aids interpretation by summarizing complex multivariate relationships in terms of a few key components. In actuarial work, this makes it possible to build parsimonious pricing models from high-dimensional data.
Principal Component Analysis
PCA finds orthogonal linear combinations of the original variables that maximize variance. The first principal component (PC1) is the direction in the variable space along which the data varies the most. The second principal component (PC2) is the orthogonal direction with the next most variance, and so on. Mathematically, the principal components are the eigenvectors of the covariance (or correlation) matrix, and the eigenvalues give the variance explained by each component.
To perform PCA, start with an n x p data matrix X (n observations, p variables). Standardize the variables (subtract the mean and divide by the standard deviation) if they are on different scales, then compute the correlation matrix R. Find the eigenvalues lambda_1 >= lambda_2 >= ... >= lambda_p and the corresponding eigenvectors v_1, v_2, ..., v_p. The k-th principal component score for observation i is z_{ik} = v_k' * x_i, the projection of observation i onto the k-th eigenvector.
Choosing the Number of Components
The proportion of variance explained by the first k components is (lambda_1 + ... + lambda_k) / (lambda_1 + ... + lambda_p). A common rule is to retain enough components to explain 80-90% of the total variance. The scree plot displays eigenvalues in descending order; you look for an "elbow" where the eigenvalues drop sharply and then level off. Components before the elbow are retained; those after are discarded as noise.
For example, suppose a data set with 10 variables has eigenvalues 4.2, 2.1, 1.3, 0.8, 0.5, 0.4, 0.3, 0.2, 0.1, 0.1. The total variance is 10 (since the correlation matrix has trace p = 10). The first three components explain (4.2 + 2.1 + 1.3) / 10 = 76%. The first four explain 84%. The scree plot would show a clear drop after the third eigenvalue. Retaining 3 or 4 components would be reasonable.
PCA vs. Correlation Matrix
An important decision is whether to apply PCA to the covariance matrix or the correlation matrix. If variables are on vastly different scales (for example, claim amounts in dollars versus claim counts), the covariance matrix PCA will be dominated by the variable with the largest variance. The correlation matrix PCA standardizes all variables to unit variance, treating them equally. In most actuarial applications with mixed-scale variables, PCA on the correlation matrix is preferred.
Factor Analysis
Factor analysis resembles PCA but has a different model. It assumes that observed variables are linear combinations of fewer latent factors plus unique (variable-specific) error: X_j = l_{j1}*F_1 + l_{j2}*F_2 + ... + l_{jk}*F_k + e_j. The loadings l_{jk} describe how each observed variable relates to each latent factor. Unlike PCA, factor analysis explicitly models error and distinguishes shared variance (due to factors) from unique variance (due to error).
Factor analysis is more appropriate when you believe there are genuine underlying constructs driving the observed correlations. For instance, in a policyholder satisfaction survey with 20 questions, factor analysis might reveal three underlying factors: "claim experience satisfaction," "pricing satisfaction," and "service quality satisfaction." Each survey question loads primarily on one factor, providing interpretable groupings.
Partial Least Squares
PCA is unsupervised: it does not use the response variable when finding components. Partial least squares (PLS) finds components that maximize the covariance between the predictors and the response, combining dimension reduction with prediction. The first PLS component is the linear combination of X variables most correlated with Y. PLS is particularly useful when the number of predictors exceeds the number of observations, a situation where standard regression fails entirely.
In actuarial applications, PLS can handle telematics data with many correlated driving metrics by extracting the combinations most predictive of claim frequency. Unlike PCA followed by regression, PLS integrates the prediction objective into the dimension reduction step, often producing better predictive performance with fewer components.
Multidimensional Scaling and Kernel PCA
Multidimensional scaling (MDS) preserves pairwise distances between observations when projecting into lower dimensions. Classical MDS is equivalent to PCA, but metric and non-metric MDS variants can handle non-linear relationships. MDS is useful for visualization: it produces a low-dimensional map where similar observations are close together and dissimilar ones are far apart.
Kernel PCA applies PCA in a transformed feature space using the "kernel trick." By choosing an appropriate kernel function (such as polynomial or radial basis function), kernel PCA can capture non-linear relationships that standard PCA misses. For example, if two risk groups are separated by a non-linear boundary in the original variable space, kernel PCA can find a linear boundary in the transformed space.
Common Pitfalls
A frequent mistake is interpreting principal components as if they were meaningful latent variables. PCA maximizes variance, which is a mathematical criterion, not necessarily aligned with meaningful real-world constructs. Factor analysis is more appropriate when interpretable latent variables are the goal. Another error is retaining too few or too many components: too few loses important information, while too many reintroduces the problems dimension reduction was meant to solve.
Students also sometimes apply PCA to binary or categorical variables without modification. Standard PCA assumes continuous variables; for categorical data, methods like multiple correspondence analysis (MCA) are more appropriate.
Study Tips
Exam MAS-I covers PCA mechanics, interpretation, and the conceptual basis for alternatives. Practice computing principal components from a small correlation matrix: find eigenvalues and eigenvectors, compute component scores, and determine the proportion of variance explained. Know the difference between PCA (maximizes variance, unsupervised) and PLS (maximizes covariance with response, supervised). Understand when factor analysis is preferred over PCA. Be prepared to interpret a scree plot and justify the number of retained components.