ARIMA Models: Identification, Estimation, and Forecasting
Study ARIMA models for non-stationary time series on Exam SRM.
Non-Stationarity and Differencing
Many real-world time series are non-stationary, exhibiting trends, changing means, or evolving variance. ARMA models require stationarity, so non-stationary series must be transformed before modeling. The most common transformation is differencing. The first difference W_t = Y_t - Y_{t-1} removes a linear trend. If the first-differenced series still shows non-stationarity, a second difference can be applied: V_t = W_t - W_{t-1} = Y_t - 2*Y_{t-1} + Y_{t-2}.
An ARIMA(p, d, q) model applies d differences to make the series stationary, then fits an ARMA(p, q) to the differenced series. The "I" in ARIMA stands for "integrated," indicating that the original series is the d-th order integral (cumulative sum) of the stationary ARMA process. In practice, d is almost always 0, 1, or 2. Taking more than two differences is rarely needed and can introduce artificial patterns.
Two important special cases deserve attention. A random walk Y_t = Y_{t-1} + epsilon_t is an ARIMA(0, 1, 0). Its first difference W_t = epsilon_t is white noise, confirming that one difference achieves stationarity. A random walk with drift Y_t = c + Y_{t-1} + epsilon_t is also ARIMA(0, 1, 0) with a drift term. The drift c produces a linear trend in the original series: E[Y_T] = Y_0 + c*T.
The Box-Jenkins Methodology
The Box-Jenkins approach to time series modeling consists of three iterative stages. Each stage serves a distinct purpose, and the process may cycle back to earlier stages if diagnostics reveal problems.
Stage 1, Identification: Determine the orders p, d, and q. First, assess whether differencing is needed by examining the time series plot and the sample ACF. A non-stationary series has an ACF that decays very slowly (does not die out within a few lags). Apply differences until the ACF shows a stationary pattern. Then examine the ACF and PACF of the differenced series to identify p and q using the standard patterns: ACF cuts off at lag q with PACF tailing off suggests MA(q); PACF cuts off at lag p with ACF tailing off suggests AR(p); both tailing off suggests ARMA.
Stage 2, Estimation: Fit the candidate model(s) using maximum likelihood or conditional least squares. Most statistical software performs this automatically. Record the estimated parameters, their standard errors, and information criteria (AIC, BIC). If multiple candidate models are plausible, fit them all and compare.
Stage 3, Diagnostic Checking: Verify that the residuals from the fitted model behave like white noise. The Ljung-Box test statistic Q = n(n+2) * sum_{k=1}^{K} (r_k^2 / (n-k)) tests whether the first K residual autocorrelations are jointly zero. Under the null hypothesis of adequate fit, Q follows a chi-squared distribution with K - p - q degrees of freedom. A significant Q (p-value < 0.05) indicates the model is inadequate and more terms may be needed. Also inspect the residual ACF plot for any remaining patterns.
Determining the Order of Differencing
Deciding how many times to difference is a critical step. Over-differencing introduces unnecessary complexity and can create artificial MA patterns. Under-differencing leaves non-stationarity in the series, invalidating the ARMA analysis.
Practical guidelines: Plot the series. If there is an obvious trend (upward or downward), try d = 1. After differencing, plot the ACF. If the ACF of the first-differenced series still decays very slowly, try d = 2. Unit root tests (such as the Augmented Dickey-Fuller test) provide a formal statistical test for non-stationarity: if the test fails to reject the null hypothesis of a unit root, differencing is needed.
For example, consider quarterly insurance premium income that grows steadily over time. The original series has a clear upward trend and a slowly decaying ACF. After first differencing, the trend is removed, and the ACF and PACF show patterns consistent with a low-order ARMA model. This first-differenced series can be modeled as ARMA(p, q), making the original series ARIMA(p, 1, q).
Seasonal ARIMA Models
Many actuarial time series exhibit seasonality (regular patterns repeating every m periods, such as quarterly or monthly cycles). The seasonal ARIMA model, written ARIMA(p, d, q) x (P, D, Q)_m, adds seasonal differencing and seasonal AR/MA terms. The seasonal difference is Y_t - Y_{t-m}. Seasonal AR and MA terms operate at multiples of the seasonal period: Phi_1 * Y_{t-m} or Theta_1 * epsilon_{t-m}.
For example, ARIMA(1, 1, 0) x (1, 1, 0)_4 for quarterly data applies both a regular first difference and a seasonal difference (lag 4), then fits an AR(1) component at lag 1 and a seasonal AR(1) at lag 4. The seasonal ACF and PACF (examined at lags m, 2m, 3m, ...) guide the choice of P and Q, just as the regular ACF and PACF guide p and q.
Forecasting with ARIMA Models
ARIMA forecasts are minimum mean squared error predictions given the model is correctly specified. The forecast function depends on the model structure. For ARIMA(0, 1, 0), the random walk, the forecast for all future periods is simply Y_T (the last observed value). For ARIMA(0, 1, 0) with drift, the forecast is Y_T + c*h, where h is the forecast horizon.
For more complex ARIMA models, forecasts are computed recursively. Write the model in terms of the original series and substitute forecasts for unknown future values and zeros for unknown future errors. For example, for ARIMA(1, 1, 1): the one-step forecast uses the last observed value and the last estimated residual. For h >= 2, future residuals are set to zero.
Forecast intervals widen with the horizon h, reflecting increasing uncertainty. The interval is Y_hat_{T+h} +/- z_{alpha/2} * sqrt(sum_{j=0}^{h-1} psi_j^2 * sigma^2), where the psi weights come from the MA(infinity) representation of the model. Short-horizon forecasts are more precise than long-horizon ones, and ARIMA forecasts eventually converge to the long-run mean (for stationary components) or continue along the drift trend (for integrated components).
Study Tips for Exam SRM
For Exam SRM, focus on these key skills. First, be able to determine the order of differencing from a time series plot and ACF. Second, apply the Box-Jenkins identification procedure using ACF and PACF patterns. Third, interpret the Ljung-Box test for residual diagnostics. Fourth, compute one-step and multi-step forecasts for simple ARIMA models like ARIMA(1, 0, 0), ARIMA(0, 1, 0), and ARIMA(0, 1, 1). Practice questions that give you an ACF/PACF plot and ask you to identify the appropriate ARIMA model, as this is one of the most common exam formats. Remember that AIC and BIC are your best tools for choosing between competing models when ACF/PACF identification is ambiguous.