ARIMA Models: Identification, Estimation, and Forecasting

Non-Stationarity and Differencing

Many real-world time series are non-stationary: they have trends, changing means, or changing variances. An ARIMA(p, d, q) model applies d differences to make the series stationary, then fits an ARMA(p, q) to the differenced series. The first difference is W_t = Y_t - Y_{t-1}. If one difference is not enough, take a second difference. In practice, d is usually 0, 1, or 2.

A random walk Y_t = Y_{t-1} + epsilon_t is an ARIMA(0, 1, 0). A random walk with drift Y_t = c + Y_{t-1} + epsilon_t adds a constant trend.

Model Identification

The Box-Jenkins methodology has three steps: (1) Identification: examine the ACF and PACF of the (possibly differenced) series to determine p and q. (2) Estimation: fit the model parameters using maximum likelihood or conditional least squares. (3) Diagnostic checking: verify that residuals are white noise (no significant autocorrelation). If the Ljung-Box test on residuals is significant, the model is inadequate.

Forecasting

ARIMA forecasts use the estimated model to predict future values. For an AR(1) model, the h-step ahead forecast is Y_hat_{T+h} = phi^h * Y_T (assuming mean zero). Forecast intervals widen as the horizon increases. For MA components, forecasts revert to the mean once the horizon exceeds the MA order. On Exam SRM, practice computing one-step and multi-step forecasts from fitted ARIMA models, and understand that ARIMA forecasts are optimal (minimum mean squared error) linear predictions given the model is correctly specified.