ACF of absolute or squared residuals?

Question

When fitting a model (e.g. ARIMA but really any model for this matter) I've been taught to examine the residuals for autocorrelation. This is true with any model - you expect the residuals to be white noise, i.e. i.i.d if the model fits the data (see image below).

However, taking the ACF of the residuals does not prove they are independent because you can have a series that is uncorrelated but not independent. For such series, taking the ACF of absolute values might show strong correlation after lag 0.

So when examining the residuals to determine the model fits the data and all that is left is white noise, should one also look at the ACF of absolute or squared residuals too?

Answer 1

Your intuition is correct. I.i.d. errors is the ideal, and autocorrelation in their absolute values or squares violates that. For some models and some of their uses, uncorrelated errors may be good enough.

E.g. in a regression with time series data, uncorrelated but dependent errors could yield consistent and unbiased estimates and prediction intervals that have correct coverage unconditionally (on average over time). This may be good enough. Meanwhile, i.i.d. errors could yield consistent, unbiased and efficient estimates and prediction intervals with correct coverage conditional on all available information. This is better.

Examining autocorrelation in absolute values and/or squares of residuals is standard practice when one suspects presence of autoregressive conditional heteroskedasticity. This is common e.g. in financial variables such as stock prices or their logarithmic returns. Model such as GARCH are used to account for such autocorrelation.

Answer 2

Firstly, it is worth noting the relationship of independence to uncorrelatedness. If $\varepsilon_1,...,\varepsilon_n$ are independent, that is equivalent to saying that, for any univariate transformation function $g: \mathbb{R} \rightarrow \mathbb{R}$, you have:

$$\mathbb{Cov}(g(\varepsilon_i),g(\varepsilon_j)) = 0.$$

(And consequently, the correlation is also zero so long as the transformation leads to a non-zero variance.) So, independence between the residuals is equivalent to saying that there is zero covariance under all possible univariate transformations of the random variables. One way of looking at this is to say that zero correlation for a specific transformation $g$ is an aspect of independence. Consequently, if you want to test whether a set of random variables are independent, this is the same as testing that they are uncorrelated under a variety of univariate transformations (unfortunately an infinite variety!).

Consequently, your intuition here is quite reasonable. By testing for correlation between absolute or squared residuals, you are testing two other aspects of independence. If the residuals were independent then with sufficient data you would find zero correlation in both these tests. Now, bear in mind that there are some complications with testing multiple hypotheses on a single set of data, because you have a "multiple comparisons" problem. Moreover, in most regression/time-series models, independence of the error terms leads to residuals that are almost independent, but not quite, so you may need to think about whether you actually want to test the residuals for strict independence or not. In any case, setting aside those complicating issues, your basic idea is fine --- correlation between absolute values or squared values would entail a kind of dependence.