4
$\begingroup$

When fitting a model (e.g. ARIMA but really any model for this matter) I've been taught to examine the residuals for autocorrelation. This is true with any model - you expect the residuals to be white noise, i.e. i.i.d if the model fits the data (see image below).

However, taking the ACF of the residuals does not prove they are independent because you can have a series that is uncorrelated but not independent. For such series, taking the ACF of absolute values might show strong correlation after lag 0.

So when examining the residuals to determine the model fits the data and all that is left is white noise, should one also look at the ACF of absolute or squared residuals too?

enter image description here

$\endgroup$
1
  • $\begingroup$ Besides the two insightful answers you already have, you might like to read about the BDS test which is generally used for testing nonlinearity but is actually a test for checking 'iid'. The idea is that even when there is zero correlation some nonlinear or other form of dependence may be lurking and needs to be ruled out. $\endgroup$
    – Dayne
    Aug 16, 2021 at 4:39

2 Answers 2

4
$\begingroup$

Your intuition is correct. I.i.d. errors is the ideal, and autocorrelation in their absolute values or squares violates that. For some models and some of their uses, uncorrelated errors may be good enough.

E.g. in a regression with time series data, uncorrelated but dependent errors could yield consistent and unbiased estimates and prediction intervals that have correct coverage unconditionally (on average over time). This may be good enough. Meanwhile, i.i.d. errors could yield consistent, unbiased and efficient estimates and prediction intervals with correct coverage conditional on all available information. This is better.

Examining autocorrelation in absolute values and/or squares of residuals is standard practice when one suspects presence of autoregressive conditional heteroskedasticity. This is common e.g. in financial variables such as stock prices or their logarithmic returns. Model such as GARCH are used to account for such autocorrelation.

$\endgroup$
2
  • $\begingroup$ Richard: just to be clear, did you mean 'errors' in place of 'residuals' in your second para? I am asking because if it is true for residuals also then the estimation method must also be specified. $\endgroup$
    – Dayne
    Aug 16, 2021 at 4:38
  • 1
    $\begingroup$ @Dayne, errors might be a better term. I think residuals are used both for $\varepsilon$ and $\hat\varepsilon$ while errors just for $\varepsilon$, so there is less ambiguity if the latter term is used for $\varepsilon$. I have updated my answer accordingly. I think it brings it in line with Ben's answer, and it is nice to have shared terminology in a single thread. $\endgroup$ Aug 16, 2021 at 5:29
3
$\begingroup$

Firstly, it is worth noting the relationship of independence to uncorrelatedness. If $\varepsilon_1,...,\varepsilon_n$ are independent, that is equivalent to saying that, for any univariate transformation function $g: \mathbb{R} \rightarrow \mathbb{R}$, you have:

$$\mathbb{Cov}(g(\varepsilon_i),g(\varepsilon_j)) = 0.$$

(And consequently, the correlation is also zero so long as the transformation leads to a non-zero variance.) So, independence between the residuals is equivalent to saying that there is zero covariance under all possible univariate transformations of the random variables. One way of looking at this is to say that zero correlation for a specific transformation $g$ is an aspect of independence. Consequently, if you want to test whether a set of random variables are independent, this is the same as testing that they are uncorrelated under a variety of univariate transformations (unfortunately an infinite variety!).

Consequently, your intuition here is quite reasonable. By testing for correlation between absolute or squared residuals, you are testing two other aspects of independence. If the residuals were independent then with sufficient data you would find zero correlation in both these tests. Now, bear in mind that there are some complications with testing multiple hypotheses on a single set of data, because you have a "multiple comparisons" problem. Moreover, in most regression/time-series models, independence of the error terms leads to residuals that are almost independent, but not quite, so you may need to think about whether you actually want to test the residuals for strict independence or not. In any case, setting aside those complicating issues, your basic idea is fine --- correlation between absolute values or squared values would entail a kind of dependence.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.