In linear regression, we are usually required to run the model diagnostics to see whether the assumptions such as normality assumption, independence assumption, homoskedasticity assumption, are properly met. I am just worried about these tests are designed for independent data, but the residuals are dependent. Why people just use these tests without any explanations of validation? Does the dependent nature of residuals affect the results of the tests? Thanks!


Residual dependency does affect things, but hopefully if we have a reasonable amount of data, it should not affect things much. Under standard OLS estimation (and assuming that the underlying error terms in the model are uncorrelated and homoskedastic) the correlation matrix for the residuals is known to be:

$$\mathbb{Corr}(\mathbf{R}) = \mathbf{I} - \mathbf{h} = \mathbf{I} - \mathbf{x}(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T},$$

where the matrix $\mathbf{h} = \mathbf{x}(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}$ is known as the hat matrix. Now, if the number of data points $n$ is large, we may appeal to asymptotic results for the hat matrix to approximate the correlation of the residual vector. These asymptotic results depend on conditions for the limits of the design matrix, which of course depends on the underlying sampling distribution for the explanatory data. In some broad cases (e.g., if the underlying explanatory vectors for the data points are themselves IID) we can establish that $\mathbf{h} \rightarrow \mathbf{0}$ so that the residual variables are asymptotically uncorrelated. (This relates closely to the Grenander conditions for consistency of the OLS estimator.)

For this reason, for large samples, residual dependence usually reflects an underlying correlation in the error terms in the model. Note also that diagnostic tests aren't usually done with the raw residuals. Instead, we generally use (externally) studentised residuals that are already adjusted to remove the heteroskedasticity induced by the OLS estimation process. While correlation between residual values remains, it should be small so long as we have a reasonable amount of data and the underlying error terms are, in fact, uncorrelated.

  • $\begingroup$ Thank you for your reply. First of all, why the correlation matrix for the residuals look like this? Shouldn't the (i,j) entry be -hij/sqrt(1-hii)sqrt(1-hjj)? Secondly, I don't understand why h goes to zero, even after consulting the linke of the Grenander conditions. Finally, why the studendized residuals, which standardize deleted residuals, "already adjusted to remove the heteroskedasticity induced by the OLS estimation process". Looking forward to your reply. Thank you very much in advance! $\endgroup$
    – Lizzy
    Apr 12 at 5:43
  • $\begingroup$ This is quite a large topic, which would usually be treated in a larger text or lecture notes. I'm afraid my answer will not be a substitute for this. If you would like to learn more about the details, I recomment you read a text or lecture notes on the asymptotic theory for OLS estimation. Sorry I can't be more helpful. $\endgroup$
    – Ben
    Apr 12 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.