3
$\begingroup$

One of the assumptions of classical linear regression is that errors are uncorrelated. Does this imply that the residuals are uncorrelated as well? For concreteness, assume that we are using an OLS estimator for the model parameters.

My intuition tells me "no", which seems supported by the absence of any proofs about uncorrelated residuals that I can find.

Improve this question
$\endgroup$
6
  • 5
    $\begingroup$ Short answer: no. This is mentioned in many posts here on CV (but is hard to search for, because many posts concern correlations among the explanatory variables) and is obvious when you note that (at least when there's an intercept) the residuals must sum to zero. $\endgroup$
    – whuber ♦
    Commented Aug 12 at 1:03
  • 1
    $\begingroup$ @whuber The requirement that the residuals sum to zero certainty show dependence. What I don’t think I’ve ever seen made explicit is why that dependence is correlation. $\endgroup$
    – Dave
    Commented Aug 12 at 2:35
  • $\begingroup$ @Dave Expand the expression $0=\operatorname{Var}(1)=\operatorname{Var}\sum \varepsilon_i=\sum \operatorname{Var}(\varepsilon_i)+\sum_{i\ne j}\operatorname{Cov}(\varepsilon_i,\varepsilon_j)$ to deduce the latter sum must include at least one negative term, QED. $\endgroup$
    – whuber ♦
    Commented Aug 12 at 12:40
  • 1
    $\begingroup$ @whuber found duplicate as first hit of regression residuals are correlated hat matrix site:https://stats.stackexchange.com $\endgroup$
    – kjetil b halvorsen ♦
    Commented Aug 12 at 13:57
  • 1
    $\begingroup$ @kjetilbhalvorsen Good find--and good example of how to search. $\endgroup$
    – whuber ♦
    Commented Aug 12 at 15:14

1 Answer 1

Reset to default
5
$\begingroup$

It's easy to show using the definition of the residuals $\mathbf e=(\mathbf I_n-\mathbf H) \mathbf Y, $ where $\mathbf H$ is the hat matrix and is idempotent, that if the error $\boldsymbol{\varepsilon}\sim \mathsf N(\boldsymbol 0,\sigma^2\mathbf I),$ then $\mathbf e$ follows a singular (as $\mathbf I_n-\mathbf H$ is not of full rank and hence singular) normal distribution with $\mathbb E[\mathbf e]=\mathbf 0$ and $\operatorname{Var}(\mathbf e) =\sigma^2(\mathbf I_n-\mathbf H). $ That is, $\operatorname{Cov}(e_i, e_j) =-\sigma^2 h_{ij}$ where $h_{ij}= \mathbf x_i^\top (\mathbf X^\top\mathbf X)^{-1}\mathbf x_j.$

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.