2
$\begingroup$

http://web.stanford.edu/~mrosenfe/soc_meth_proj3/matrix_OLS_NYU_notes.pdf

On page 4, it says that $x_k'e = 0$ implies that each regressor has zero sample correlation with the residuals.

I don't see why this is true because to find the covariance of $x_k$ and $e$ we would also need for $E[e]$ or $E[x_k]$ to be $0$ as well. Yet this isn't assumed.

Maybe I don't understand what this means because it also says $X$ is not necessarily uncorrelated with the disturbances.

Help appreciated.

$\endgroup$
0

1 Answer 1

1
$\begingroup$

it says that $x_k'e=0$ implies that each regressor has zero sample correlation with the residuals.

Note that the statement in doubt does not mention correlation, but sample correlation. This means you do not have to involve expected values there. However, sample correlation has the following formula:

$$ r_{x_k,e} = \frac{\sum_{i=1}^n{(x_{ki}-\bar{x_k})(e_i-\bar{e})}}{(n-1)s_{x_k}s_e} $$

where $s_{x_k}$ and $s_{e}$ are sample standard deviations.

In order to have sample correlation equal with $0$, we need only the sum to be zero. But:

$$ \sum_{i=1}^n(x_{ki}-\bar{x_k})(e_i-\bar{e}) = \sum_{i=1}^nx_{ki}e_i -n\bar{x_k}\bar{e} = x_k'e -n\bar{x_k}\bar{e} = -n\bar{x_k}\bar{e}$$

From this point I see no reason that previous expression to be $0$, unless:

  • $X$ contains a constant term, and obviously $\sum_{i=1}^{n}e_i=0$, and the sample correlation is $0$
  • $x_k$ columns of $X$ are already centered, because that would mean that $\sum_{i=1}^nx_{ki}=0$, for all $k$ columns, and also, sample correlationwould be $0$.

I would be very interested if somebody find a proof of that, but from what I recall, two vectors are orthogonal if their dot product is $0$, and two vectors have sample correlation $0$ if their centered vectors dot produc equals $0$. In other words orthogonality property works directly on vectors and zero correlation property works on centered vectors.

Maybe I don't understand what this means because it also says $X$ is not necessarily uncorrelated with the disturbances.

Here you have to note that while the previous discussion was related with residuals (denoted with $e$). This statement implies that you cannot infer something about disturbances (denoted with $\epsilon)$ directly from the relation with residuals.The relations between independent variables and residuals are derived because you optimized by least mean squares, also input variables and residuals are known. However disturbances are not known and some statistical assumtions might be assumed (Gauss Markov assumptions).

Hope it helps somehow.

$\endgroup$
1
  • 1
    $\begingroup$ You are correct (+1). As a counterexample consider a design matrix $X=(1,2,3)^\prime$ and response $y=(7,7,7)^\prime$. The OLS solution is $\beta=(3)$, whence the fit is $X^\prime\beta=(1,2,3)^\prime(3) = (3,6,9)^\prime$ with residuals $(7,7,7)^\prime-(3,6,9)^\prime=(4,1,-2)^\prime.$ The correlation between $x_1 = (1,2,3)^\prime$ and the residuals is $-1$, not $0$. $\endgroup$
    – whuber
    Nov 19, 2014 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.