0
$\begingroup$

I know that formal definitions of the variance covariance matrix has been given elsewhere. However I am not a mathematician and it was very difficult for me to follow them. Thus, I'll make a few short questions to try and clarify. Please answer in the most visual/less formula based as possible (I understand that a bit of formulas may be necessary, but please keep it low).

I think that, ideally, the residuals of an OLS linear regression should be a random variable, with mean zero, not showing autocorrelation, or correlation with any of the variables involved in the regression.

I heard that this means that when express the error as a matrix, it should look like:

               var(x1) cov(x1,x2) cov(x1,x3)
               cov(x2,x1) var(x2) cov(x2,x3)
               cov(x3,x1) cov(xe,x2) var(x3)

Now, if the residuals are independent after the OLS regression, the matrix should look like:

               1 0 0 
               0 1 0
               0 0 1

So, first: Am I right in what I just wrote?

Second: Why a diagonal matrix with units in the diagonal is expected when residuals are independent?

Third: How can a variance be calculated for a single observation? I have seen a previous formal answer for it (that one needs to imagine that numbers in the diagonal could take other values). I still cannot understand that. The variance is normally calculated for a variable, but here we are talking of one single observation. shouldn't it be called something else? besides, if I apply the variance formula (number-mean)*(number-mean) it does not need to take value one. So, why units in the diagonal?

Third, why the off diagonal positions in the matrix should be zero? It seems one can simulate a random error variable, and take any two observations situated off diagonal and obtain with them non zero covariance. For example imagine i get x1=-3 and x2 = 5 as part of a random vector representing the error after an OLS (with mean 0, as expected). Then, (eg. cov(x1, x2)=(-3-0)*(5-0)=-15?? that is far from zero.

Thank you very much in advance!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.