I know that formal definitions of the variance covariance matrix has been given elsewhere. However I am not a mathematician and it was very difficult for me to follow them. Thus, I'll make a few short questions to try and clarify. Please answer in the most visual/less formula based as possible (I understand that a bit of formulas may be necessary, but please keep it low).

I think that, ideally, the residuals of an OLS linear regression should be a random variable, with mean zero, not showing autocorrelation, or correlation with any of the variables involved in the regression.

I heard that this means that when express the error as a matrix, it should look like:

               var(x1) cov(x1,x2) cov(x1,x3)
               cov(x2,x1) var(x2) cov(x2,x3)
               cov(x3,x1) cov(xe,x2) var(x3)

Now, if the residuals are independent after the OLS regression, the matrix should look like:

               1 0 0 
               0 1 0
               0 0 1

So, first: Am I right in what I just wrote?

Second: Why a diagonal matrix with units in the diagonal is expected when residuals are independent?

Third: How can a variance be calculated for a single observation? I have seen a previous formal answer for it (that one needs to imagine that numbers in the diagonal could take other values). I still cannot understand that. The variance is normally calculated for a variable, but here we are talking of one single observation. shouldn't it be called something else? besides, if I apply the variance formula (number-mean)*(number-mean) it does not need to take value one. So, why units in the diagonal?

Third, why the off diagonal positions in the matrix should be zero? It seems one can simulate a random error variable, and take any two observations situated off diagonal and obtain with them non zero covariance. For example imagine i get x1=-3 and x2 = 5 as part of a random vector representing the error after an OLS (with mean 0, as expected). Then, (eg. cov(x1, x2)=(-3-0)*(5-0)=-15?? that is far from zero.

Thank you very much in advance!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.