I know that formal definitions of the variance covariance matrix has been given elsewhere. However I am not a mathematician and it was very difficult for me to follow them. Thus, I'll make a few short questions to try and clarify. Please answer in the most visual/less formula based as possible (I understand that a bit of formulas may be necessary, but please keep it low).
I think that, ideally, the residuals of an OLS linear regression should be a random variable, with mean zero, not showing autocorrelation, or correlation with any of the variables involved in the regression.
I heard that this means that when express the error as a matrix, it should look like:
var(x1) cov(x1,x2) cov(x1,x3) cov(x2,x1) var(x2) cov(x2,x3) cov(x3,x1) cov(xe,x2) var(x3)
Now, if the residuals are independent after the OLS regression, the matrix should look like:
1 0 0 0 1 0 0 0 1
So, first: Am I right in what I just wrote?
Second: Why a diagonal matrix with units in the diagonal is expected when residuals are independent?
Third: How can a variance be calculated for a single observation? I have seen a previous formal answer for it (that one needs to imagine that numbers in the diagonal could take other values). I still cannot understand that. The variance is normally calculated for a variable, but here we are talking of one single observation. shouldn't it be called something else? besides, if I apply the variance formula (number-mean)*(number-mean) it does not need to take value one. So, why units in the diagonal?
Third, why the off diagonal positions in the matrix should be zero? It seems one can simulate a random error variable, and take any two observations situated off diagonal and obtain with them non zero covariance. For example imagine i get x1=-3 and x2 = 5 as part of a random vector representing the error after an OLS (with mean 0, as expected). Then, (eg. cov(x1, x2)=(-3-0)*(5-0)=-15?? that is far from zero.
Thank you very much in advance!