Understanding covariance of errors in regression

Question

I am having a hard time understanding the elements of an error covariance matrix for a class. Can someone clarify?

First, the diagonal. The variance is $E(e_i^2) - E(e_i)^2$. $E(e_i) = 0$, so it's just $E(e_i^2)$. Ok . . . so that is the square of the actual value of $e_i$? $y_i - ŷ_i$? A non-zero number, i.e., how much that error value "varies" from the expected zero? So, how does the E work in this?

The off-diagonal terms = $E(e_ie_j)$? The expected value of their product? What is that telling you?

My notes say: "The graph suggests that the variance around the regression line is constant: $\sigma^2 = \sigma_i^2$ "

I can't see it. Where is the constant? The ei values are all over the place, though their mean/expected value does look like zero. And why would the off-diagonal terms be zero? I would expect constant error variance to look something like:

Answer 1

There might be a little confusion in your subscripts. The 'i' in y_i usually represent a line/individual. But here, the 'i' in e_i would rather code for the value on the horizontal axis. Your problem can be seen has a simple linear regression of one variable on the other (http://en.wikipedia.org/wiki/Linear_regression).

So what you are looking is the e_i variables, each corresponding to all points with the same value on horizontal axis. Now, remember that the assumption in regression is often that errors will behave like gaussian distributions (It is the case for your simple linear regression). So you will never expect something like your second graph because constant variance just means that the error values will lie in approximately the same interval.

The conclusion you want to make is that there is no heteroscedasticity (http://en.wikipedia.org/wiki/Heteroscedasticity). If there was, It would have been necessary to change your Normal hypothesis.

Hope that helps.