I am having a hard time understanding the elements of an error covariance matrix for a class. Can someone clarify?

First, the diagonal. The variance is $E(e_i^2) - E(e_i)^2$. $E(e_i) = 0$, so it's just $E(e_i^2)$. Ok . . . so that is the square of the actual value of $e_i$? $y_i - ŷ_i$? A non-zero number, i.e., how much that error value "varies" from the expected zero? So, how does the E work in this?

The off-diagonal terms = $E(e_ie_j)$? The expected value of their product? What is that telling you?

My notes say: "The graph suggests that the variance around the regression line is constant: $\sigma^2 = \sigma_i^2$ "

enter image description here

I can't see it. Where is the constant? The ei values are all over the place, though their mean/expected value does look like zero. And why would the off-diagonal terms be zero? I would expect constant error variance to look something like:

enter image description here


1 Answer 1


There might be a little confusion in your subscripts. The 'i' in y_i usually represent a line/individual. But here, the 'i' in e_i would rather code for the value on the horizontal axis. Your problem can be seen has a simple linear regression of one variable on the other (http://en.wikipedia.org/wiki/Linear_regression).

So what you are looking is the e_i variables, each corresponding to all points with the same value on horizontal axis. Now, remember that the assumption in regression is often that errors will behave like gaussian distributions (It is the case for your simple linear regression). So you will never expect something like your second graph because constant variance just means that the error values will lie in approximately the same interval.

The conclusion you want to make is that there is no heteroscedasticity (http://en.wikipedia.org/wiki/Heteroscedasticity). If there was, It would have been necessary to change your Normal hypothesis.

Hope that helps.

  • $\begingroup$ A little. At least, I did not know the i's were different. (Really they are?) Ok, I can imagine little normal distributions all lined up along the regression line . . . but then what is going in the error covariance matrix? $\endgroup$
    – J Kelly
    Mar 10, 2014 at 20:10
  • $\begingroup$ well if you see no pattern the error covariance matrix must be diagonal with all variance equals, which is the case you want to have. If not, the regression performs better in some cases than other. $\endgroup$
    – Scratch
    Mar 11, 2014 at 9:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.