4
$\begingroup$

I'm trying to prove $E(\hat{\sigma}^2)=\sigma^2$, where $\hat{\sigma}^2$ is the error variance and $\sigma^2$ the population variance.

I'm working on the following equation:

$\sum \hat{u}_i^2=\sum(u_i-\bar{u})^2+(\hat{\beta_1}-\beta_1)^2\sum(x_i-\bar{x})^2-2(\hat{\beta}_1-\beta_1)\sum u_i(x_i-\bar{x})$

I'm having trouble proving that $\sum(u_i-\bar{u})^2=(n-1)\sigma^2$

Can someone give me a hint on how to start?

$u_i$ is the error term and $\bar{u}$ is the average of the error term, not the expected value.

proof:

$\sum (u_i-\bar{u})^2=E[\sum u_i^2 - n(\bar{u})^2]$

$\sum E(u_i^2)-E(n(\bar{u})^2)$

$n\sigma^2-nE(\bar{u})^2$

$n\sigma^2-nE(\bar{u})^2$

$n\sigma^2-nE(\frac{\sum u_i}{n})^2$

$n\sigma^2-\frac{1}{n}E(\sum u_i)^2$

$n\sigma^2-\frac{1}{n}E(\sum u_i \sum u_i)$

$\endgroup$
3
$\begingroup$

Little wonder you're having trouble proving $\sum(u_i-\bar{u})^2=(n-1)\sigma^2$, since it isn't the case.

What you should be proving is $E\left[\sum(u_i-\bar{u})^2\right]=(n-1)\sigma^2$.

There are some complete derivations to be found on site - including in the regression context and even using the same notation as you have. Some can be found in the related links in the sidebar to the right.

Outside the regression context (which isn't necessary to the derivation) there are a number of other derivations on site, some relatively simple, some a bit more complicated.

To get you started -- the most obvious strategy is to expand the square and take the expectation inside the sum, then simplify. You may need to rely on some known basic facts about variance and expectation.


Edit:

Thanks for showing some working.

  1. The first line is wrong. You need to be careful about leaving out $E()$s -- make sure they don't just appear and disappear at random times.

  2. Be careful of how you write the expectation of a squared term; in particular if you mean $E[(\bar{u})^2]$ you should probably write that because if you write $E(\bar{u})^2$ it might be interpreted as $(E[\bar{u}])^2$.

  3. You can write the square of the sum of $u_i$ out term by term and see that it's the sum of the squares plus twice the sum of cross products $u_iu_j$, (for $i<j$). However...

  4. If you use basic properties of variance to write down the variance of $\bar{u}$ you'll probably find it easier to find $E[(\bar{u})^2]$

$\endgroup$
  • $\begingroup$ I've been trying really hard proving the equation above without success. I didn't come further than this: $n \sigma^2-1/n(\sum u_i)^2$. I checked the other derivations but wasn't able to figure out how to turn the last term into the sum of the variances. May you help me one more time by giving me a hint/rule on how to deal with the last term? $\endgroup$ – S. Ming Sep 25 '16 at 23:44
  • $\begingroup$ @S.Ming There's no expectation in your expression so you know immediately that you made a mistake somewhere. Show your working in your question via an edit at the end $\endgroup$ – Glen_b Sep 26 '16 at 0:31
  • $\begingroup$ I started with $\sum E(u_i^2)-E(n\bar{u}^2)$. The first term gives $n \sigma^2$. For the second term I assumed that $\bar{u}$ is a constant and thus took it out of the expectation. That was probably my mistake. Anyway, it gave me $n \sigma^2-n\bar{u}^2$. I rewrote the second term then to $n(\frac{\sum u_i}{n})^2$. By taking $1/n$ out of the brackets, I came up with the result in my first comment: $n\sigma^2-1/n(\sum u_i)^2$. Assuming that $\bar{u}$ isn't a constant, then I still could take the expectation of $1/nE((\sum u_i)^2)$. How do I simplify this now? $\endgroup$ – S. Ming Sep 26 '16 at 1:36
  • $\begingroup$ Please put the attempt into your question, at the end, via an edit, as I asked before. $\endgroup$ – Glen_b Sep 26 '16 at 2:13
  • $\begingroup$ Added my attempt.Thought should it put into the comment section, my bad. $\endgroup$ – S. Ming Sep 26 '16 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.