In simple linear regression, it was proved that $(n-2)MS_{RES}/\sigma^2$ follows a $\chi^2_{n-2}$ distribution.In order to prove this,the fact that $e_i$ or $y_i - \hat{y_i}$ follows a Normal distribution with mean 0 and variance $\sigma^2$ was used.However,when I tried to solve for the variance of $i^{th}$ residual error($e_i$ or $y_i -\hat{y}_i$),it is coming out to be $\sigma^2 (1- 1/n - (Xi-\bar{X})^2/Sxx)$.I would like to know whether anyone can prove var($e_i$) = $\sigma^2$ using simple linear regression(No matrix algebra).

  • $\begingroup$ I do not quite grasp your question. AFAIK a single residual error does not have its own variance (i.e. the variance of $e$ exists, but $e_i$ is just the single value resulting from $y_i - \hat{y_i}$). If your question is focused on solving or reproducing the prove (where a reference is missing) that is definitely not my forte. I do think you omitted some steps in your solving of these equations. Showing your thought process might help others explain more clearly. $\endgroup$ – IWS Aug 15 '17 at 12:59
  • 2
    $\begingroup$ In simple linear regression, $y_i$ is a random variable, i.e. $y_i$ ~ N($\beta_0 + \beta_1x_i$,$\sigma^2$)and what we have is one such realization of that random variable($y_i$),similarly $\hat{y_i}$ is also a random variable since $\hat{y_i}$ = $\hat{\beta_o}$+ $\hat{\beta_1}x_i$ is a combination of random variables $\hat\beta_0$ and $\hat\beta_1$.Thus $e_i$= $y_i - \hat{y_i}$ will also be a random variable with some mean and variance.Although I agree that we have only one such realization of $e_i$ but that doesn't mean that it will not have any variance. $\endgroup$ – Sudhanshu Vashisth Aug 15 '17 at 15:04
  • $\begingroup$ You need to sum these variances over $i$. The desired result drops right out. $\endgroup$ – whuber Aug 15 '17 at 16:12
  • $\begingroup$ Sorry,I didn't get it.As per me,are you saying that $variance(\sum{e_i})$ = $(n-2)\sigma^2$.But how will it prove that $\sum{e_i^2}/\sigma^2$ will follow a $\chi^2_{n-2}$ ? In order for this result to hold true, $e_i$ should follow N(0,$\sigma^2$)so that ${e_i^2}/\sigma^2$ follows a $\chi^2_1$ and hence $\sum{e_i^2}/\sigma^2$ will follow $\chi^2_{n-2}$ $\endgroup$ – Sudhanshu Vashisth Aug 15 '17 at 16:50
  • $\begingroup$ See stats.stackexchange.com/… for various answers. $\endgroup$ – whuber Aug 16 '17 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.