Consider the standard linear regression model $$ y = X \beta + \varepsilon, $$ where the error $\varepsilon$ has fixed variance $\sigma^2$. We can make an unbiased estimate of the error variance in a linear regression model using the OLS estimator $s^2 = \frac{1}{n-p}\sum_{i=1}^n(y_i-\hat y_i)^2$ where $p$ is the number of regressors.

I am wondering what is the variance of the estimate $s^2$ and what is it's probability distribution? If we need to make assumptions about $\varepsilon$ before we can say anything about the distribution of $s^2$, then assume $\varepsilon$ is normally distributed.

I'm interested in the case of stochastic regressors (as opposed to fixed), if that makes a difference.

  • $\begingroup$ en.wikipedia.org/wiki/… $\endgroup$ – user2974951 Nov 6 '20 at 11:44
  • $\begingroup$ I don't see the variance or probability distribution of $s^2$ listed there. $\endgroup$ – Bertus101 Nov 6 '20 at 12:45
  • 1
    $\begingroup$ Hints: (1) You must make a strong distributional assumption, as you suggest. (2) Because $\hat y,$ being a linear combination of the $y_i,$ is Normal when $\varepsilon$ is Normal, the distribution of a suitable multiple of $s^2$ is a chi-square distribution. The rest is just calculation. $\endgroup$ – whuber Nov 6 '20 at 12:54
  • $\begingroup$ Ok, I'll try it out thanks. $\endgroup$ – Bertus101 Nov 6 '20 at 13:41
  • 2
    $\begingroup$ Something else is that as $n$ increases, it asymptotically converges in distribution to the normal distribution by a central limit theorem argument (under certain regularity conditions). This is consistent with @whuber's comment for the special case where $\epsilon$ is normal: the $\chi^2$ distribution with $k = n-p$ degrees of freedom approaches the normal distribution for large $k$. $\endgroup$ – Matthew Gunn Nov 6 '20 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.