3
$\begingroup$

In a proof I have seen, a linear regression model of the following form was assumed:

$Y = X\beta + \epsilon$

Where $\epsilon \sim N(0, \sigma^2I)$. The proof involved looking at the distribution of the MLE estimate of $\sigma^2$ as $n$ went to infinity. The author had assumed that as $n$ grew large, the estimate of $\sigma^2$, $\hat{\sigma}^2$, followed a central limit theorem (i.e. Eventually became normally distributed with some variance).

This left me thinking - if you assume a central limit theorem for $\hat{\sigma}^2$, could you also assume a central limit theorem for $\hat{\sigma}$? i.e. Is there any reason that it would be more valid to assume a central limit theorem for the variance estimate than the standard deviation estimate or vice versa? Is there any theory that supports taking either of these and in general which is the more appropriate assumption for the linear regression model?

$\endgroup$

1 Answer 1

3
$\begingroup$

If $\hat\sigma^2$ is consistent and asymptotically Normal (and the true value is positive), the same has to be true of $\hat\sigma$, by the delta method.

From $$\sqrt{n}(\hat\sigma^2-\sigma^2)\stackrel{d}{\to} N(0, v)$$ you can conclude $$\sqrt{n}(\hat\sigma-\sigma)\stackrel{d}{\to} N(0, v/\sigma)$$ and vice versa.

There's no simple way to decide which one will have closer to a Normal distribution at a given sample size. You could expect $\hat\sigma^2$ to be closer because that's the scale the CLT is happening on, or expect $\hat\sigma$ to be closer because the square root will reduce the skewness. Asymptotic arguments beyond first order tend to be hard in statistics.

As a terminology issue, I wouldn't describe this as a central limit theorem for $\sigma$, though. A central limit theorem is about the distribution of sums or means, and while $\hat\sigma^2$ genuinely is a mean, $\hat\sigma$ isn't.

$\endgroup$
4
  • $\begingroup$ Thank you! One thing I'm not sure I see though is how the lines you have written follow by the delta method. Would you mind clarifying this a little more? Also I'm not sure what you mean about the variance estimator being a mean? $\endgroup$
    – JDoe2
    Jun 20, 2020 at 10:49
  • $\begingroup$ The variance estimator is just the mean of $(y-\hat y)^2)$. The delta method is here: en.wikipedia.org/wiki/Delta_method $\endgroup$ Jun 25, 2020 at 7:51
  • $\begingroup$ Oh I see! Thanks! Yes, I'm just a little confused on which function g you used above. $\endgroup$
    – JDoe2
    Jun 26, 2020 at 11:51
  • $\begingroup$ either $g(x)=x^2$ or $g(x)=\sqrt{x}$,depending on which way you're going $\endgroup$ Jun 27, 2020 at 5:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.