# Is $\hat{\sigma^2}=\hat{\sigma}^2$?

In simple linear regression $$Y_i=b_0+b_1X_i+\varepsilon_i$$ where $$\varepsilon_i\sim N(0, \sigma^2)$$ is it true to say that the estimator for variance and the estimator for SD squared are equal? I mean$$\hat{\sigma^2}=\hat{\sigma}^2$$

Also is there a generalized answer to this question (not in linear regression specifically)?

Thank you.

This depends on the estimation procedure. It is generally not true that $$\hat{\sigma^2}=\hat{\sigma}^2$$. However, maximum likelihood estimates are an important exception.

• “This depends on the estimation procedure” means that it would be perfectly valid to take the variance estimate as the usual $\frac{1}{n-1}\sum(x_i-\bar{x})^2$ and the standard deviation estimate as $\sqrt{\frac{1}{n}\sum(x_i-\bar{x})^2}$. Then $\hat{\sigma^2}\ne \hat{\sigma}^2$.
– Dave
Commented Jan 3, 2020 at 12:06

This is not true if you want unbiased estimators.

For the model $$Y_i = \mu + \epsilon_i$$, the sample standard deviation $$s = \sqrt{\frac{\sum (y_i-\bar{y})^2}{n-1}}$$ (square root of the unbiased estimate of $$\sigma^2$$) is a biased estimator of $$\sigma$$. Namely, $$\mathbb{E}[s] = c_n \sigma$$ with $$c_n = \sqrt{\frac{2}{n-1}}\frac{\Gamma\bigl(\frac{n}{2}\bigr)}{\Gamma\bigl(\frac{n-1}{2}\bigr)}.$$ Thus, an unbiased estimate of $$\sigma$$ is $$\hat\sigma = \frac{s}{c_n}$$, and one does not have the equality $$(\hat\sigma)^2 = \widehat{\sigma^2}$$ when $$\widehat{\sigma^2} = s^2$$ is the well-known unbiased estimate of $$\sigma^2$$.

I don't know the result for the simple linear regression, but it should be similar.

If you want maximum likelihood estimators, then it is true that $$\widehat{f(\theta)} = f(\hat\theta)$$ for any monotonic function $$f$$.

• This doesn't (yet) answer the question about estimators of variance (although its thrust is clear). Could you provide a more explicit response?
– whuber
Commented Jan 3, 2020 at 15:17
• @whuber I edited. Is it better? Commented Jan 3, 2020 at 15:42