estimator of sample standard deviation

Question

In this post, Why is sample standard deviation a biased estimator of $\sigma$?

my understanding is that the author takes the positive square root i.e $E(+\sqrt(S^2))$ and not $E(-\sqrt(S^2))$.

If we were to take the negative square root value, the final expression would be negative, and 1- that value would be >1, in which case E(S) would over-estimate $\sigma$. Would this be possible?

However, from the plot, it seems that the bias seems to vary from 0 to 1, which suggests that the bias always an underestimate of $\sigma$

My query is whether there was any reason for taking the positive square root of $S^2$, and whether the bias of the sample standard deviation can theoretically represent either an underestimate or overestimate of $\sigma$

Answer 1

The standard deviation must be positive - should you take the negative square root of the variance, it will be negative and thus would not make sense.

Now you certainly know (at least from the post you mentioned) that an unbiased estimate of the population variance, computed from a set of observations from this population $x_1, \dots, x_n$, is $S^2=\frac{1}{n-1}\sum_{i=1}^n (x_i-\bar x)^2$. Unbiasedness means $E[S^2] = \sigma^2$. Both are positive numbers, so you can take the (positive) square root of either side and the equality still holds,

\begin{align*} \sqrt{E[S^2]} &= \sqrt{\sigma^2}\\ &\Leftrightarrow \\ \sqrt{E[S^2]} &= \sigma \end{align*}

Where $\sigma$ is the standard error. Now, by Jensen's inequality, since the square root is a concave function, you have

$$ \sigma = \sqrt{E[S^2]} \geq E[\sqrt{S^2}] $$

so you actually systematically under-estimate the standard error.

[Edit] I just saw that the argument was given in another answer from the post you linked. The original poster defined the bias of an estimator $s$ of $\sigma$ as $\sigma - E[s]$ rather than $E[s] - \sigma$.