Why is sample standard deviation a biased estimator of $\sigma$? According to the Wikipedia article on unbiased estimation of standard deviation the sample SD 
$$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$
is a biased estimator of the SD of the population. It states that $E(\sqrt{s^2}) \neq \sqrt{E(s^2)}$. 
NB. Random variables are independent and each $x_{i} \sim N(\mu,\sigma^{2})$ 
My question is two-fold:


*

*What is the proof of the biasedness?

*How does one compute the expectation of the sample standard deviation 


My knowledge of maths/stats is only intermediate.
 A: @NRH's answer to this question gives a nice, simple proof of the biasedness of the sample standard deviation. Here I will explicitly calculate the expectation of the sample standard deviation (the original poster's second question) from a normally distributed sample, at which point the bias is clear. 
The unbiased sample variance of a set of points $x_1, ..., x_n$ is 
$$ s^{2} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \overline{x})^2 $$ 
If the $x_i$'s are normally distributed, it is a fact that 
$$ \frac{(n-1)s^2}{\sigma^2} \sim \chi^{2}_{n-1} $$ 
where $\sigma^2$ is the true variance. The $\chi^2_{k}$ distribution has probability density 
$$ p(x) = \frac{(1/2)^{k/2}}{\Gamma(k/2)} x^{k/2 - 1}e^{-x/2} $$ 
using this we can derive the expected value of $s$; 
$$ \begin{align} E(s) &= \sqrt{\frac{\sigma^2}{n-1}} E \left( \sqrt{\frac{s^2(n-1)}{\sigma^2}} \right) \\
&= \sqrt{\frac{\sigma^2}{n-1}}
\int_{0}^{\infty}
\sqrt{x} \frac{(1/2)^{(n-1)/2}}{\Gamma((n-1)/2)} x^{((n-1)/2) - 1}e^{-x/2}  \ dx \end{align} $$ 
which follows from the definition of expected value and  fact that $ \sqrt{\frac{s^2(n-1)}{\sigma^2}}$ is the square root of a $\chi^2$ distributed variable. The trick now is to rearrange terms so that the integrand becomes another $\chi^2$ density: 
$$ \begin{align} E(s) &= \sqrt{\frac{\sigma^2}{n-1}}
\int_{0}^{\infty}
\frac{(1/2)^{(n-1)/2}}{\Gamma(\frac{n-1}{2})} x^{(n/2) - 1}e^{-x/2}  \ dx \\
&= \sqrt{\frac{\sigma^2}{n-1}} \cdot
\frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) }
\int_{0}^{\infty}
\frac{(1/2)^{(n-1)/2}}{\Gamma(n/2)} x^{(n/2) - 1}e^{-x/2} \ dx \\
&= \sqrt{\frac{\sigma^2}{n-1}} \cdot
\frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \cdot
\frac{ (1/2)^{(n-1)/2} }{ (1/2)^{n/2} }
\underbrace{
\int_{0}^{\infty}
\frac{(1/2)^{n/2}}{\Gamma(n/2)} x^{(n/2) - 1}e^{-x/2} \ dx}_{\chi^2_n \ {\rm density} }
\end{align} 
$$
now we know the integrand the last line is equal to 1, since it is a $\chi^2_{n}$ density. Simplifying constants a bit gives  
$$ E(s)
= \sigma \cdot \sqrt{ \frac{2}{n-1} }  \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } $$ 
Therefore the bias of $s$ is 
$$ \sigma - E(s) = \sigma \bigg(1 - \sqrt{ \frac{2}{n-1} }  \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \bigg) \sim \frac{\sigma}{4 n} \>$$ 
as $n \to \infty$.
It's not hard to see that this bias is not 0 for any finite $n$, thus proving the sample standard deviation is biased. Below the bias is plot as a function of $n$ for $\sigma=1$ in red along with $1/4n$ in blue: 

A: You don't need normality. All you need is that 
$$s^2 = \frac{1}{n-1} \sum_{i=1}^n(x_i - \bar{x})^2$$
is an unbiased estimator of the variance $\sigma^2$. Then use that the square root function is strictly concave such that (by a strong form of Jensen's inequality)
$$E(\sqrt{s^2}) < \sqrt{E(s^2)} = \sigma$$
unless the distribution of $s^2$ is degenerate at $\sigma^2$. 
A: This is a more general result without assuming of Normal distribution. The proof goes along the lines of this paper by David E. Giles.
First, we consider Taylor's expanding $g(x) = \sqrt{x}$ about $x=\sigma^2$,
we have
$$
g(x) = \sigma + \frac{1}{2 \sigma}(x-\sigma^2) -  \frac{1}{8 \sigma^3}(x-\sigma^2)^2 + R(x),
$$
where $R(x) =- \left(\frac{1}{8 \tilde \sigma^3} - \frac{1}{8 \sigma^3}\right)(x-\sigma^2)^2$ for some  $\tilde \sigma$ between $\sqrt{x}$ and $\sigma$.
Let $\kappa = E(X - \mu)^4 / \sigma^4$ be the kurtosis.
It could be shown that $E\left[\sqrt{n}(S_n^2 - \sigma^2)\right]^2 \rightarrow \sigma^4(\kappa-1)$ and $n ER(S_n^2) \rightarrow 0$ (and the proofs are beyond the discussion of this thread. See for example  CLT that states that $\sqrt{n}(S_n^2 - \sigma^2)$  converges to $N(0, \sigma^4(\kappa-1))$).
Thus,
$$
E(S_n) = Eg(S_n^2) =\sigma + \frac{1}{2 \sigma} E(S_n^2 - \sigma^2) - \frac{1}{8\sigma^3} E(S_n^2 - \sigma^2)^2 + o(n^{-1}).
$$
$$
 = \sigma - \frac{\sigma}{8}\left[ \frac{\kappa - 1}{n}\right] + o(n^{-1}).
$$
For normal distribution, setting $\kappa = 3$ gives the first order bias $-\frac{\sigma}{4n}$ as shown above.
A: Complementing NRH's answer, if someone is teaching this to a group of students who haven't studied Jensen's inequality yet, one way to go is to define the sample standard deviation
$$
  S_n = \sqrt{\sum_{i=1}^n\frac{(X_i-\bar{X}_n)^2}{n-1}} , 
$$
suppose that $S_n$ is non degenerate (therefore, $\mathrm{Var}[S_n]\ne0$), and notice the equivalences
$$
  0 < \mathrm{Var}[S_n] = \mathrm{E}[S_n^2] - \mathrm{E}^2[S_n]
    \;\;\Leftrightarrow\;\; \mathrm{E}^2[S_n] < \mathrm{E}[S_n^2]
    \;\;\Leftrightarrow\;\; \mathrm{E}[S_n] < \sqrt{\mathrm{E}[S_n^2]} =\sigma.
$$
