# Sample standard deviation is a biased estimator: Details in calculating the bias of $s$

the last step is shown as:

$$\sigma\left(1-\sqrt\frac{2}{n-1}\frac{\Gamma\frac{n}{2}}{\Gamma\frac{n-1}{2}}\right) = \sigma\left(1-\sqrt\frac{2}{n-1}\frac{((n/2)-1)!}{((n-1)/2-1)!}\right)$$

How is this equal to $$\frac{\sigma}{4n}$$?

• The claim in that answer is not that they are equal, but that they are asymptotically equivalent. – angryavian Oct 31 '20 at 3:43
• @angryavian could you show how they are asymptotically equivalent? I have shown my working above but could not get very far with this – Darya Oct 31 '20 at 3:59
• Stirling's formula will make short work of this. – whuber Oct 31 '20 at 15:44
• @whuber When I tried Stirling, I was only able to show convergence to zero, but nothing about the rate of convergence. – angryavian Oct 31 '20 at 17:47
• @Angry You didn't carry the asymptotic expansion out far enough, then. I have added an answer showing the details. – whuber Nov 2 '20 at 13:49

Making the substitution $$x = \frac{n}{2}-1$$, you essentially want to control $$1 - \frac{\Gamma(x+1)}{\Gamma(x+\frac{1}{2}) \sqrt{x + \frac{1}{2}}}$$ as $$x \to \infty$$.

Gautschi's inequality (applied with $$s=\frac{1}{2}$$) implies $$1 - \sqrt{\frac{x+1}{x+\frac{1}{2}}} <1 - \frac{\Gamma(x+1)}{\Gamma(x+\frac{1}{2}) \sqrt{x + \frac{1}{2}}} < 1 - \sqrt{\frac{x}{x+\frac{1}{2}}}$$ The upper and lower bounds can be rearranged as $$\left|1 - \frac{\Gamma(x+1)}{\Gamma(x+\frac{1}{2}) \sqrt{x + \frac{1}{2}}}\right| < \frac{1}{2x+1} \cdot \frac{1}{1 + \sqrt{1 - \frac{1}{2x+1}}} \approx \frac{1}{2(2x+1)}.$$ Plugging in $$x=\frac{n}{2}-1$$ gives a bound of $$\frac{1}{2(n-1)}$$. This is weaker than the author's claim of asymptotic equivalence with $$\frac{1}{4n}$$, but at least it is of the same order.

When $$x=\frac{n}{2}-1$$ you have $$x+1 = \frac{n}{2}$$ and $$x + \frac{1}{2} = \frac{n}{2} - 1 + \frac{1}{2} = \frac{n}{2} - \frac{1}{2} = \frac{n-1}{2}$$. So $$\frac{\Gamma(x+1)}{\Gamma(x+\frac{1}{2}) \sqrt{x + \frac{1}{2}}} = \frac{\Gamma(n/2)}{\Gamma((n-1)/2) \sqrt{(n-1)/2}}$$.

• In the 1st line, how do we get the additional sqrt(x+1/2) in the denominator? I just get Gamma(x+1)/Gamma(x+(1/2)) when I substitute x=(n/2) -1 – Darya Oct 31 '20 at 8:08
• Elegant, Nice (+1) – BruceET Oct 31 '20 at 16:38
• @Darya It comes from the $\sqrt{\frac{2}{n-1}}$ term in your original expression. – angryavian Oct 31 '20 at 17:44
• @angryavian (i) when I substitute n=2x-1 into sqrt(2/(n-1), I get sqrt(1/x). (ii) could you also please show how we get 1/4n from the last line? When I plug in x=(n/2)-1 into the extreme lower right term i.e. 1/(2x+1), I get n + (1/2). (My maths level is not advanced so apologies if this may be a bit basic) – Darya Nov 1 '20 at 7:02
• @Darya Updated my answer to address your first question. For your second question, I made an error and have corrected it. – angryavian Nov 1 '20 at 19:39

The default approach for analyzing expressions involving Gamma functions is Stirling's asymptotic expansion

$$\log \Gamma(z) = \frac{1}{2}\log(2\pi) + \left(z - \frac{1}{2}\right)\log(z) - z + \frac{1}{12z} - \frac{1}{360z^3} + \cdots$$

(and usually you don't even need that final term). This gives us some intuition about how $$\Gamma$$ behaves and a basis for working out approximate values. Although this series is not a topic in an elementary Calculus course, the following analysis based on it uses only the most elementary facts about power series expansions (Taylor series) and so is something anybody can learn to do.

Calling this an "asymptotic expansion" means that when you fix the number of terms you use, then eventually -- for any $$z$$ with a suitably large size -- the approximation becomes extremely good. (This is in contrast to a power series in $$1/z,$$ which for a fixed $$z$$ ought to get better and better as more terms in the series are included.)

This expansion is so good that it is used in almost all computing software to compute values of $$\Gamma.$$ For example, here is a comparison of computations of $$\Gamma(z)$$ for $$z=2,4,6,8:$$

                       2         4           6    8
Stirling       0.9999787 5.9999956 119.9999880 5040
R              1.0000000 6.0000000 120.0000000 5040
Relative error 0.9999787 0.9999993   0.9999999    1


"R" refers to the value returned by the gamma function in the R software. Look how close the approximation is even for $$z=2!$$

To apply this expansion, take the logarithm of the expression you wish to analyze, focusing on product terms that will simplify:

$$w=\log\left(\sqrt\frac{2}{n-1}\frac{\Gamma\left(\frac{n}{2}\right)}{\Gamma\left(\frac{n-1}{2}\right)}\right) = \frac{1}{2}\left(\log 2 - \log(n-1)\right) + \log \Gamma\left(\frac{n}{2}\right) - \log\Gamma\left(\frac{n-1}{2}\right)$$

(You can find many accounts of Stirling's approximation in terms of $$\Gamma$$ itself. These are less useful than the log Gamma series because working with the logs amounts to doing some algebraic addition and subtraction, which is relatively simple.)

Now just substitute a suitable number of terms of the asymptotic series for the $$\log \Gamma$$ components. Sometimes you can get away with carrying the series out to the $$-z$$ term, but often there is so much cancellation that you need the $$1/(12z)$$ term to learn anything useful. Focusing on the log Gamma functions in the foregoing, it is clear the constant terms $$(1/2)\log(2\pi)$$ will cancel. Write down the rest: \begin{aligned} \log \Gamma\left(\frac{n}{2}\right) - \log\Gamma\left(\frac{n-1}{2}\right)&\approx \left(\frac{n}{2} - \frac{1}{2}\right)\log\left(\frac{n}{2}\right) - \frac{n}{2} + \frac{1}{12\left(\frac{n}{2}\right)}\\ &- \left[\left(\frac{n-1}{2} - \frac{1}{2}\right)\log\left(\frac{n-1}{2}\right) - \frac{n-1}{2} + \frac{1}{12\left(\frac{n-1}{2}\right)}\right] \end{aligned}

Now we add the $$\frac{1}{2}\left(\log 2 - \log(n-1)\right)$$ terms back in and simplify as much as we can, freely using approximations for large $$n$$ (that is, small $$\epsilon=1/(n-1)$$) using the power series $$\log(1 + \epsilon) = \epsilon - \epsilon^2/2 + O(\epsilon^3):$$

\begin{aligned} w &\approx \frac{n-1}{2}\log\left(\frac{n}{n-1}\right) - \frac{1}{2} - \frac{1}{6n(n-1)} \\ &= \frac{n-1}{2}\left(\frac{1}{n-1} - \frac{1}{2(n-1)^2} + O((n-1)^{-3})\right) - \frac{1}{2} - \frac{1}{6n(n-1)} \\ &= -\frac{1}{4(n-1)} + O(n^{-2}). \end{aligned}

That wasn't particularly painful. The $$O(n^{-p})$$ analysis of $$\log$$ and the extensive cancellation are characteristic of calculations with Gamma functions.

Returning to the original question, it concerns an expression we may readily work out using the Taylor series $$\exp(\epsilon) = 1 + \epsilon + O(\epsilon^2):$$

$$\sigma(1 - \exp(w)) = \sigma\left(1 - (1 - \frac{1}{4(n-1)} + O\left(n^{-2}\right)\right) = \frac{\sigma}{4(n-1)} + O(n^{-2}).$$

This agrees with the equality in the question (because $$1/(n-1)=1/n$$ modulo $$O(n^{-2})$$).

It should now be clear that by taking more terms in the asymptotic expansion and in the Taylor series of $$\log$$ and $$\exp$$ you can obtain a higher-order approximation of the form $$\sigma((1/4)(n-1)^{-1} + a_2(n-1)^{-2} + \cdots + a_p(n-1)^{-p}.)$$ Just don't go overboard with this: for small $$n,$$ using these additional terms will make the approximation worse; the improvement is only for extremely large values of $$n.$$

• +1 Thanks for writing this out! Just curious, how difficult is it to derive the first few terms of Stirling's asymptotic expansion (beyond the commonly used main term)? – angryavian Nov 8 '20 at 18:07
• @angryavian Usually once you obtain the coefficient of $1/z$ you're off and running. The accounts in Whitaker & Watson and in Ahlfors analyze an expression obtained by Binet, $$\log\Gamma(z) = \left(z-\frac{1}{2}\right)\log z - z + \frac{1}{2}\log(2\pi) + 2\int_0^\infty \frac{\arctan(t/z)}{e^{2\pi t}-1}\,\mathrm{d}t.$$ – whuber Nov 8 '20 at 18:21

Comment: Using R to visualize the speed of convergence.

n = seq(5,300,by=5)
c = 4*n*(1-sqrt(2/(n-1))*gamma(n/2)/gamma((n-1)/2))
plot(n,c); abline(h=1, col="green2", lwd=2) • You will learn a great deal by instead plotting $1/(c-1)$ against $n.$ You can also compute values of c for much larger arguments n by using lgamma instead of gamma for the computation. – whuber Oct 31 '20 at 17:27
• @whuber. Thanks. Got a nice linear plot. I knew about lgamma but though $n=300$ was large enough to give the idea. // Sorry for the delay, pre-election activities. – BruceET Nov 2 '20 at 2:42