# Convergence rate of a non-linear function of the sample mean

We have a iid sequence of random variables $X_1, X_2, \dots, X_n$, where $E(X_i) = \mu$ and $var(X_i) = \sigma^2$. The sample mean $\bar{X}$ converges to $\mu$ at rate $\sqrt{n}$ thanks to the LLN.

If we have a continuous function $f()$, the continuous mapping theorem assures that $f(\bar X)$ converges to $f(\mu)$.

My question is the following: at what rate does $f(\bar X)$ converge to $f(\mu)$?

Asymptotically I would say $\sqrt{n}$, given that $f()$ is continuous and hence locally linear. But can we have convergence rates very different from $\sqrt{n}$ in small samples?

Whenever $f$ is differentiable at $\mu$, the theory is nice: $f(\bar{X})$ will converge at rate $\sqrt{n}$. To get the exact rate of convergence (including constants) you can use the delta method (http://en.wikipedia.org/wiki/Delta_method).
If $f$ is not differentiable, then weird stuff can happen and $\sqrt{n}$ convergence is not guaranteed. In most applications, thankfully, we don't need to worry about this.
• Thanks Stefan. $f$ is differentiable in my case, so you are right certainly right about what happens when the sample size $n$ is large enough, thanks to the delta method. But my doubt is whether $E((f(\bar{X})-f(\mu))^2) = O(n^{-1})$ also for small $n$. I was doing some simulation that seem to suggest that in some cases it takes a while to the asymptotic rate to "kick-in" but this might be due to programming or numerical errors so I'm still in doubt. Jun 5, 2013 at 21:52
• Well, the statement $g(n) = O(n^{-1})$ means that $ng(n)$ is bounded as $n \rightarrow \infty$. It doesn't really make sense to ask whether $g(n)$ is $O(n^{-1})$ for small $n$. That's one of the major downsides of using asymptotics: There are in general no guarantees that the results will hold in the range you're interested in. (That being said... In general, the delta-method asymptotics will kick in slower when $f''$ is large. To see this, you can look at the higher-order terms in the delta method expansion.) Jun 5, 2013 at 22:03