We have a iid sequence of random variables $X_1, X_2, \dots, X_n$, where $E(X_i) = \mu$ and $var(X_i) = \sigma^2$. The sample mean $\bar{X}$ converges to $\mu$ at rate $\sqrt{n}$ thanks to the LLN.

If we have a continuous function $f()$, the continuous mapping theorem assures that $f(\bar X)$ converges to $f(\mu)$.

My question is the following: at what rate does $f(\bar X)$ converge to $f(\mu)$?

Asymptotically I would say $\sqrt{n}$, given that $f()$ is continuous and hence locally linear. But can we have convergence rates very different from $\sqrt{n}$ in small samples?


Whenever $f$ is differentiable at $\mu$, the theory is nice: $f(\bar{X})$ will converge at rate $\sqrt{n}$. To get the exact rate of convergence (including constants) you can use the delta method (http://en.wikipedia.org/wiki/Delta_method).

If $f$ is not differentiable, then weird stuff can happen and $\sqrt{n}$ convergence is not guaranteed. In most applications, thankfully, we don't need to worry about this.

  • $\begingroup$ Thanks Stefan. $f$ is differentiable in my case, so you are right certainly right about what happens when the sample size $n$ is large enough, thanks to the delta method. But my doubt is whether $E((f(\bar{X})-f(\mu))^2) = O(n^{-1})$ also for small $n$. I was doing some simulation that seem to suggest that in some cases it takes a while to the asymptotic rate to "kick-in" but this might be due to programming or numerical errors so I'm still in doubt. $\endgroup$ – Matteo Fasiolo Jun 5 '13 at 21:52
  • $\begingroup$ Well, the statement $g(n) = O(n^{-1})$ means that $ng(n)$ is bounded as $n \rightarrow \infty$. It doesn't really make sense to ask whether $g(n)$ is $O(n^{-1})$ for small $n$. That's one of the major downsides of using asymptotics: There are in general no guarantees that the results will hold in the range you're interested in. (That being said... In general, the delta-method asymptotics will kick in slower when $f''$ is large. To see this, you can look at the higher-order terms in the delta method expansion.) $\endgroup$ – Stefan Wager Jun 5 '13 at 22:03
  • $\begingroup$ "To see this, you can look at the higher-order terms in the delta method expansion" that's exactly what I'm going to do tomorrow morning (I don't want to have nightmares tonight :) ). I got interested in small samples because in my application it looks like the asymptotic rate kicks in really slowly, so that (having finite computational power) small sample behaviour becomes important. $\endgroup$ – Matteo Fasiolo Jun 5 '13 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.