# The distribution of an estimator

I was reading the Wikipedia article on Robust Statistics: http://en.wikipedia.org/wiki/Robust_statistics#Properties_of_M-estimators

What is meant by the distribution of an estimator, as used here,

It can be shown that M-estimators are asymptotically normally distributed, so that as long as their standard errors can be computed, an approximate approach to inference is available. Since M-estimators are normal only asymptotically, for small sample sizes it might be appropriate to use an alternative approach to inference, such as the bootstrap.

-
Since the value of an estimator is a function of the observed data, and the observed data is assumed to follow some distribution, it follows that the value of the estimator also follows some distribution. As the sample size gets large, estimators in the class of "M-estimators" become distributed according to a Normal (also known as Gaussian) distribution. –  jbowman Mar 20 '12 at 19:51
@jbownam: that looks like an answer, especially the first two lines. –  Henry Mar 20 '12 at 20:07
As a bootstrapper I do not object to the suggestion to use bootstrap in small samples. The bootstrap often works well in small samples. The reason small samples are mentioned is because you can't rely on the asymptotic normality when the sample size is small. The bootstrap will approximate the sampling distribution for the m-estimator at the given fixed sample size. It is an approximate method and we say the bootstrap works if we can show consistency theoretically. –  Michael Chernick May 6 '12 at 16:48
Requiring consistency does not rule out the possibility that a consistent bootstrap procedure provides a good approximate result in small samples. Like almost any statistical technique the bootstrap will not perform well in very small samples. –  Michael Chernick May 6 '12 at 16:48
@jbowman, following Henry, perhaps you should convert your comment to an answer. Then the OP can accept it, as it does answer the question & she has acknowledged that (& I can upvote it ;-) ). –  gung May 6 '12 at 19:37
show 3 more comments