2
$\begingroup$

As central limit theorem suggests, sampling distribution is approaching normal on the large sample sizes regardless of the initial distribution of the variable.

And it's always been true for me until I stumbled on this one.

I have a sample of 50K observation. I want to bootstrap a confidence interval around the mean. I take a sample of size 20K with replacement, calculate its mean and repeat it 10,000 times. Then I plot a histogram of it and what I expect to see is something like normal distribution (as always). However, what I see is this: enter image description here

Then I noticed that there were 3 huge outliers. Once I filtered them out, the sampling distribution became normal as expected: enter image description here

Now the questions: how come that initial sampling distribution did not have approximately normal shape (1) and, as logic suggests, does that mean that bootstrapping is fragile to outliers even with such a large sample sizes and number of repetitions 10,000 and even 100,000 times (2)?

$\endgroup$
3
$\begingroup$

It doesn't matter how large a sample size you choose, there's always distributions for which that sample size is not sufficient to make sample means look close to normal, even though the CLT holds for that distribution.

See the example here, where huge sample sizes are not sufficient.

$\endgroup$
  • $\begingroup$ What does it mean than that CLT holds true for those distributions? I thought CLT implies that sampling distribution is approaching normal. If this property is not true, than what properties are? $\endgroup$ – Alexander Dyachenko Nov 27 '19 at 13:17
  • 1
    $\begingroup$ The sampling distribution of the standardized mean is approaching the normal. As $n\to\infty$ its cdf will go to that of a standard normal. That property is definitely true - provably so. It's just that the sample size is nowhere near large enough for it to look close to normal yet. Indeed even a sample size of ten million still isn't remotely close enough. $\endgroup$ – Glen_b -Reinstate Monica Nov 27 '19 at 15:44
  • $\begingroup$ Thanks! Am I correct thinking that the best (and possibly only) way to deal with such distributions in the real world is to bound it (i.e. using arbitrary number of IQRs)? $\endgroup$ – Alexander Dyachenko Nov 28 '19 at 8:47
  • $\begingroup$ ...or use outlier-robust statistics like median? $\endgroup$ – Alexander Dyachenko Nov 28 '19 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.