2
$\begingroup$

In many statistics courses, bootstrapping (and other random sampling with replacement methods) are suggested as ways to improve the confidence level in a statistic and improve our inference. Some even say it is a "powerful" method.

However it seems intuitively incorrect...

Say we this is our population (N = 15) : 1 - 3 - 3 - 4 - 5 - 1 - 2 - 3 - 4 - 1 - 2 - 2- 3 - 4 - 9

and this is our sample (n = 5 ) : 1 - 4 - 4 - 2 - 9

if we use the bootstrap method the 9 value has a 1/5 chance of being randomly selected each time where as in our population the 9 value only represents 1/15 of all values!

Bootstrapping can make us believe that the values we have in our sample are more frequent than they really are. Therefore it feels like there is a huge bias... Or am I missing something?

I have been looking for discussion/publications on this but I haven't found any, there seems to be a consensus on the fact that it's a powerful method, but I can't help but feel like this is a biased method that will make us overconfident of our sample.

This seems so obvious and simple that I can't imagine that all the statisticians never thought of that, so I'm guessing I'm just missing something quite elemental...

Improve this question
$\endgroup$
2
  • $\begingroup$ Good question, but perhaps previously addressed here stats.stackexchange.com/questions/112147/… . $\endgroup$
    – AJKOER
    Jul 1 '20 at 15:01
  • $\begingroup$ Note, one of the cited references does suggest n>20, to avoid poor performance. $\endgroup$
    – AJKOER
    Jul 1 '20 at 15:09
2
$\begingroup$

My answer is for small samples of under 20, I agree with the recommended advice of not employing a bootstrap approach. This likely due to the fact that bootstrap does not work well if rare events are missing from the empirical distribution sample (per comment here).

Perhaps better is to fit the data to a selected distribution based on knowledge of the likely mechanics underlying the data generation process.

You may wish to make several suggested parent distribution for the fitting exercise. Then, graphically and statistically, assess the goodness-of-fit taking into account the particular application intended (for example, extreme value estimation, you want a particularly good fit in the tail region).

Then, proceed to use Monte Carlo techniques to simulate data points.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.