2
$\begingroup$

In this youtube video about bootstrap resampling, the creator states that when the number of bootstrap processes is small, the distribution of the parameter being estimated can no longer be thought to have a normal distribution (see from the 8:50 mark).

Since the parameter's distribution is not normal, the standard deviation can't be obtained via the standard formula:

enter image description here

The author thus states that one needs to use the CDF to calculate where the 68% confidence interval values are and obtain from there an estimation of the standard deviation.

My question is: where can I get a source for this workaround for obtaining the standard deviation when the number of times the bootstrap process is repeated is small?

I'm interested in this because I'm using bootstrap with replacement to estimate the uncertainty of a parameter and I can't possibly repeat it thousands of times (not even dozens actually) because it would be impossibly time consuming.

Improve this question
$\endgroup$
4
  • $\begingroup$ In many case the whole point of using the bootstrap is that the sampling distribution of the statistic is unknown, so in general I would not expect that distribution to be normal (Gaussian) even with a large number of bootstrap replications. $\endgroup$
    – Maarten Buis
    Commented Feb 5, 2014 at 16:28
  • $\begingroup$ @MaartenBuis so are you saying one should always use the CDF method to calculate the uncertainty? $\endgroup$
    – Gabriel
    Commented Feb 5, 2014 at 16:53
  • 1
    $\begingroup$ "always" is a big word, but looking at the percentiles (with possible adjustments) for computing confidence intervals fits more naturally within the logic of the bootstrap. $\endgroup$
    – Maarten Buis
    Commented Feb 5, 2014 at 17:17
  • $\begingroup$ from what I heard, isn't the bootstrap generally supposed to be done with replacement though? $\endgroup$
    – user38457
    Commented Feb 5, 2014 at 19:11

1 Answer 1

Reset to default
1
$\begingroup$

A few reactions that are too long for a comment:

Don't ask us for sources of what somebody claimed on youtube. Ask the creator of the video. The idea of using the normal quantiles to estimate the scale of the distribution obviously comes from robust statistics, and my vague recollection is that it may be due to Tukey. Note that with 20 bootstrap replicates, the 68% confidence interval is going to be far more lousy that the estimate of $s$ that you would get from the standard formula.

The formula for $s^2$ provides an unbiased estimator no matter whether the underlying distribution is normal or not (but it has to have the second moment, of course). What the normality affects is whether that $s^2$ (scaled by $\sigma^2$) will have a $\chi^2$ distribution (and hence the ratios with $s$ in the denominator will have a true $t$-distribution), but unless something really crazy is going on, the $t$-distributions are remarkably robust to non-normality.

To improve performance of the bootstrap with small number of replicates, one can consider balanced schemes that allows one to produce exactly the same estimates of variance that the complete perfect bootstrap would. Most people are just used to Monte Carlo versions of the bootstrap, so their hair may stand up on their heads when they would hear about deterministic bootstrap, but one should not forget that the distant goal is the complete bootstrap, and whatever gets us closer to it is helpful.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.