At a recent conference somebody claimed that the size of the bootstrap replications should always be 999 rather than 1000. Which argument supports this claim?
$\begingroup$
$\endgroup$
5
-
3$\begingroup$ I personally prefer using integer powers of two - being a computer scientist, they are what I consider to be "round numbers". More seriously, halving and doubling seem to be less of a blunt instrument than multiplying and dividing by ten. 5 or 10 seems to be an anthropocentric decimal number perspective. As they will be implemented by a digital computer using binary representations, I would be mildly surprising if there was a good reason for this (but you live and learn ;o). Some "simple" decimals don't have finite binary representations so what is simply to us may not be for the computer. $\endgroup$– Dikran MarsupialCommented Sep 16, 2022 at 16:41
-
6$\begingroup$ It really shouldn't matter. $\endgroup$– AdamOCommented Sep 16, 2022 at 17:38
-
1$\begingroup$ If you use $1000$ and your statistic is always an integer, then a bootstrap average will have no more than three decimal places. I see nothing wrong with that. Perhaps the risk is that it will end $\ldots50000$ raising a rounding question (though this is an argument against using any even number) $\endgroup$– HenryCommented Sep 16, 2022 at 19:00
-
$\begingroup$ @Tim - I had assumed (possibly wrongly) that $1000$ was the number of replications rather than the samples size with replacement $\endgroup$– HenryCommented Sep 16, 2022 at 19:03
-
3$\begingroup$ It would be fair to characterize this rule as numerology. One byproduct of following it would be that you almost always obtain p-values (and other proportions) that can be expanded to arbitrary precision, usually without any visual evidence of repetition, suggesting to the unwary that you have achieved far greater precision in your results than permitted by the number of replications. $\endgroup$– whuber ♦Commented Sep 16, 2022 at 20:24
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
I believe that the comment is in reference to the number of bootstrap samples rather than to the size of each sample. It might simplify estimates of some common percentiles from bootstrapped samples.
Davison and Hinkley argue on pages 18-19 that
if $X_1,...X_N$ are independently distributed with CDF $K$ and if $X_{(j)}$ denotes the $j$th ordered value, then $$\text{E}(X_{(j)}) = K^{-1}\left(\frac{j}{N+1}\right) .$$ This implies that a sensible estimate of $K^{-1}(p)$ is $X_{((N+1)p)}$, assuming that $(N+1)p$ is an integer.
So if you want to estimate the 2.5th and 97.5th quantiles to get a 95% confidence interval for some value, take 999 bootstrap samples, put the values in order, and select the 25th and the 975th.
I suppose that avoids choosing among the multiple ways of estimating quantiles.
-
3$\begingroup$ +1 That would explain the "999" recommendation but it doesn't help with "not a multiple of 5 or 10." That remains a mystery. $\endgroup$– whuber ♦Commented Sep 16, 2022 at 21:37
-
3$\begingroup$ @whuber it's hard to have the product $(N+1)p$ to be an integer for $N$ a multiple of 10 and common choices of $p$ (0.01, 0.025, 0.25). I won't press this mysterious numerology beyond that. $\endgroup$– EdMCommented Sep 16, 2022 at 21:59