1
$\begingroup$

The ggplot2 package in R includes a dataset called diamonds. The dataset can be accessed by loading ggplot2 like this:

library(ggplot2)

I'm using the boot package to calculate a 95% confidence interval for the mean of the table variable. The table variable has 53,940 observations, and therefore when I tried to use 10,000 bootstrap replicates R crashed:

library(boot)

boot_diamonds_10000 <- boot(diamonds,function(data,indices) mean(data[indices,]$table), R=10000)

I then tried using 1000, 100 and 10 bootstrap replicates like as below. The 1000 and 100 replicates are still slow function calls, but 10 replicates is faster:

boot_diamonds_1000 <- boot(diamonds,function(data,indices) mean(data[indices,]$table), R=1000)

boot_diamonds_100 <- boot(diamonds,function(data,indices) mean(data[indices,]$table), R=100)

boot_diamonds_10 <- boot(diamonds,function(data,indices) mean(data[indices,]$table), R=10)

These all give pretty much the same 95% confidence intervals:

quantile(boot_diamonds_1000$t, c(0.025,0.975))

# 2.5%    97.5% 
# 57.43890 57.47682

quantile(boot_diamonds_100$t, c(0.025,0.975))

# 2.5%    97.5% 
# 57.43638 57.47438 

quantile(boot_diamonds_10$t, c(0.025,0.975))

# 2.5%    97.5% 
# 57.44636 57.46841

To avoid crashing R or waiting for slow functions calls, is it reasonable to use 10 bootstrap replicates when the sample size (53,940) is so high?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ No, 10 bootstrap replications is not enough. With that big sample you should consider using more powerful computer (more RAM). The other thing you could do is to write bootstrap function that saves only aggregated data and calculates things on-the-fly (instead of storing everything) to save RAM. $\endgroup$
    – Tim ♦
    Oct 2, 2015 at 8:11
  • $\begingroup$ @Tim the 95% confidence interval is the same regardless of whether I use 10000, 1000, 100 or 10 replicates. So why is 10 not enough? $\endgroup$
    – luciano
    Oct 2, 2015 at 9:55
  • 2
    $\begingroup$ No, you need a lot more then 10, regardless of sample size. However, With a larger sample size the central limit theorem really kicks in and your standard errors get super small (why all your confidence intervals look the same). So I would say instead, that when the sample size is large, you may get away without doing a bootstrap at all...of course that may not apply as well to more complex estimates. $\endgroup$
    – Zachary Blumenfeld
    Oct 2, 2015 at 9:58
  • $\begingroup$ @ZacharyBlumenfeld haven't you contradicted yourself there? $\endgroup$
    – luciano
    Oct 2, 2015 at 10:03
  • 2
    $\begingroup$ What I mean with the whole central limit thing is that parametric estimate for the standard error of the sample average $\frac{\hat \sigma^2}{N}$ will actually approximate really well here because you have so much data. Given that, you can cut down on computation time by not even using a bootstrap in the first place. If you don't believe me plot a density of the 1000 bootstrap sample over the theoretical distribution of the sample average using $\frac{\hat \sigma^2}{N}$, they will probably look very similar. $\endgroup$
    – Zachary Blumenfeld
    Oct 2, 2015 at 10:28

0

Reset to default