7
$\begingroup$

Maybe this question depends on the given data, but is there a "better" bootstrap method than the others? I'm simply using a one variable dataset (that consists of the differences between football scores (2 teams) over the past 15 weeks)..

First note the right skew of this data, I feel like this will take into consideration which bootstrap I would recommend as being "better" or most accurate for the representation of the data.

enter image description here

First here is the standard bootstrap interval

N <- 10^4
n <- length(Differences)
Differences.mean <- numeric(N)
for(i in 1:N)
{
x <- sample(Differences, n, replace = TRUE)
Differences.mean[i]<- mean(x)
}

lower = mean(Differences.mean)-1.96*sd(Differences.mean) #Lower CI
upper = mean(Differences.mean)+1.96*sd(Differences.mean) #Upper CI
= (8.875, 10.916)

 mean(Differences.mean)-m  #The bias is fairly small also
= -.0019

Here is a bootstrap percentile interval

 quantile(Differences.mean,c(.025,.975)
 = (8.893, 10.938)

Lastly here is the Bootstrap T interval

Tstar = numeric(N)
for(i in 1:N)
{
y =sample(Differences, size = n, replace = TRUE)
Tstar[i] = (mean(y)-m) / (sd(y)/sqrt(n))
}
q1 = quantile(Tstar,.025) #empirical quantiles for bootstrap t (lower)
q2 = quantile(Tstar,.975) #empirical quantiles for bootstrap t (upper)

mean(Differences)-(q2*sd(Differences/sqrt(n)))
mean(Differences)-(q1*sd(Differences/sqrt(n)))
= (8.925, 10.997)

In addition even the t confidence interval seems pretty accurate

 t.test(Differences, conf.level = .95, alternative = "two.sided")
 = (8.867, 10.928)

My conclusion would be to choose the bootstrap t interval because it reflects the right skew of the data, it is stretched further to the right than any of the others. My sample size is 224. I think sample size plays a huge role in my conclusion, but my initial question was "is there a better bootstrap method than the others?".. Maybe it really does depend on the data and sample size. Hopefully this isn't too broad.

Improve this question
$\endgroup$
8
  • $\begingroup$ Is this a homework problem? $\endgroup$
    – Jon
    Dec 21 '16 at 19:54
  • 1
    $\begingroup$ Here's a good paper to read, "Theoretical Comparison of Bootstrap Confidence Intervals": projecteuclid.org/download/pdf_1/euclid.aos/1176350933 $\endgroup$
    – Jon
    Dec 21 '16 at 19:55
  • 2
    $\begingroup$ It looks like you have some good ideas. Look at Tim Hesterberg's work on bootstrap t. I don't quite understand your graph because it does not show how far on the negative side the curve goes. I am a little surprised at how close the intervals are for all methods. I would have expected the standard bootstrap to do much worse. $\endgroup$
    – Michael R. Chernick
    Dec 21 '16 at 20:25
  • 1
    $\begingroup$ Did you consider BC and BCa? $\endgroup$
    – Michael R. Chernick
    Dec 21 '16 at 20:27
  • 1
    $\begingroup$ You could look at these books by authors Efron and Tibshirani 1993. Davison and Hinkley 1997 and Chernick 2007. My book was published by Wiley. Efron Chapman and Hall and Davison and Hinkley Cambridge University Press I think. $\endgroup$
    – Michael R. Chernick
    Dec 21 '16 at 20:38
1
$\begingroup$

As Michael Chernick notes, it would be useful to also look at the bias-corrected (BC) and bias-corrected and accelerated (BCa) bootstrap.

The BCa variant in particular attempts to deal with skewness in data, as you apparently have. DiCiccio & Efron (1996, Statistical Science) found that it performs well, as do Davison & Hinkley, Bootstrap Methods and their Applications (1997).

Why does my bootstrap interval have terrible coverage? is related, and I would especially recommend the article by Canto et al. (2006) that I cite there. And in the end, I agree that the answer is likely related to sample size, as well as your underlying distribution, and the pivotality or non of the statistic you are bootstrapping.

Improve this answer
$\endgroup$
1
  • $\begingroup$ It is very important to simulate the non-coverage probabilities on both sides of the confidence interval. When I did that for the log-normal distribution, all bootstrap intervals have terrible coverage, except for bootstrap t which I didn't try. It would also be worth using a standard bootstrap package to check your bootstrap t result. $\endgroup$
    – Frank Harrell
    Nov 7 '17 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.