1
$\begingroup$

I used boot function in R to do bootstrap for 40 times and used boot.ci to get the "normal" confidence interval. The following is my R code:
1. Define the statistic used in boot function uni_boot <-function(data,indices,vari){ d = data[indices,] unifit = coxph(as.formula(paste('Surv(time, status)~', vari)) ,data = d) # return hazard ratio summary(unifit)$coef[2] } 2.Bootstrap r1 <- boot(data = data, statistic = uni_boot, R = 40, vari = variable) r2 <- boot.ci(boot.out = r1, type = "norm") Then, I examine the following things
Result of r1 object:
original bias std. error t1* 1.053145 0.03274176 0.1714448 Result of r2 object
Intervals : Level Normal 95% ( 0.684, 1.356 ) Calculations and Intervals on Original Scale The bootstrap sample mean would be mean(r1$t) 1.085887 which is original plus the bias. However, when I examine the the formula used to calculate the bootstrap interval. It is original-bias-V^1/2*Z(1-alpha) or original-bias-V^1/2*Z(alpha).
I followed the formula and did the calculation:
1.053145-0.03274176+1.96*0.1714448 got me the upper bound 1.356 1.053145-0.03274176-1.96*0.1714448 got me the lower bound 0.684 My understanding of the confidence interval would be the bootstrap mean in the middle of the boot CI. However, the boot CI midpoint here turns to be 1.024. Sometimes, the boot CI from boot.ci wouldn't cover the bootstrap mean mean(r1$t).
Anything I understand wrongly?

$\endgroup$
2
  • $\begingroup$ I think the difference is that in one place you are subtracting the bias but in the other you are adding it. That does not seem correct to me. $\endgroup$
    – mdewey
    Oct 18, 2018 at 16:11
  • $\begingroup$ Thank you for replying me. I also got confused about this contradictory. Which way is the correct one? Theoretically, I should minus the bias. However, then the mean is different from mean(r1$t). In my opinion, r1$t is the values we got from 40 times bootstraps. $\endgroup$
    – purod
    Oct 18, 2018 at 16:33

1 Answer 1

3
$\begingroup$

Yes, the right way to correct for bias with bootstrapping is to subtract the bias from the value obtained on the original sample. When you get stuck thinking about bootstrapping, remember the guiding principle:

The population is to the sample as the sample is to the bootstrap samples.

Your value from the original sample is 1.053; you perform bootstrap resamples from the original sample and find a mean value of 1.086. The mean from the bootstrap samples was thus 0.033 higher than the value from the original sample: a bias of +0.033.

Applying the above principle, you use that bias value to estimate that the value from the original sample is 0.033 higher than the population value. So the bias-corrected estimate of the population value is 1.02.

The CI based on bootstrapping are supposed to represent the CI in the original population, while the values from the bootstrapped samples themselves are doubly biased from the population values (once from the original sample versus the population, and then again from the bootstrapped samples versus the original sample). With bias, you thus shouldn't be alarmed that the values obtained from the bootstrapped samples are beyond the final CI estimates for the population.

Issues of confidence intervals from bootstrapping can be even trickier; the "normal" confidence intervals assume symmetry about the (bias-corrected) mean, an assumption that might not best describe the situation in practice. With bias and asymmetry some other frequently used bootstrapping-based estimates of confidence intervals can be misleading. I struggled with these issues extensively until I forced myself to apply the above guiding principle systematically.

$\endgroup$
2
  • $\begingroup$ @Nicole please turn these comments into a new question. You can point to this question as the principle to apply, then in the new question ask how to apply the principle to your situation. In your new question please say more about the number of cases and events (if this is survival analysis) and the type of modeling (e.g., nature of variable selection, is there penalization as in ridge or LASSO, criteria for evaluating the model, what you mean by the "evaluation score"). Those details will be important for suggesting how to proceed in your case. $\endgroup$
    – EdM
    Oct 21, 2018 at 16:57
  • $\begingroup$ A new question about the application of the bootstrap's rule is posted. Evaluate the performance of a model with bootstrap $\endgroup$
    – purod
    Oct 22, 2018 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.