How should bootstrap bias correction be applied when combining multiple biased estimates?

Question

I am trying to use non-parametric bootstrap methods to get confidence intervals for a multi-step estimator that combines multiple variables:

$\hat\theta = \sqrt{\sum\limits_{i=1}^{n} {\left(\sum\limits_{t=1}^{m}|\overline{X_{i,t}} - \overline{Y_{i,t}}|\right)}^2}$

I find a bootstrapped distribution $\hat\theta^\ast_b$ by resampling each $X_{i,t}$ and $Y_{i,t}$ independently 10,000 times.

Since $|\overline{X_{i,t}} - \overline{Y_{i,t}}|$ is already a biased estimator, the bootstrap distributions at this stage (orange in the figure below) are already systematically positively biased relative to the estimated $|\overline{X_{i,t}} - \overline{Y_{i,t}}|$ (blue line in the figure below), especially when $|\overline{X_{i,t}} - \overline{Y_{i,t}}|$ is small:

The vector norm operation then compounds these biases. The result is that $\hat\theta$ (green line in the plot below) sometimes falls completely outside of the bootstrapped distribution $\hat\theta^\ast_b$ (red in the plot below).

I understand that BCa (bias corrected and accelerated) bootstrap is a preferred method for generating confidence intervals in the presence of bias and skew. However, this is problematic here, since $z_0 = -\inf$ when all $\hat\theta^\ast_b > \hat\theta$.

Thus my questions are:

Would it be valid to use bootstrap bias estimation to adjust for estimated bias by reporting $\tilde\theta$, where $\tilde\theta = 2\times{\hat\theta} - \bar{\hat\theta^\ast}$, and reporting percentile confidence intervals by re-centering $\hat\theta^\ast_b$ around $\tilde\theta$?

Should such bias correction be applied to each $|\overline{X_{i,t}} - \overline{Y_{i,t}}|$ before combining them?

If all I am interested in is relative values of $\hat\theta$ under different conditions, should I just report the means of the bootstrapped distributions, $\overline{\hat\theta^\ast}$ with percentile confidence intervals?

Is it even valid to use non-parametric bootstrap methods in this case?

Thanks very much for your consideration.

Answer 1

What you are plotting is the distribution of your statistic among the bootstrapped samples, the basis for the percentile bootstrap estimate of CI. Oddly enough, the percentile bootstrap method doesn't actually follow the bootstrap principle, which is that the relationship of your bootstrapped re-samples to your data sample represents the relationship of your data sample to the underlying population.

If you were following that principle, you'd be estimating the CI for the population by looking at the relationship between the values calculated from the bootstrapped samples and the value calculated on the full data sample. That's not what the percentile bootstrap does. As shown on this page, in this respect the percentile bootstrap is doubly biased in its mean values and, with a skewed distribution, reverses the direction of the skew.

The empirical or basic bootstrap, in contrast, follows the bootstrap principle. You examine the distribution of the differences of the statistic values among your bootstrapped samples from the value calculated on your data sample. That provides an estimate of the bias in your statistic, which you then use to correct both the mean value and the CI from your sample back to the estimated values in the population.

You can see that difference clearly on this page if you compare values of percentile and basic bootstrap CI for the necessarily biased plug-in estimate of the Shannon entropy. The entropy calculated on the data sample was 3.68, and bootstrapping gave an estimate of bias of -0.25. Thus the estimate for the population, corrected for bias, is 3.93. That's nicely within the CI calculated by the basic method (3.81, 4.05) but wildly beyond those calculated by the percentile method (3.31, 3.55), which didn't even include the original plug-in estimate of the entropy.

So I'd recommend starting with the empirical/basic bootstrap here. That provides the bias estimate needed to correct the statistic calculated on your data sample, and the CI, back to the underlying population. You have all the data you need for that, it's just based on the distribution of the differences of the values of the bootstrapped statistic values from the statistic calculated on the full data sample. As your distribution doesn't seem to have much skew, that should represent your situation pretty well, although I can't promise that your 95% CI will necessarily have 95% coverage. But that's always a problem with a non-pivotal statistic. See the references in this answer for further information.