# If bootstrap works for $\hat{\alpha}$ and for $\hat{\beta}$, does it work for $\widehat{\alpha\beta}$?

Let $$\textbf{X}_1, \dots, \textbf{X}_n$$ be an (iid) random sample. From this random sample, we compute $$\hat{\alpha}$$ (an estimation of a certain parameter $$\alpha$$). Let $$\textbf{Y}_1, \dots, \textbf{Y}_n$$ be an (iid) random sample. From this random sample, we compute $$\hat{\beta}$$ (an estimation of a certain parameter $$\beta$$).

I would like to obtain CI (confidence intervals) for parameter $${\alpha\beta}$$.

I utilize the basic bootstrap technique in order to obtain CI for $$\alpha$$ and $$\beta$$ separately. This involves random sampling, with replacement, from our random sample to generate multiple bootstrap samples, each matching the size of our original dataset. For each bootstrap sample, I calculate the estimate of the statistic. Subsequently, we determine the 95 -quantiles of the re-sampled statistics to derive the confidence intervals.

Say that we proved the asymptotical correctness of the bootstrap CI for $$\hat{\alpha}$$ and for $$\hat{\beta}$$.

Question: How can we estimate confidence intervals for parameter $${\alpha\beta}$$ if $$X$$-sequence and $$Y$$-sequence are independent? Can we do something if they are not?

• Are you not just calculating the product of the coefficients in each bootstrap iteration?
– Dave
Commented Feb 10 at 21:21
• Are X and Y different measures from the same population, eg, height and weight of respondents? Or are they the same measure from different populations? Or are they different measures from different populations? Do X and Y have a defined relationship? Commented Feb 11 at 23:21

This is more complicated than it sounds, so is a good question.

To start off, the bootstrap doesn't work for $$\hat\alpha^2$$ when $$\alpha=0$$, which is a classic example of bootstrap failure. The reason it doesn't work is that it relies on the delta-method, which doesn't exactly fail but does become much less helpful when the function you're computing has derivative zero at the true parameters (but not at the 'true' parameters in the bootstrap world)

(Here I mean the function $$(a,b)\mapsto ab$$)

So, this deserves simulation. Here are qqplots for the sampling distribution and four examples of the bootstrap distribution in samples of size 1000 from some Normal distributions

f <- function(m1, m2){
r <- replicate(10000,{
x <- rnorm(1000,m=m1)
y <- rnorm(1000,m=m2)
alphahat <- mean(x)
betahat <- mean(y)
alphahat*betahat - m1*m2
})
qqnorm(r, main=paste0("x mean is ",m1,"; y mean is ",m2))

for (i in 1:4){
x <- rnorm(1000, m=m1)
y <- rnorm(1000, m=m2)

s <- replicate(10000, {
xstar <- sample(x, replace=TRUE)
ystar <- sample(y, replace=TRUE)
mean(xstar)*mean(ystar) - mean(x)*mean(y)
})

a <- qqnorm(s, plot.it=FALSE)
}
}


At (0,0) the shape is qualitatively right, but the bootstrap distributions are further than you might expect from the sampling distribution. At (10,10) and (10,0) everything is fine: normality, good bootstrap approximation. At (0.1, 0) it's starting to go bad again.

### When does it work?

If $$\alpha$$ and $$\beta$$ are both not near zero then we have $$\sqrt{n}(\hat\alpha-\alpha)$$ and $$\sqrt{n}(\hat\beta-\beta)$$ both asymptotically Normal and the bootstrap is correct for each. The product function is differentiable at $$(\alpha,\beta)$$ with non-zero derivative, and everything is fine.

If $$\alpha=0$$ and $$\beta$$ is large, the well-behaved delta-method term $$(\hat\alpha-\alpha)\frac{\partial(\alpha\beta)}{\partial \alpha}=(\hat\alpha-\alpha)\beta$$ dominates the small badly-behaved term and everything is still fine (and similarly if $$\beta=0$$ and $$\alpha$$ is large)

But if $$\alpha=0$$ and $$\beta$$ is small or vice versa, you get breakdown in the bootstrap and no asymptotic Normality.

### Extra credit

Things get even worse with non-zero correlation. Here I take $$X\sim N(0,1)$$ and $$Y=X+N(0,1)$$

or with a qqplot of r vs s, where you can see the $$X^2$$ component of $$XY$$ behaving differently from the error component.

• This is an amazing answer Thomas. A lot of statisticians don’t realize what approximations are being made by the bootstrap, nor the need to compare the bootstrap distribution to the sampling distribution in a simulation. Would you mind commenting a bit more on the latter, i.e., why one should care when the bootstrap distribution departs from the sampling distribution? I’ve just studied this empirically, e.g., in a very unbalanced Y case with binary logistic regression, the bootstrap distribution looks bad and the bootstrap confidence interval coverage is bad. Commented Feb 11 at 14:29
• The idea of the bootstrap is that the (knowable) distribution of $\theta^*-\hat\theta$ is close to the (unknown) distribution of $\hat\theta-\theta_0$ and so can be used as an approximation to it. If they don't agree then either the bootstrap just isn't working or the sample size is too small. It's also a useful heuristic that if the large-sample bootstrap distribution isn't Normal, the bootstrap probably isn't working and you should simulate carefully. I I have a list of settings where the bootstrap doesn't work here: notstatschat.rbind.io/2017/02/01/when-the-bootstrap-doesnt-work Commented Feb 11 at 20:01
• Fantastically helpful. On the overfitting issue near the end of your blog, the regular bootstrap estimates the bias in measures of predictive performance that are caused by overfitting very well as long as you don't use a discontinuous score rules such as classification accuracy. The bootstrap breaks down (underestimates overfitting) as p gets > N, but will still report an awful amount of overfitting so you're cooked anyway. Commented Feb 11 at 20:27