What's the point of reporting bootstrap bias?

Question

Suppose the data $X = (X_1,X_2,\cdots,X_n)$ is a vector of iid observations $X_i$ where each $X_i$ has marginal distribution $F(\theta)$. Suppose we observe $x = (x_1,\cdots,x_n)$ and $\hat \theta = \hat \theta(X,F)$ and $t = t(x)$ are estimators and the estimate respectively. Define $\mathrm{Bias}(\hat \theta) := E(\hat \theta) - \theta$.

1. It seems to me that R bootstrap routine boot returns $\overline{t} - t$ as the estimate for the bias. Is this true?
2. If this is true, due to the law of large numbers, it seems to me that $\overline{t} - t(x)$ is an estimate of $E(t) - t(x)$, not $E(\hat \theta) - \theta$ which is the $\mathrm{Bias}(\hat \theta)$. Is this true or false?
3. If it is false, how can we see that $\overline{t} - t$ indeed estimate $\mathrm{Bias}(\hat \theta)$?

Answer 1

The bootstrap bias estimate is an estimate of $E(\hat \theta_n) - \theta$, where $\theta$ is some function of the population and $\hat \theta_n$ is that function evaluated in your sample of size $n$. It estimates the bias in approximating $\theta$ with $\hat \theta_n$, for which you only have $n$ observations. A well-known example of this type of bias is the variance estimate $\frac{\sum (x_i - \bar x)}{n}$, which has expectation $\frac{n-1}{n}\sigma^2$. This is not the same as sampling bias, which has to do with how the sample was gathered (let's say you mistakenly sampled 99 women and one man when this ratio usually should be closer to 50-50). Sampling bias cannot be estimated by this method.

How this works:

If you had the population at hand you could calculate the true $\theta$ directly and draw several samples of size $n$ to estimate $\hat {E}[\hat \theta_n]$ empirically. Then $\hat E[\hat \theta_n] - \theta$ is an estimate of the bias due to having an $n$-sized sample.

When bootstrapping you use your sample to approximate this process. The empirical distribution function $\hat F$ is an estimate of the true distribution function $F$. The act of sampling from $\hat F$ is in a sense an estimate of the act of sampling from $F$. The idea is then to push the whole sample-from-the-population process above one step down:

1. Calculate $\hat \theta$ in your original sample as an estimate of $\theta$.
2. Calculate $\hat E[\hat \theta^*_n]$ from bootstrap samples as an estimate of $\hat E[\hat \theta_n]$.

The idea is that the bootstrapped $\hat \theta_n^*$ should be biased from the "true" $\hat \theta_n$ in the same way that the sampled $\hat \theta_n$ is biased from the true $\theta$. Here is a link to a previous answer of mine where I show that the bootstrap bias estimate for the $1/n$ estimate of variance is pretty good.