# Bounding the distributional error introduced by bootstrapping

Suppose I have some data $$x_N$$ of `size' $$N$$ which were drawn according to some measure $$P_N(x_N)$$.

I'm imagining $$x_N$$ as being any of:

• an exchangeable / iid sequence of length $$N$$
• a stationary sequence of length $$N$$
• a stationary random field with $$N$$ sites
• (other examples with some sort of symmetry / stationarity assumption, such that a form of bootstrapping is available)

Suppose now that, for some downstream task (e.g. computing approximate confidence intervals), I'm interested in bootstrapping $$x_N$$, i.e. I have some conditional distribution $$\beta_N ( x_N \to y_N)$$ which takes as input my data $$x_N$$, and outputs a bootstrapped version of the data, $$y_N$$, which has the same type as the original data.

The marginal law of $$y_N$$ is then given by

\begin{align} Q_N ( y_N ) = \int P_N ( x_N ) \beta_N ( x_N \to y_N) dx_N. \end{align}

Now, typically it will be the case that $$Q_N \neq P_N$$. However, as $$N$$ grows, it intuitively seems that one should have $$Q_N \approx P_N$$ in some appropriate metric. This would be one way of justifying using bootstrap samples for approximate inference; in the large-sample regime, you are somehow not perturbing things too badly.

What I am looking for is a more precise mathematical statement of the above intuition, i.e. for a given class of data-generating processes $$\{ P_N \}_{N \geqslant 1}$$, and a family of bootstrap resampling mechanisms $$\{ \beta_N \}_{N \geqslant 1}$$, I would like conditions under which a bound of the form

$$D ( Q_N, P_N ) \leqslant \varepsilon(N),$$

where $$D$$ is some discrepancy measure (e.g. some transport distance), and $$\varepsilon(N)$$ is some relatively-explicit sequence which tends to $$0$$.

If my intuition is false, i.e. $$\lim\inf_{N \to \infty} D ( Q_N, P_N ) = \delta > 0$$ for most useful $$D$$, then it would be useful to understand the failure modes, and to possibly revise my question to adjust for them. Understanding the qualitative differences between $$Q_N$$ and $$P_N$$ is of general interest to me; the bounds are just a specific manifestation of this.

As a final comment, I'm particularly interested in how one proves results of this form for non-iid $$x_N$$, e.g. the case of stationary time series, stationary random fields, etc.