Suppose I have some data $x_N$ of `size' $N$ which were drawn according to some measure $P_N(x_N)$.

I'm imagining $x_N$ as being any of:

  • an exchangeable / iid sequence of length $N$
  • a stationary sequence of length $N$
  • a stationary random field with $N$ sites
  • (other examples with some sort of symmetry / stationarity assumption, such that a form of bootstrapping is available)

Suppose now that, for some downstream task (e.g. computing approximate confidence intervals), I'm interested in bootstrapping $x_N$, i.e. I have some conditional distribution $\beta_N ( x_N \to y_N)$ which takes as input my data $x_N$, and outputs a bootstrapped version of the data, $y_N$, which has the same type as the original data.

The marginal law of $y_N$ is then given by

\begin{align} Q_N ( y_N ) = \int P_N ( x_N ) \beta_N ( x_N \to y_N) dx_N. \end{align}

Now, typically it will be the case that $Q_N \neq P_N$. However, as $N$ grows, it intuitively seems that one should have $Q_N \approx P_N$ in some appropriate metric. This would be one way of justifying using bootstrap samples for approximate inference; in the large-sample regime, you are somehow not perturbing things too badly.

What I am looking for is a more precise mathematical statement of the above intuition, i.e. for a given class of data-generating processes $\{ P_N \}_{N \geqslant 1}$, and a family of bootstrap resampling mechanisms $\{ \beta_N \}_{N \geqslant 1}$, I would like conditions under which a bound of the form

$$D ( Q_N, P_N ) \leqslant \varepsilon(N),$$

where $D$ is some discrepancy measure (e.g. some transport distance), and $\varepsilon(N)$ is some relatively-explicit sequence which tends to $0$.

If my intuition is false, i.e. $\lim\inf_{N \to \infty} D ( Q_N, P_N ) = \delta > 0$ for most useful $D$, then it would be useful to understand the failure modes, and to possibly revise my question to adjust for them. Understanding the qualitative differences between $Q_N$ and $P_N$ is of general interest to me; the bounds are just a specific manifestation of this.

As a final comment, I'm particularly interested in how one proves results of this form for non-iid $x_N$, e.g. the case of stationary time series, stationary random fields, etc.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.