Might the Nyquist theorem point toward an answer for the number of bootstrap samples required?

Question

It seems like the generally accepted answer to, "How many bootstrap replications should I run?" has been, "It depends." It depends seems like a correct answer. Nevertheless, I was wondering whether there might be a parallel between bootstrap observations and samples taken from a continuous process in such a way as to allow for a better answer.

Specifically, the Nyquist theorem appears to suggest one sample at at least twice the occupied bandwidth. Density plots of observed (or bootstrapped) distributions also have 'bandwidth'. Can these two types of bandwidth be equated for the purposes of specifying a desired degree of accuracy? That is, if attempting to get a bootstrapped distribution that is sufficient (maybe excessive) could one terminate once one had no fewer than two observations within each (perhaps rolling?) bin that is the desired degree of accuracy?

For point estimates it would then seem to be fully sufficient (maybe excessive) to sample until one collects two bootstrap samples within a minimum bin size of interest, e.g. if one wanted a score that was accurate to the hundredths place, then the estimate and its two nearest-sampled neighbors should be at least within .005 of each other – of course that assumes that you know where your point estimate will be, which you never will. So then, if you had a prior point estimate, you could wait until that condition was satisfied and then sample twice again as much to be 'sure'. Alternatively, if you really care about the fidelity of the entire distribution, then you could sample until no two observations are further apart than your nearest-neighbor threshold (eliminating observations that are identical). Of course it would be important to consider that some thresholds can never be met (e.g. a bootstrap of the mean of 1, 2, and 3 below ⅓‎).

This (of course) doesn't cover all types of bootstraps that one might want to perform, but it seems like it might cover some. This might end up being a matter of opinion, but I'm hoping there is an answer or informed opinion sufficient to keep the question alive.

Answer 1

Nope. Nyquist is not blind. It requires an understanding of "the characteristic time" of the system. It then asserts that in order to make a Fourier approximation the sampling time needs to be an integer fraction of that characteristic time. the smallest nontrivial integer fraction is 1/2.

In the real (non-theoretic) world, it is often very helpful to account for signal variation, to diagnose system changes, to account for uncertainties in the modeling and environment, to sample much more than the minimum integer subsample of 2. Useful and stable systems tend to sample at the "statistics magic sample numbers" of 5 or 30. That means there are either 5, or 30 measurements between "characteristic minimum" times for the assumed system.

All of this presupposes that you know your characteristic time. Statisticians are "trust but verify" folks and this is one reason that they want to uniformly randomly sample a system. The minimum distance is not prescribed. They are not operating in integers multiples fixed steps, but reals.

For the Metropolis-Hastings algorithm, it can take 20k or substantially more steps to get clean integration using that random sampling. The discipline around convergence that Bayesian method are going to speak to much more deeper fundamentals than Nyquist. They are going to look like "it depends" but .. it does depend. Nyquist depends too. In the investigation you are likely to find more of why it depends.

Best of luck.