3
$\begingroup$

It seems like the generally accepted answer to, "How many bootstrap replications should I run?" has been, "It depends." It depends seems like a correct answer. Nevertheless, I was wondering whether there might be a parallel between bootstrap observations and samples taken from a continuous process in such a way as to allow for a better answer.

Specifically, the Nyquist theorem appears to suggest one sample at at least twice the occupied bandwidth. Density plots of observed (or bootstrapped) distributions also have 'bandwidth'. Can these two types of bandwidth be equated for the purposes of specifying a desired degree of accuracy? That is, if attempting to get a bootstrapped distribution that is sufficient (maybe excessive) could one terminate once one had no fewer than two observations within each (perhaps rolling?) bin that is the desired degree of accuracy?

For point estimates it would then seem to be fully sufficient (maybe excessive) to sample until one collects two bootstrap samples within a minimum bin size of interest, e.g. if one wanted a score that was accurate to the hundredths place, then the estimate and its two nearest-sampled neighbors should be at least within .005 of each other – of course that assumes that you know where your point estimate will be, which you never will. So then, if you had a prior point estimate, you could wait until that condition was satisfied and then sample twice again as much to be 'sure'. Alternatively, if you really care about the fidelity of the entire distribution, then you could sample until no two observations are further apart than your nearest-neighbor threshold (eliminating observations that are identical). Of course it would be important to consider that some thresholds can never be met (e.g. a bootstrap of the mean of 1, 2, and 3 below ⅓‎).

This (of course) doesn't cover all types of bootstraps that one might want to perform, but it seems like it might cover some. This might end up being a matter of opinion, but I'm hoping there is an answer or informed opinion sufficient to keep the question alive.

$\endgroup$
  • $\begingroup$ For something related, but not quite germane, you might look at Bernard Widrow's 1956 ScD thesis, A Study of Rough Amplitude Quantization by Means of Nyquist Sampling Theory (MIT). There is a separate journal article with the same author and title. At the risk of sounding somewhat unappreciative, it is essentially just an application of Nyquist theory along the amplitude dimension as opposed to the time dimension. $\endgroup$ – cardinal May 8 '14 at 2:39
  • $\begingroup$ Also related: Assumptions regarding bootstrap estimates of uncertainty $\endgroup$ – cardinal May 8 '14 at 2:44
  • $\begingroup$ This question is a misguided. And the other answers given here don't address the misconceptions in this question at all. The bootstrap is about "sampling" in the statistical sense i.e how to estimate uncertainty in estimates of your model given a random sample of observations. On the other hand, Nyquist sampling does not require any notion of randomness. It is all about discretizing a continuous periodic signal that could be entirely deterministic. $\endgroup$ – user3303 Apr 29 '15 at 21:38
1
$\begingroup$

Nope. Nyquist is not blind. It requires an understanding of "the characteristic time" of the system. It then asserts that in order to make a Fourier approximation the sampling time needs to be an integer fraction of that characteristic time. the smallest nontrivial integer fraction is 1/2.

In the real (non-theoretic) world, it is often very helpful to account for signal variation, to diagnose system changes, to account for uncertainties in the modeling and environment, to sample much more than the minimum integer subsample of 2. Useful and stable systems tend to sample at the "statistics magic sample numbers" of 5 or 30. That means there are either 5, or 30 measurements between "characteristic minimum" times for the assumed system.

All of this presupposes that you know your characteristic time. Statisticians are "trust but verify" folks and this is one reason that they want to uniformly randomly sample a system. The minimum distance is not prescribed. They are not operating in integers multiples fixed steps, but reals.

For the Metropolis-Hastings algorithm, it can take 20k or substantially more steps to get clean integration using that random sampling. The discipline around convergence that Bayesian method are going to speak to much more deeper fundamentals than Nyquist. They are going to look like "it depends" but .. it does depend. Nyquist depends too. In the investigation you are likely to find more of why it depends.

Best of luck.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.