7
$\begingroup$

From what I understand, Bootstrapping is incredibly useful in a Frequentist setting. In frequentist stats: we are trying to estimate long-run probabilities. In practice, we do not have an infinite number of samples. The bootstrap allows us to simulate an infinite number of re-samples. From what I understand, this is probably the most useful tool in Frequentist statistics.

Is the bootstrapping procedure essentially useless to a Bayesian? Bayesians only rely beliefs, and by resampling the original data: I doubt the belief would change.

Is the bootstrap useless in the Bayesian school of stats?

Although there exists a "Bayesian bootstrap", I am referring specifically to the Frequentist bootstrap.

$\endgroup$
  • $\begingroup$ What do you mean when you write "Bayesians only reply beliefs"? Please elaborate. $\endgroup$ – Greenparker Mar 23 '16 at 20:22
  • $\begingroup$ Bayesians interpret probability as a belief of the parameter that can be updated. Frequentists interpret probability as a long-run probability over time. $\endgroup$ – user46925 Mar 23 '16 at 20:30
  • $\begingroup$ Bootstrap can be interpret from Bayesian perspective, see stats.stackexchange.com/questions/71782/… or sumsar.net/blog/2015/04/… $\endgroup$ – Tim Mar 23 '16 at 20:40
  • 2
    $\begingroup$ Bootstrap is a basic form of non-parametric inference. It can also be implemented in a Bayesian framework, rather than just used to infer frequentist properties of some procedures. $\endgroup$ – Xi'an Mar 23 '16 at 21:12
8
$\begingroup$

Bradley Efron has written about this as well as recently participating in a JRSS webinar titled Frequentist Accuracy of Bayesian Estimates (here: http://www.rss.org.uk/RSS/Events/Online_and_virtual_events/Journal_club/Past_Journal_webinars/RSS/Events/Online_and_virtual_events_sub/Past_Journal_webinars.aspx?hkey=5c97f80b-3f97-401b-ad75-2ee6ff5f6c0c) where the discussant was Andrew Gelman.

Efron makes explicit use of the parametric bootstrap to develop a "frequentist standard deviation of a Bayesian point estimate..."

In the absence of relevant prior experience, popular Bayesian estimation techniques usually begin with some form of 'uninformative' prior distribution intended to have minimal inferential influence. Bayes' rule will still produce nice-looking estimates and credible intervals, but these lack the logical force attached to experience-based priors and require further justification. This paper concerns the frequentist assessment of Bayes estimates. A simple formula is shown to give the frequentist standard deviation of a Bayesian point estimate. The same simulations required for the point estimate also produce the standard deviation. Exponential family models make the calculations particularly simple, and bring in a connection to the parametric bootstrap.

So, no, the bootstrap is not "useless" to a Bayesian.

$\endgroup$
  • 8
    $\begingroup$ Since the goal is to assess the frequentist properties of a Bayesian procedure, this is only useful to a frequentist using a Bayesian procedure. $\endgroup$ – Xi'an Mar 23 '16 at 21:04
  • 1
    $\begingroup$ @Xi'an That's a fairly literal, even ideologically driven, interpretation of the abstract. Screen the webinar, note that Efron and Gelman find multiple points of agreement and concordance in their discussion and see if you still support your observation. In my view, if one is interested in thoroughly addressing a research issue then anwering it from multiple points of view that are independent of ideology would seem prudent. If so, then Efron's metric should not be useful only to a frequentist. $\endgroup$ – Mike Hunter Mar 26 '16 at 10:48
5
$\begingroup$

First, your interpretation of Bayesian statistics seems to be a bit restrictive. Bayesian methods do not necessarily rely on belief, e.g. objective Bayesians view the prior as a catalyst needed to express the parameters distribution having observed the data.

Second, when belief is available it is not related to the observations. The prior by definition is independent from the observed data and I guess that when stating "Bayesians only rely beliefs, and by resampling the original data: I doubt the belief would change" you misinterpret the meaning of the posterior distribution.

Finally, bootstrap can be used to estimate certain kind of posterior distributions. The answer Is it possible to interpret the bootstrap from a Bayesian perspective? gives you the details but here is an extract from the answer:

Hence we might think of the bootstrap distribution as a “poor man’s” Bayes posterior. By perturbing the data, the bootstrap approximates the Bayesian effect of perturbing the parameters, and is typically much simpler to carry out.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy