19
$\begingroup$

I have a rather complicated decision analysis problem involving reliability testing and the logical approach (to me) seems to involve using MCMC to support a Bayesian analysis. However, it has been suggested that it would be more appropriate to use a bootstrapping approach. Could someone suggest a reference (or three) that might support the use of either technique over the other (even for particular situations)? FWIW, I have data from multiple, disparate sources and few/zero failure observations. I also have data at the subsystem and system levels.

It seems a comparison like this should be available, but I've had not luck searching the usual suspects. Thanks in advance for any pointers.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Given that the classical bootstrap can be thought of as a computer-implemented maximum likelihood method (i.e. a non-bayesian (flat prior) technique), would it be better to rephrase your question to something like "when to use frequentist vs. bayesian technique?" Some background on bootstrap: stats.stackexchange.com/questions/18469/… $\endgroup$
    – Yevgeny
    Mar 28, 2012 at 17:40
  • 1
    $\begingroup$ Hmmm..I guess I disagree. Hopefully 'bootstrap' specifically suggests characterization of the interval; a bit more focused than just 'frequentist'. At least 'bootstrap' will keep most of the religious fanatics at bay. Also, thanks for the link, but I was familiar with your previous comment before I posted this. $\endgroup$
    – Aengus
    Mar 28, 2012 at 18:41
  • 1
    $\begingroup$ Let me rephrase, do you have any useful prior information, or does the problem have a hierarchical (nested) structure? If so, then a bayesian technique is probably better (especially if the number of model parameters is large relative to the amount of available data, so estimation would benefit from "bayesian shrinking"). Otherwise MLE/bootstrap is sufficient. $\endgroup$
    – Yevgeny
    Mar 28, 2012 at 19:07
  • $\begingroup$ I guess another possible approach is to use mixed-effects models (e.g. using R package lme4) to model the hierarchical structure you've aluded to. That would also help to stabilize the estimates for (hierarchical) models with large number of parameter. $\endgroup$
    – Yevgeny
    Mar 28, 2012 at 21:49
  • 1
    $\begingroup$ A bootstrap analysis can very well be viewed as a Bayesian analysis, so your question could almost as well be "When to use the bootstrap vs. another Bayesian model" (Your question spurred me to write up this interpretation of the bootstrap as a Bayesian model: sumsar.net/blog/2015/04/…). Given the question, I agree with @Yevgeny that we would probably need more information regarding your specific problem before we could recommend a model. $\endgroup$
    – Rasmus Bååth
    Apr 21, 2015 at 19:46

2 Answers 2

Reset to default
16
$\begingroup$

To my thinking, your problem description points to two main issues. First:

I have a rather complicated decision analysis...

Assuming you've got a loss function in hand, you need to decide whether you care about frequentist risk or posterior expected loss. The bootstrap lets you approximate functionals of the data distribution, so it will help with the former; and posterior samples from MCMC will let you assess the latter. But...

I also have data at the subsystem and system levels

so these data have hierarchical structure. The Bayesian approach models such data very naturally, whereas the bootstrap was originally designed for data modelled as i.i.d. While it has been extended to hierarchical data (see references in the introduction of this paper), such approaches are relatively underdeveloped (according to the abstract of this article) .

To summarize: if it really is frequentist risk that you care about, then some original research in the application of the bootstrap to decision theory may be necessary. However, if minimizing posterior expected loss is a more natural fit to your decision problem, Bayes is definitely the way to go.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, I had not run across either of these; the latter article seems particularly interesting. $\endgroup$
    – Aengus
    Mar 26, 2012 at 16:01
8
$\begingroup$

I've read that the non-parametric bootstrap can be seen as a special case of a Bayesian model with a discrete (very)non informative prior, where the assumptions being made in the model is that the data is discrete, and the domain of your target distribution is completely observed in your sample.

Here are two references:

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.