5
$\begingroup$

Several times I have come across (in lectures, papers, etc.) informal/tangential remarks that have the following basic form:

Philosophical issues aside, Bayesian methods are extremely convenient.

...with the implication being that they are more "convenient" (or "handy", "easy", etc.) than competing ("frequentist"?) methods1. As one can tell from the assertion's preamble, such remarks usually come from people who would not describe themselves as "Bayesians". (This is what I meant to suggest by the adverb "widely" in this post's title: i.e. even among "non-Bayesians", Bayesian methods are considered particularly convenient.)

What is it about Bayesian methods that make them so convenient?

The best answer to this question I can come up with is that Bayesian methods obviate the need for making some otherwise rather arbitrary decisions about parameters, and whole models. Instead, the Bayesian formalism serves as a principled way to simultaneously consider entire ensembles of such parameters and models (weighted by their corresponding priors).

Now, my grasp of "Bayesian statistics" is tenuous at best, so I don't put much faith in this explanation. I would love to see a more informed elaboration.


1 One example I can cite is a recent essay by Gelman and Shalizi's, where the authors disavow the "usual story" (i.e. the conventional interpretation of Bayesian statistics) as their reason for using Bayesian methods, and say instead they use them because with them they can "fit rich enough models" (presumably richer than they could fit using non-Bayesian methods). Unfortunately, this is the only hard reference I can provide. The other ones were in the form of informal lecture remarks, or in past reading that I can no longer remember specifically.

$\endgroup$
  • 5
    $\begingroup$ You may be interested in my question List of situations where a Bayesian approach is simpler, more practical, or more convenient. (Although I think this Q is distinct enough that it is not a duplicate.) $\endgroup$ – gung Sep 16 '13 at 14:04
  • $\begingroup$ @gung: thanks++, that's a very informative thread! $\endgroup$ – kjo Sep 16 '13 at 14:15
  • $\begingroup$ Extremely convenient compared to what? Frequentist? Sure in some situations but that is quite a generalization... $\endgroup$ – Glen Sep 16 '13 at 14:49
  • $\begingroup$ @Glen: I'm sure you're right. Such remarks were invariably addressed at audiences of statisticians. I assume that those making them felt confident that their listeners knew what they were referring to without having to get into specifics. For a non-statistician like me, however, such remarks are just puzzling. Hence my question here. $\endgroup$ – kjo Sep 16 '13 at 15:15
  • $\begingroup$ When putting together moderately complex models, computational approaches such as MCMC are often relatively straightforward to implement (though that's sometimes not saying a lot). These make many Bayesian models tractable, often much more so than their frequentist counterparts (take a fairly complex model that you could just about do with a frequentist approach; tack on dealing with outliers and model selection; you may now have a very difficult problem in a frequentist approach). In that sense, at least, Bayesian methods can be convenient. $\endgroup$ – Glen_b Sep 17 '13 at 0:00
8
$\begingroup$

In my personal opinion I think that Bayesian methods are extremely convenient in that you need to specify only four main ingredients for solving problems of inference, prediction, and decision making.

These questions encompass almost all of the discipline of statistics: describing a data set $D$, generalizing outward inferentially from $D$, predicting new data $D^*$, and helping people make decisions in the presence of uncertainty.

Given the set $\mathcal{B}$, of propositions summarizing your background assumptions and judgments about how the world works as far as $\theta$, $D$ and future data $D^*$ are concerned:

(a) It's natural (and indeed you must be prepared as a Bayesian) to specify two conditional probability distributions:

  • $p(\theta|\mathcal{B})$, to quantify all information about  external to $D$ that You judge relevant; and
  • $p(D|\theta,\mathcal{B})$, to quantify your predictive uncertainty, given $\theta$, about the data set $D$ before it's arrived.

(b) Given the distributions in (a), the distribution $p(\theta|D,\mathcal{B})$ quantifi es all relevant information about $\theta$, both internal and external to $D$, and must be computed via Bayes's Theorem: $$p(\theta|D,\mathcal{B})=c\times p(D|\theta,\mathcal{B})p(\theta|\mathcal{B})\hspace{.75cm}\text{(inference)}$$ where $c > 0$ is a normalizing constant chosen so that the left-hand side of of the above equation integrates (or sums) over $\Theta$ to 1.

(c) Your predictive distribution $p(D^*|\theta,D,\mathcal{B})$ for future data $D^*$ given the observed data set $D$ must be expressible as follows:

$$p(D^*|D,\mathcal{B})=\int_{\Theta}p(D^*|\theta,D,\mathcal{B})p(\theta|D,\mathcal{B})d\theta$$ often there's no information about $D^*$ contained in $D$ if is known, in which case this expression simplifies to $$p(D^*|\mathcal{B})=\int_{\Theta}p(D^*|\theta,D,\mathcal{B})p(\theta|D,\mathcal{B})d\theta\hspace{.75cm}\text{(prediction)}$$ (d) to make a sensible decision about which action $a$ you should take in the face of your uncertainty about $\theta$, however, you must be prepared to specify

(i) the set $\mathcal{A}$ of feasible actions among which you're choosing, and

(ii) a utility function $U(a,\theta)$, taking values on $\mathbb{R}$ and quantifying your judgments about the rewards (monetary or otherwise) that would ensue if you chose action $a$ and the unknown actually took the value $\theta$ (without loss of generality you can take large values of $U(a,\theta$) to be better than small values) then the optimal decision is to choose the action $a$ that maximizes the expectation of $U(a,\theta)$ over $p(\theta|D,\mathcal{B})$, i.e., $$a=\arg\max_{a\in\mathcal{A}}\mathbb{E}_{(\theta|D,\mathcal{B})}U(a,\theta)=\arg\max_{a\in\mathcal{A}}\int_{\Theta}U(a,\theta)p(\theta|D,\mathcal{B})d\theta\hspace{.75cm}\text{(decision making)}$$

And thus, there is a very simple and straight forward framework for conducting statistical analysis under the Bayesian framework. Once again, all you need to specify are the following four ingredients: $p(\theta|\mathcal{B}),p(D|\theta,\mathcal{B})$,the possible actions $a$, and the utility function $U(a,\theta)$.

Now as convenient as that is, there is no complete guidance under the Bayesian framework that actually tells you how to spefficy those four ingredients and this is actually the subjective and hard part from the Bayesian point of view. The true convenience of the Bayesian framework comes from the fact that if you are prepared to specify those four ingredients, then there is a clear routine for conducting most all of statistical reasoning. Once again though this is just my opinion.

$\endgroup$
1
$\begingroup$

Reasons why I think they're convenient, working in a field that's still somewhat dominated by frequentist statistics, but where a "Bayesian" analysis carry some cachet:

  • Credible intervals are awesome, in that they can be interpreted the way people really want to interpret confidence intervals from frequentist analysis. This is generally a benefit of Bayesian methods, they're easier to translate into "plain English" statements.
  • By "Bayesian" sometimes people mean "MCMC". It's pretty easy to argue that this is incorrect, but a "Bayesian" analysis using MCMC and uninformative priors can occasionally plow through computational problems that likelihood based methods will struggle with.
  • A properly done analysis with a very carefully considered set of priors can already set the stage for "How this finding changes the field's understanding of $TOPIC" with no additional input of effort.

That being said, I don't know that I'd actually consider them more convenient than conventional frequentist statistics from an analysis/coding point of view, though well written packages, and things like the BAYES statement in SAS help a great deal.

$\endgroup$
  • 1
    $\begingroup$ Downvoter, any particular reason you object to my answer? $\endgroup$ – Fomite Sep 18 '13 at 4:09
1
$\begingroup$

I'm just a rookie in all of this, but it seems to me that the Bayesian "Prior * Likelihood = Posterior" model is simple and flexible, which means "convenient".

For example, your priors are explicitly stated and your procedure does not change because you choose different priors. I could look at your analysis and decide that different priors seem more realistic and could otherwise use the same procedure as you. I may be oversimplifying here, but my feeling about frequentist methods is that they tend to have implicit prior-like assumptions and you choose your method based on what assumptions you accept.

Priors can easily be used to keep your posteriors away from awkward boundary conditions (negative values for counts, for example). The Bayesian procedure is also naturally iterative and mirrors the scientific process: the results of prior experiments/theory become your priors, and the posteriors of your experiment naturally become priors for further experimentation.

Posteriors are convenient because they are not points or even intervals, but rather distributions. This makes it convenient to do various things without complicated follow-on analysis. For example, if you want to determine if two parameters are equal, you can simply look at the differences of their posterior samples, take appropriate quantiles, and decide whether they are equal (difference near zero) or not.

A frequentist test can reject or fail to reject the null. A Bayesian analysis has three possibilities: accept, reject, and not-enough-information. It's convenient when you can actually accept a hypothesis rather than failing to reject it.

As EpiGrad says, Credible Intervals match human intuition and actually answer the question that most of us have. They're simple to calculate and don't require a model to do so: for a 95% CI you take the 0.025 and 0.975 quantiles of your posterior. Simple.

As EpiGrad also says, some people may be confusing MCMC and Bayesian methods, since real-world Bayesina analysis will almost always be MCMC-based. MCMC is a nice, generic black-box procedure, which is convenient -- you could live your whole Bayesian life in BUGS, JAGS, or Stan. Convenient. Though, of course, there can be a lot of tuning and analyzing involved in determining that the MCMC procedure ha actually converged and actually explores the entire density. Which might be the dark underbelly of Bayesian statistics.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.