63
$\begingroup$

There have been many debates within statistics between Bayesians and frequentists. I generally find these rather off-putting (although I think it has died down). On the other hand, I've met several people who take an entirely pragmatic view of the issue, saying that sometimes it is more convenient to conduct a frequentist analysis and sometimes it's easier to run a Bayesian analysis. I find this perspective practical and refreshing.

It occurs to me that it would be helpful to have a list of such cases. Because there are too many statistical analyses, and because I assume that it is ordinarily more practical to conduct a frequentist analysis (coding a t-test in WinBUGS is considerably more involved than the single function call required to perform the frequentist-based version in R, for example), it would be nice to have a list of the situations where a Bayesian approach is simpler, more practical, and / or more convenient than a frequentist approach.


(Two answers that I have no interest in are: 'always', and 'never'. I understand people have strong opinions, but please don't air them here. If this thread becomes a venue for petty squabbling, I will probably delete it. My goal here is to develop a resource that will be useful for an analyst with a job to do, not an axe to grind.)

People are welcome to suggest more than one case, but please use separate answers to do so, so that each situation can be evaluated (voted / discussed) individually. Answers should list: (1) what the nature of the situation is, and (2) why the Bayesian approach is simpler in this case. Some code (say, in WinBUGS) demonstrating how the analysis would be done and why the Bayesian version is more practical would be ideal, but I expect will be too cumbersome. If it can be done easily I would appreciate it, but please include why either way.

Finally, I recognize that I have not defined what it means for one approach to be 'simpler' than another. The truth is, I'm not entirely sure what it should mean for one approach to be more practical than the other. I'm open to different suggestions, just specify your interpretation when you explain why a Bayesian analysis is more convenient in the situation you discuss.

$\endgroup$
  • 7
    $\begingroup$ Multi-level modelling is definitely easier for bayesian, especially conceptually. $\endgroup$ – probabilityislogic Aug 6 '13 at 12:36
  • $\begingroup$ The questioner may not like this, but there is no getting around actually thinking about and understanding what the inference or modeling methods are actually doing, and how to interpret them. It makes little sense to seek which is simpler when they are achieving different goals. $\endgroup$ – user48782 Jun 21 '14 at 23:52
  • 1
    $\begingroup$ I'm fine w/ the advice to think clearly about what you are trying to do & how the analyses work, @Mayo. I've given that advice myself ;-). I am also familiar w/ the idea that Bayesian & frequentist analyses make different assumptions about the nature of probability. However, as I note in the question, I have met several people (ie, stats PhD's who understand the issues quite well) who say that there are times when they are sufficiently similar & that 1 is more convenient. The other answers demonstrate it is possible to give such examples. I would love to see any additional cases you know of. $\endgroup$ – gung - Reinstate Monica Jun 23 '14 at 3:11
  • $\begingroup$ Just curious - when you say "frequentist analysis is more practical" are you talking about the software - such as using lm () in R being easier to use? Or is there something else? $\endgroup$ – probabilityislogic Dec 9 '17 at 13:37
  • $\begingroup$ @probabilityislogic, essentially. My example is using t.test() as opposed to coding up a Bayesian t-test in WinBUGS, which requires a great deal more code. Perhaps instead of "more practical", I should have said 'easier'. $\endgroup$ – gung - Reinstate Monica Dec 9 '17 at 20:12
26
$\begingroup$

(1) In contexts where the likelihood function is intractable (at least numerically), the use of the Bayesian approach, by means of Approximate Bayesian Computation (ABC), has gained ground over some frequentist competitors such as composite likelihoods (1, 2) or the empirical likelihood because it tends to be easier to implement (not necessarily correct). Due to this, the use of ABC has become popular in areas where it is common to come across intractable likelihoods such as biology, genetics, and ecology. Here, we could mention an ocean of examples.

Some examples of intractable likelihoods are

  • Superposed processes. Cox and Smith (1954) proposed a model in the context of neurophysiology which consists of $N$ superposed point processes. For example consider the times between the electrical pulses observed at some part of the brain that were emited by several neurones during a certain period. This sample contains non iid observations which makes difficult to construct the corresponding likelihood, complicating the estimation of the corresponding parameters. A (partial)frequentist solution was recently proposed in this paper. The implementation of the ABC approach has also been recently studied and it can be found here.

  • Population genetics is another example of models leading to intractable likelihoods. In this case the intractability has a different nature: the likelihood is expressed in terms of a multidimensional integral (sometimes of dimension $1000+$) which would take a couple of decades just to evaluate it at a single point. This area is probably ABC's headquarters.

$\endgroup$
  • 1
    $\begingroup$ These are great! Can you split them into 2 answers (so that I can upvote you twice ;-), and provide just one, typical (toy) example? Thanks. $\endgroup$ – gung - Reinstate Monica Oct 29 '12 at 15:15
  • 2
    $\begingroup$ @Procrastinator Could you elaborate a bit on why in some contexts the likelihood is intractable. Given that you mention biology, genetics and ecology, I imagine that it is linked to the complex dependencies between parameters. I think this would be particularly useful for people not familiar with Bayesian analysis (of which I am), especially given that the wikipedia page on ABC you link to does not provide a lot of context. Thanks $\endgroup$ – Antoine Vernet Oct 29 '12 at 15:16
15
$\begingroup$

As Bayesian software improves, the "easier to apply" issue becomes moot. Bayesian software is becoming packaged in easier and easier forms. A recent case in point is from an article titled, Bayesian estimation supersedes the t test. The following web site provides links to the article and software: http://www.indiana.edu/~kruschke/BEST/

An excerpt from the article's introduction:

... some people have the impression that conclusions from NHST and Bayesian methods tend to agree in simple situations such as comparison of two groups: “Thus, if your primary question of interest can be simply expressed in a form amenable to a t test, say, there really is no need to try and apply the full Bayesian machinery to so simple a problem” (Brooks, 2003, p. 2694). This article shows, to the contrary, that Bayesian parameter estimation provides much richer information than the NHST t test and that its conclusions can differ from those of the NHST t test. Decisions based on Bayesian parameter estimation are better founded than those based on NHST, whether the decisions derived by the two methods agree or not.

$\endgroup$
  • 10
    $\begingroup$ I gather your answer is either always or 'it will be always soon'. This is the answer of a Bayesian partisan. $\endgroup$ – gung - Reinstate Monica Oct 31 '12 at 3:40
  • 3
    $\begingroup$ And here is an online javascript implementation of Kruschke's BEST. Bayesian analysis in the browser :) sumsar.net/best_online $\endgroup$ – Rasmus Bååth Jan 8 '13 at 16:08
13
$\begingroup$

(2) Stress-strength models. The use of stress-strength models is popular in reliability. The basic idea consists of estimating the parameter $\theta=P(X<Y)$ where $X$ and $Y$ are random variables. Interestingly, the calculation of the profile likelihood of this parameter is quite difficult in general (even numerically ) except for some toy examples such as the exponential or normal case. For this reason, ad hoc frequentist solutions need to be considered such as the empirical likelihood (see) or confidence intervals whose construction is difficult as well in a general framework. On the other hand, the use of a Bayesian approach is very simple given that if you have a sample of the posterior distribution of the parameters of the distributions of $X$ and $Y$, then you can easily transform them into a sample of the posterior of $\theta$.

Let $X$ be a random variable with density and distribution given respectively by $f(x;\xi_1)$ and $F(x;\xi_1)$. Similarly, let $Y$ be a random variable with density and distribution given respectively by $g(y;\xi_2)$ and $G(y;\xi_2)$. Then

$$\theta = \int F(y;\xi_1)g(y;\xi_2)dy. \tag{$\star$}$$

Note that this parameter is a function of the parameters $(\xi_1,\xi_2)$. In the exponential and normal cases, this can be expressed in closed form (see) but this is not the case in general (see this paper for an example). This complicates the calculation of the profile likelihood of $\theta$ and consequently the classical interval inference on this parameter. The main problem can be summarised as follows "The parameter of interest is an unknown/complicated function of the model-parameters and therefore we cannot find a reparameterisation that involves the parameter of interest".

From a Bayesian perspective this is not an issue given that if we have a sample from the posterior distribution of $(\xi_1,\xi_2)$, then we can simply input these samples into $(\star)$ in order to obtain a sample of the posterior of $\theta$ and provide interval inference for this parameter.

$\endgroup$
  • 4
    $\begingroup$ +1 to both your answers - these are interesting examples. You seem to assume that the frequentist solution must be likelihood-based though, seemingly ignoring nonparametric methods. In this particular case, the nonparametric frequentist methods of Chapter 5 in the book that you linked to seem to me to be at least as easy to use as the Bayesian methods. $\endgroup$ – MånsT Oct 30 '12 at 10:16
  • $\begingroup$ @MånsT Good point. Indeed, this answer focuses on Bayesian vs. likelihood-based inference. I wrote an answer some time ago on a nonparameteric solution of this problem which, as you point out, is as easy as the Bayesian approach. $\endgroup$ – user10525 Oct 30 '12 at 10:39
  • $\begingroup$ @MånsT & Procrastinator, I also had upvoted this. It isn't clear to me what "easiest" should mean in the context of this question / the comments made to me over the years that prompted it. Mostly I wanted to exclude answers where the analyst has a theoretical position such that one is always better for reasons unrelated to the situation. Such a position is defensible, of course, and there are many places on the internet to view such arguments (including some on CV), but b/c of that I thought it would be interesting to have a place where people list cases where they would use the other & why. $\endgroup$ – gung - Reinstate Monica Oct 31 '12 at 4:01
13
$\begingroup$

I am trained in frequentist statistics (econometrics actually), but I have never had a confrontational stance towards the Bayesian approach, since my point of view is that the philosophical source of this "epic" battle was fundamentally misguided from the start (I have aired my views here). In fact I plan to also train myself in the Bayesian approach in the immediate future.

Why? Because one of the aspects of frequentist statistics that fascinates me the most as a mathematical and conceptual endeavor, at the same time it troubles me the most: sample-size asymptotics. At least in econometrics, almost no serious paper today claims that any of the various estimators usually applied in frequentist econometrics possesses any of the desirable "small-sample" properties we would want from an estimator. They all rely on asymptotic properties to justify their use. Most of the tests used have desirable properties only asymptotically... But we are not in "z-land / t-land" anymore: all the sophisticated (and formidable) apparatus of modern frequentist estimation and inference is also highly idiosyncratic- meaning that sometimes, a laaaaaaaaa...aaaarge sample is indeed needed in order for these precious asymptotic properties to emerge and affect favorably the estimates derived from the estimators, as has been proven by various simulations. Meaning tens of thousands of observations -which although they start to become available for some fields of economic activity (like labor or financial markets), there are others (like macroeconomics) in which they will never do (at least during my life span). And I am pretty bothered by that, because it renders the derived results truly uncertain (not just stochastic).

Bayesian econometrics for small samples do not rely on asymptotic results. "But they rely on the subjective prior!" is the usual response... to which, my simple, practical, answer is the following : "if the phenomenon is old and studied before, the prior can be estimated from past data. If the phenomenon is new, by what else if not by subjective arguments can we start the discussion about it?

$\endgroup$
  • 5
    $\begingroup$ This is a worthwhile perspective, but note that there are frequentist approaches that do not rely so much on asymptotics, such as Monte Carlo simulations, bootstrapping, and permutation tests. $\endgroup$ – gung - Reinstate Monica Sep 16 '13 at 21:09
  • 1
    $\begingroup$ And if one tries to get the best from both worlds? An attempt: Bartels, Christian (2017): Using prior knowledge in frequentist tests. figshare. doi.org/10.6084/m9.figshare.4819597.v3 Retrieved: 17 18, May 10, 2017 (GMT) $\endgroup$ – user36160 May 10 '17 at 17:28
13
$\begingroup$

This is a late reply, nevertheless I hope it adds something. I have been trained in telecommunication where most of the time we use the Bayesian approach.

Here is a simple example: Suppose you can transmit four possible signals of +5, +2.5, -2.5, and -5 volts. One of the signals from this set is transmitted, but the signal is corrupted by Gaussian noise by the time it reaches the receiving end. In practice, the signal is also attenuated, but we will drop this issue for simplicity. The question is: If you are at the receiving end, how do you design a detector that tell you which one of these signals was originally transmitted?

This problem obviously lies in the domain of hypothesis testing. However, you can't use p-values, since significance testing can potentially reject all four possible hypotheses, and you know that one of these signals was actually transmitted. We can use Neyman-Pearson method to design a detector in principle, but this method works best for binary hypotheses. For multiple hypotheses, it becomes too clumsy when you need to deal with a number constrains for false alarm probabilities. A simple alternative is given by the Bayesian hypothesis testing. Any of these signals could have been chosen to be transmitted, so the prior is equiprobable. In such equiprobable cases, the method boils down to choosing the signal with maximum likelihood. This method can be given a nice geometric interpretation: choose the signal which happens to be closest to the received signal. This also leads to the partition of the decision space into a number of decision regions, such that if the received signal were to fall within a particular region, then it is decided that the hypothesis associated with that decision region is true. Thus the design of a detector is made easy.

$\endgroup$
  • 4
    $\begingroup$ Thank you sharing your experience with us. Welcome to our site! $\endgroup$ – whuber Jul 30 '14 at 21:31
4
$\begingroup$

So called 'Frequentist' statistical tests are typically equivalent to the in principle more complex Bayesian approach under certain assumptions. When these assumptions are applicable, then either approach will give the same result, so it is safe to use the easier to apply Frequentist test. The Bayesian approach is safer in general because is makes the assumptions explicit but if you know what you are doing the Frequentist test is often just as good as a Bayesian approach and typically easier to apply.

$\endgroup$
  • 1
    $\begingroup$ Thanks. You note that Frequentist tests are "typically easier to apply". Do you know of a specific situation when this is not the case (if F are only "typically" easier, there must be some such cases). For the present, we can set aside issues of which approach would be "safer", & just concentrate on the easier to apply part. $\endgroup$ – gung - Reinstate Monica Oct 29 '12 at 4:41
4
$\begingroup$

(I'll try what I thought would be the most typical kind of answer.)

Let's say you have a situation where there are several variables and one response, and you know a good deal about how one of the variables ought to be related to the response, but not as much about the others.

In a situation like this, if you were to run a standard multiple regression analysis, that prior knowledge would not be taken into account. A meta-analysis might be conducted afterwards, which might be interesting in shedding light on whether the current result was consistent with the other findings and might allow a slightly more precise estimate (by including the prior knowledge at that point). But that approach wouldn't allow what was known about that variable to influence the estimates of the other variables.

Another option is that it would be possible to code, and optimize over, your own function that fixes the relationship with the variable in question, and finds parameter values for the other variables that maximize the likelihood of the data given that restriction. The problem here is that whereas the first option does not adequately constrain the beta estimate, this approach over-constrains it.

It may be possible to jury-rig some algorithm that would address the situation more appropriately, situations like this seem like ideal candidates for Bayesian analysis. Anyone not dogmatically opposed to the Bayesian approach ought to be willing to try it in cases like this.

$\endgroup$
2
$\begingroup$

An area of research in which the Bayesian methods are extremely straightforward and the Frequentist methods are extremely hard to follow is that of Optimal Design.

In a simple version of the problem, you would like to estimate a single regression coefficient of a logistic regression as efficiently as possible. You are allowed to take a single sample with $x^{(1)}$ equal to whatever you would like, update your estimate for $\beta$ and then choose your next $x^{(2)}$, etc. until your estimate for $\beta$ meets some accuracy level.

The tricky part is that the true value of $\beta$ will dictate what the optimal choice of $x^{(i)}$ is. You might consider using the current estimate of $\hat \beta$ of $\beta$ with the understanding that you are ignoring the error in $\hat \beta$. As such, you can get a maybe only mildly sub-optimal choice of $x^{(i)}$ given a reasonable estimate of $\beta$.

But what about when you first start? You have no Frequentist estimate of $\beta$, because you have no data. So you'll need to gather some data (definitely in a very suboptimal manner), without a lot of guiding theory to tell you what to pick. And even after a few picks, the Hauck-Donner effect can still prevent you from having a defined estimate of $\beta$. If you read up on the Frequentist literature about how to deal with this, it's basically "randomly pick $x$'s until there exists a value of $x$ such that there are 0's and 1's above and below that point" (which means the Hauck-Donner effect will not occur).

From the Bayesian perspective, this problem is very easy.

  1. Start your prior belief about $\beta$.
  2. Find the $x$ that will have the maximum effect on posterior distribution
  3. Sample using value of $x$ chosen from (2) and update your posterior
  4. Repeat steps 2 & 3 until desired accuracy is met

The Frequentist literature will bend over backwards to get you try to find reasonable values of $x$ for which you can hopefully take samples at and avoid the Hauck-Donner effect so that you can start taking sub-optimal samples... whereas the Bayesian method is all very easy and takes into account the uncertainty in the parameter of interest.

$\endgroup$
2
$\begingroup$

Perhaps one of the most straightforward and common cases where the Bayesian approach is easier is the quantifying the uncertainty of parameters.

In this answer, I'm not referring to the interpretation of confidence intervals vs. credible intervals. For the moment, let's assume that a user is fine with using either method.

With that said, in the Bayesian framework, it's straight forward; it's the marginal variance of the posterior for any individual parameter of interest. Assuming you can sample from the posterior, then just take your samples and compute your variances. Done!

In the Frequentist case, this is usually only straightforward in some cases and it's a real pain when it's not. If we have a large number of samples vs. small number of parameters (and who really knows how large is large enough), we can use MLE theory to derive CI's. However, those criteria don't always hold, especially for interesting cases (i.e., mixed effects models). Sometimes we can use bootstrapping, but sometimes we can't! In the cases we can't, it can be really, really hard to derive error estimates, and often require a bit of cleverness (i.e., Greenwood's formula for deriving SE's for Kaplan Meier curves). "Using some cleverness" is not always a reliable recipe!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.