In a textbook Probability Theory: The Logic of Science written by E. T. Jaynes and others, on page 13 it reads that:

For many years, there has been controversy over ‘frequentist’ versus ‘Bayesian’ methods of inference, in which the writer has been an outspoken partisan on the Bayesian side.

From this answer, if I am not wrong, I conjecture that any task which suits a frequentist approach can be solved also in a Bayesian way, and vice versa.

I later learned that the rise of MCMC algorithms in the 80's lead to an exponential rise in the use of Bayes in methods as complex models built through hierarchical distributions suddenly were tractable(reference: Mixtures of Conjugate Priors and MCMC).

Then my question is that if the Bayesian approach is better than the frequentist (1. Jaynes taught me that in that book; 2. for instance more parameters in models like the hidden Markov model can learn more information and then be better to reach the reality, the posterior distribution than the Markov model; 3) more reasons from this answer to this question: What is the importance of probabilistic machine learning?) and it has been tractable, then why frequentist methods still dominate the machine learning field?

In the answers to this question I learned why frequentist is better practically. But I wonder if Bayesian can be as practical as frequentist approaches? How can it be?

  • 1
    $\begingroup$ Both Bayesian and frequentist statistics are huge areas on their own, so asking for "the bottlenecks" will require a book with multiple volumes to answer. Can you narrow it down? $\endgroup$ Commented Oct 5, 2018 at 14:57
  • $\begingroup$ @MaartenBuis Sorry for that. Only some points are enough and details are not needed. $\endgroup$ Commented Oct 5, 2018 at 22:32
  • 1
    $\begingroup$ Might want to compare this question / answer: stats.stackexchange.com/questions/20558/… $\endgroup$
    – jbowman
    Commented Oct 5, 2018 at 22:38

2 Answers 2


MCMC is just an algorithm to sample from a distribution. On its own it is neither frequentist nor Bayesian. However, this algorithm has proven to be extremely useful for Bayesians. It has made it feasible to sample from the posterior distribution, and with those samples in hand, describe the posterior distribution. Before its use, real use of Bayesian statistics was mostly limited to models with conjugate priors. Meaning that before MCMC Bayesian statistics was more a theoretical, but unpractical, area of statistics. So most statistician would associate MCMC with Bayesian rather than frequentist statistics.

Is see both approaches as necessarily flawed approximations to answer to an unanswerable question: How do I give an answer when I don't have all the information? In the way I use statistics, the difference between the outcomes of these two methods is negligible. So I use pragmatic reasons to choose my method: unless Bayesian statistics has a clear added value, I will use frequentist because it is easier to "sell" my articles to journals. The potential added value of a Bayesian analysis would for example be if I have a prior that I really want to include in my analysis, and that prior is strong enough to influence my results. Especially, the latter is a fairly hard condition to meet in the type of analyses I do and the type of data I usually work with. So in the end I almost never use Bayesian statistics.

Some claim that the added value of the Bayesian is that the inference is more intuitive, and I kinda get their argument. However, it ignores the fact that most of my audience is not trained in Bayesian statistics. So until the majority is trained in Bayesian statistics, its potential cannot be realized. This is a catch 22: people are trained in frequentist statistics because that is what everybody uses, and everybody uses frequentists statistics because that is what everybody is trained in. However, we have to work with the world we live in, and not with the world we would want to live in.

  • 2
    $\begingroup$ Thanks for your awesome answer! But it seems that I have not fully understood the difference between these to approaches. I will update the question later. $\endgroup$ Commented Oct 5, 2018 at 12:35
  • 1
    $\begingroup$ More than two years later I updated the question. Could you please help check if it makes sense now? Thanks. $\endgroup$ Commented Jan 16, 2021 at 8:22
  • 1
    $\begingroup$ I find more and more scholars become secular Bayesian, for instance this guy. $\endgroup$ Commented Jan 16, 2021 at 8:39
  • 2
    $\begingroup$ Here you can find people who are not secular Bayesians! $\endgroup$ Commented Jan 16, 2021 at 12:47
  • 3
    $\begingroup$ The fact that most of the audience is not trained in Bayesian statistics doesn't hold water as an argument against Bayes. Much of the audience is trained in frequentist statistics yet still don't understand frequentist inference at all. $\endgroup$ Commented Jan 16, 2021 at 13:11

The reasons for the scarce applications of Bayesian methods are two faceted: 1) additional layer of complexity; 2) lack of computationally cheap algorithms to handle Bayesian methods and big data, and people may don't keep up with the trends in this direction.

Some practical approaches:

  1. MCMC is still expensive and hence variational inference approach is usually applied, for instance, MC dropout.
  2. For the online predicting, usually we use an expensive Bayesian neural network as a teacher to distill its knowledge to a point estimate student for deployment.
  3. Design better regularizations from the perspectives of Bayesian, for example Vadam.


  1. Hands-on Bayesian Neural Networks - a Tutorial for Deep Learning Users

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.