# When (if ever) is a frequentist approach substantively better than a Bayesian?

Background: I do not have an formal training in Bayesian statistics (though I am very interested in learning more), but I know enough--I think--to get the gist of why many feel as though they are preferable to Frequentist statistics. Even the undergraduates in the introductory statistics (in social sciences) class I am teaching find the Bayesian approach appealing--"Why are we interested in calculating the probability of the data, given the null? Why can't we just quantify the probability of the null hypothesis? Or the alternative hypothesis? And I've also read threads like these, which attest to the empirical benefits of Bayesian statistics as well. But then I came across this quote by Blasco (2001; emphasis added):

If the animal breeder is not interested in the philosophical problems associated with induction, but in tools to solve problems, both Bayesian and frequentist schools of inference are well established and it is not necessary to justify why one or the other school is preferred. Neither of them now has operational difficulties, with the exception of some complex cases...To choose one school or the other should be related to whether there are solutions in one school that the other does not offer, to how easily the problems are solved, and to how comfortable the scientist feels with the particular way of expression results.

The Question: The Blasco quote seems to suggest that there might be times when a Frequentist approach is actually preferable to a Bayesian one. And so I am curious: when would a frequentist approach be preferable over a Bayesian approach? I'm interested in answers that tackle the question both conceptually (i.e., when is knowing the probability of the data conditioned on the null hypothesis especially useful?) and empirically (i.e., under what conditions do Frequentist methods excel vs. Bayesian?).

It would also be preferable if answers were conveyed as accessibly as possible--it would be nice to take some responses back to my class to share with my students (though I understand some level of technicality is required).

Finally, despite being a regular user of Frequentist statistics, I am actually open to the possibility that Bayesian just wins across the board.

• When you deal with objective probabilities, i.e. naturally stochastic processes. For instance, radioactive decay has nothing to do with your subjective beliefs or unknown information, or pretty much anything else. It just goes its own pace, and atoms truly randomly break up. Commented Feb 4, 2016 at 16:31
• See this recent question that unfortunately ended up closed as too broad (I voted to reopen but it never was): stats.stackexchange.com/questions/192572. You are asking almost exactly the same thing. Check the answer there. Commented Feb 4, 2016 at 16:34
• @Aksakal: I would love to have this discussion but it's off-topic and we will be told off so I shut up (and calculate). Commented Feb 4, 2016 at 16:51
• "Bayesians address the question everyone is interested in by using assumptions no-one believes, while frequentists use impeccable logic to deal with an issue of no interest to anyone" -- Louis Lyons Commented Feb 4, 2016 at 18:42
• @jsakaluk, notice how Bayesians' strongholds are areas where there's no enough data or when the processes are unstable, i.e. social sciences, psudo sciences, life sciences etc. There's no need to be Bayesian in quantum mechanics or most of physics. Granted, you can be Bayesian there too, it's just your inferences will be no different from frequentist's Commented Feb 4, 2016 at 18:46

Here's five reasons why frequentists methods may be preferred:

• Faster. Given that Bayesian statistics often give nearly identical answers to frequentist answers (and when they don't, it's not 100% clear that Bayesian is always the way to go), the fact that frequentist statistics can be obtained often several orders of magnitude faster is a strong argument. Likewise, frequentist methods do not require as much memory to store the results. While these things may seem somewhat trivial, especially with smaller datasets, the fact that Bayesian and Frequentist typically agree in results (especially if you have lots of informative data) means that if you are going to care, you may start caring about the less important things. And of course, if you live in the big data world, these are not trivial at all.

• Non-parametric statistics. I recognize that Bayesian statistics does have non-parametric statistics, but I would argue that the frequentist side of the field has some truly undeniably practical tools, such as the Empirical Distribution Function. No method in the world will ever replace the EDF, nor the Kaplan Meier curves, etc. (although clearly that's not to say those methods are the end of an analysis).

• Less diagnostics. MCMC methods, the most common method for fitting Bayesian models, typically require more work by the user than their frequentist counter part. Usually, the diagnostic for an MLE estimate is so simple that any good algorithm implementation will do it automatically (although that's not to say every available implementation is good...). As such, frequentist algorithmic diagnostics is typically "make sure there's no red text when fitting the model". Given that all statisticians have limited bandwidth, this frees up more time to ask questions like "is my data really approximately normal?" or "are these hazards really proportional?", etc.

• Valid inference under model misspecification. We've all heard that "All models are wrong but some are useful", but different areas of research take this more or less seriously. The Frequentist literature is full of methods for fixing up inference when the model is misspecified: bootstrap estimator, cross-validation, sandwich estimator (link also discusses general MLE inference under model misspecification), generalized estimation equations (GEE's), quasi-likelihood methods, etc. As far as I know, there is very little in the Bayesian literature about inference under model misspecification (although there's a lot of discussion of model checking, i.e., posterior predictive checks). I don't think this just by chance: evaluating how an estimator behaves over repeated trials does not require the estimator to be based on a "true" model, but using Bayes theorem does!

• Freedom from the prior (this is probably the most common reason for why people don't use Bayesian methods for everything). The strength of the Bayesian standpoint is often touted as the use of priors. However, in all of the applied fields I have worked in, the idea of an informative prior in the analysis is not considered. Reading literature on how to elicit priors from non-statistical experts gives good reasoning for this; I've read papers that say things like (cruel straw-man like paraphrasing my own) "Ask the researcher who hired you because they have trouble understanding statistics to give a range that they are 90% certain the effect size they have trouble imagining will be in. This range will typically be too narrow, so arbitrarily try to get them to widen it a little. Ask them if their belief looks like a gamma distribution. You will probably have to draw a gamma distribution for them, and show how it can have heavy tails if the shape parameter is small. This will also involve explaining what a PDF is to them."(note: I don't think even statisticians are really able to accurately say a priori whether they are 90% or 95% certain whether the effect size lies in a range, and this difference can have a substantial effect on the analysis!). Truth be told, I'm being quite unkind and there may be situations where eliciting a prior may be a little more straightforward. But you can see how this is a can of worms. Even if you switch to non-informative priors, it can still be a problem; when transforming parameters, what are easily mistaken for non-informative priors suddenly can be seen as very informative! Another example of this is that I've talked with several researchers who adamantly do not want to hear what another expert's interpretation of the data is because empirically, the other experts tend to be over confident. They'd rather just know what can be inferred from the other expert's data and then come to their own conclusion. I can't recall where I heard it, but somewhere I read the phrase "if you're a Bayesian, you want everyone to be a Frequentist". I interpret that to mean that theoretically, if you're a Bayesian and someone describes their analysis results, you should first try to remove the influence of their prior and then figure out what the impact would be if you had used your own. This little exercise would be simplified if they had given you a confidence interval rather than a credible interval!

Of course, if you abandon informative priors, there is still utility in Bayesian analyses. Personally, this where I believe their highest utility lies; there are some problems that are extremely hard to get any answer from in using MLE methods but can be solved quite easily with MCMC. But my view on this being Bayesian's highest utility is due to strong priors on my part, so take it with a grain of salt.

• (+1) Nice answer, though I'm assuming you meant do not require as much memory to store the results? Commented Feb 5, 2016 at 14:23
• In terms of freedom from priors: are you saying that the less you have to think about and understand your problem, the better? I know several software vendors who would like to talk to you, so you can point-n-click -- or better yet, one-click -- and have an answer to any problem you can imagine! Heck, you don't even need a problem, just feed your data into their website and they'll find all possible problems and solve them, toot sweet ! (Sorry, couldn't resist answering with a cruel straw-man-like comment.) Commented Feb 7, 2016 at 16:57
• @Wayne: I know you are joking, but that's 100% correct. Statistics is a tool for answering real world problems. I really want to emphasize that it is a tool, not a final product. Regardless of what side of the throughly hashed out "Frequentist vs Bayesian" argument (I sit on "whichever gives me the best answer to my question" side, which means I like both for different problems), there's no arguing that ease of use is a very real utility for any tool. Commented Feb 7, 2016 at 17:13
• @CliffAB: Ease-of-use is important, and as you say if the results are of equal quality, why choose harder-to-use? At the same time, thinking about, making explicit, and understanding priors (not Bayesian, I mean literally the priors that every scientist, every field, and every study has) is critical to good science. Bayesian statistics is explicit and forces you to think about and understand some of these issues. To the extent that this is not merely pedantic inconvenience, it's arguably good, and so its opposite isn't slam-dunk good either. Commented Feb 7, 2016 at 17:37
• It’s hard to agree with any part of this answer. Just considering nonparametric tests, Bayesian semiparametric models contain common rank tests as special cases, and Bayesian nonparametric tests work better than frequentist ones. See this and this. Commented May 5 at 12:27

A few concrete advantages of frequentist statistics:

• There are often closed-form solutions to frequentist problems whereas you would need a conjugate prior to have a closed form solution in the Bayesian analogue. This is useful for a number of reasons - one of which is computation time.
• A reason that'll, hopefully, eventually go away: laymen are taught frequentists statistics. If you want to be understood by many, you need to speak frequentist.
• An "Innocent until proven guilty" Null Hypothesis Significance Testing (NHST) approach is useful when the goal is to prove someone wrong (I'm going to assume your right and show the data overwhelming suggests you're wrong). Yes, there are NHST analogues in Bayesian but I find the frequentists versions much more straight-forward and interpretable.
• There is no such thing as a truly uninformative prior which makes some people uncomfortable.
• (+1) Thanks--could you clarify the first point a bit? As someone not well-versed in Bayesian, the point you are making about the need for a "conjugate prior" (?) is lost on me a bit... Commented Feb 4, 2016 at 19:47
• I do not think you are interpreting the frequentist hypothesis test correctly. You just gave $P(H_0\;|\; Data)$, but the p-value is actually $P(Data\;|\;H_0)$. The correct interpretation of the p-value: given the null, there is only an $\alpha$% chance of getting a result as extreme or more extreme then that observed. This misinterpretation is brought up often when arguing for a Bayesian approach. Other than that I like your answer. Commented Feb 4, 2016 at 20:05
• @ZacharyBlumenfeld Thanks for pointing that out, I had Bayesian on my mind. Ill fix it now. Commented Feb 4, 2016 at 20:10
• @jsakaluk If the posterior and prior are the same distribution, the prior is said to be conjugate - guaranteeing a closed form posterior. For example, if our data is Bernoulli and we chose a Beta($\alpha$,$\beta$) prior then we know that the posterior is Beta($\alpha + \sum_{i=1}^n x_i$, $\beta + n - \sum_{i=1}^n x_i\!$) without having to do any simulation, sampling, or intense computation. Commented Feb 4, 2016 at 20:24

The most important reason to use Frequentist approaches, which has surprisingly not yet been mentioned, is error control. Very often, research leads to dichotomous interpretations (should I do a study building on this, or not? Should be implement an intervention, or not?). Frequentist approaches allow you to strictly control your Type 1 error rate. Bayesian approaches don't (although some inherit the universal bound from likelihood approaches, but even then, error rates can be quite high in small samples and with relatively low thresholds of evidence (e.g., BF > 3). You can examine Frequentist properties of Bayes factors (see for example, this paper Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences) but that's still a Frequentist approach. I think very often, researchers care more about error control than about quantifying evidence per se (relative to some specific hypothesis), and I think at the very least, everyone cares about error control to some extent, and thus the two approaches should be used complementarily.

• Good point. I am also thinking of group-sequential methods and other forms of multiple testing, where there seems (from my narrow point of view, which may have overlooked substantial parts of the literature) to have been a lack of interest on the Bayesian side (so far) in terms of getting some kind of error control. Of course in many circumstances Bayesian methods - particularly with somewhat skeptical priors or some kind of shrinkage through a hierarchical model do control errors somewhat to some unquantifiable degree, but a lot more thinking has been done on the frequentist side there. Commented Feb 6, 2016 at 7:30
• (+1) I really like this point...as it is the reason I am philosophically a frequentist....when we do stats to help with inference, then we want our inferences to be more accurate (i.e., less error) than blind guessing. In fact, if I care at all about my inferences being actually true or false (in the sense of being validated by follow-on studies), then error rates are very important. I just cant feel comfortable with Bayesian probability (however, the methods themselves are very useful as sensible "regularlized estimators" for a quantity when sample size is small...think Agresit-Coull)
– user75138
Commented Apr 12, 2016 at 13:05
• This sounds more like decision theory than bayes/frequentist comparison. Also, with bayesian approach you don't need to worry about stopping rules....I also understand that bayes may achieve a better "balance" between type 1 and type 2 error rates.... Commented Dec 9, 2017 at 13:50
• This is entirely misleading. What you are discussion has nothing to do with “errors” as detailed here. Commented May 5 at 12:28

I think one of the biggest questions, as a statistican, you have to ask yourself is whether or not you believe in, or want to adhere to, the likelihood principle. If you don't believe in the likelihood principle then I think the frequentist paradigm to statistics can be extremely powerful, however, if you do believe in the likelihood principle, then (I believe) you most certainly have to espouse the Bayesian paradigm in or to not violate it.

In case you are unfamiliar with it, what the likelihood principle tells us is the following:

The Likelihood Principle: In making inferences or decisions about $\theta$ after some data $\mathbf{x}$ is observed, all relevant experimental information is contained in the likelihood function: $$\ell(\theta;\mathbf{x})=p(\mathbf{x}|\theta)$$ where $\mathbf{x}$ corresponds to the data observed and is thus fixed.

Furthermore, if $\mathbf{x}$ and $\mathbf{y}$ are two sample points such that $\ell(\theta;\mathbf{x})$ is proportional to $\ell(\theta;\mathbf{y})$, that is, there exists a constant $C(\mathbf{x},\mathbf{y})$ such that

$$\ell(\theta;\mathbf{x})=C(\mathbf{x},\mathbf{y})\ell(\theta;\mathbf{y})\hspace{.1in}\text{for all }\theta,$$

then the conclusions drawn from $\mathbf{x}$ and $\mathbf{y}$ should be identical.\

Note that the constant $C(\mathbf{x},\mathbf{y})$ above may be different for different $(\mathbf{x},\mathbf{y})$ pairs but $C(\mathbf{x},\mathbf{y})$ does not depend on $\theta$.

In the special case of $C(\mathbf{x},\mathbf{y})=1$, the Likelihood Principle states that if two sample points result in the same likelihood function, then they contain the same information about $\theta$. But the Likelihood Principle goes further. It states that even if two sample points have only proportional likelihoods, then they contain equivalent information about $\theta$.

Now, one of the draws of Bayesian statistics is that, under proper priors, the Bayesian paradigm never violates the likelihood principle. However, there are very simple scenarios where the frequentist paradigm will violate the likelihood principle.

Here is a very simple example based on hypothesis testing. Consider the following:

Consider an experiment where 12 Bernoulli trials were run and 3 successes were observed. Depending on the stopping rule we could characterize the data as the following:

• Binomial Distribution: $X|\theta\sim\text{Bin}(n=12,\theta)$ and Data: $x=3$
• Negative Binomial Distribution: $Y|\theta\sim\text{NegBin}(k=3,\theta)$ and Data: $y=12$

And thus we would obtain the following likelihood functions: \begin{align} \ell_1(\theta;x=3)&=\binom{12}{3}\theta^3(1-\theta)^9\\ \ell_2(\theta;y=12)&=\binom{11}{2}\theta^3(1-\theta)^9\\ \end{align} which implies that $$\ell_1(\theta;x)=C(x,y)\ell_2(\theta,y)$$ and thus, by the Likelihood Principle, we should obtain the same inferences about $\theta$ from either likelihood.

Now, imagine testing the following hypotheses from the frequentist paradigm $$H_o:\theta\geq\frac{1}{2}\hspace{.2in}\text{versus}\hspace{.2in}H_a:\theta<\frac{1}{2}$$

For the Binomial model we have the following: \begin{align} \text{p-value}&=P\left(X\leq 3|\theta=\frac{1}{2}\right)\\ &=\binom{12}{0}\left(\frac{1}{2}\right)^{12}+\binom{12}{1} \left(\frac{1}{2}\right)^{12}+ \binom{12}{2}\left(\frac{1}{2}\right)^{12}+\binom{12}{3}\left(\frac{1}{2}\right)^{12}=0.0723 \end{align}

Notice that $\binom{12}{3}\left(\frac{1}{2}\right)^{12}=\ell_1(\frac{1}{2};x=3)$ but the other terms do not satisfy the likelihood principle.

For the Negative Binomial model we have the following: \begin{align} \text{p-value}&=P\left(Y\geq 12|\theta\frac{1}{2}\right)\\ &=\binom{11}{2}\left(\frac{1}{2}\right)^{12}+\binom{12}{2}\left(\frac{1}{2}\right)^{12}+ \binom{13}{2}\left(\frac{1}{2}\right)^{12}+...=0.0375 \end{align}

From the above p-value calculations we see that in the Binomial model we would fail to reject $H_o$ but using the Negative Binomial model we would reject $H_o$. Thus, even though $\ell_1(\theta;x)\propto\ell_2(\theta;y)$ there p-values, and decisions based on these p-values, do not coincide. This p-value argument is one often used by Bayesians against the use of Frequentist p-values.

Now consider again testing the following hypotheses, but from the Bayesian paradigm $$H_o:\theta\geq\frac{1}{2}\hspace{.2in}\text{versus}\hspace{.2in}H_a:\theta<\frac{1}{2}$$

For the Binomial model we have the following: \begin{align} P\left(\theta\geq\frac{1}{2}|x\right)=\int_{1/2}^1\pi(\theta|x)dx=\int_{1/2}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \bigg/\int_{0}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \end{align}

Similarly, for the Negative Binomial model we have the following: \begin{align} P\left(\theta\geq\frac{1}{2}|y\right)=\int_{1/2}^1\pi(\theta|x)dx=\int_{1/2}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \bigg/\int_{0}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \end{align}

Now using Bayesian decision rules, pick $H_o$ if $P(\theta\geq\frac{1}{2}|x)>\frac{1}{2}$ (or some other threshold) and repeat similarly for $y$.

However, $P\left(\theta\geq\frac{1}{2}|x\right)=P\left(\theta\geq\frac{1}{2}|y\right)$ and so we arrive at the same conclusion and thus this approach satisfies the likelihood Principle.

And so to conclude my ramblings, if you don't care about the likelihood principle then being frequentist is great! (If you can't tell, I'm a Bayesian :) )

• I appreciate the clearly thoughtful (and likely time-consuming) response, but I feel like this answer is a bit of a departure from the "answers... conveyed as accessibly as possible..." mandate of the question. Commented Feb 6, 2016 at 3:19
• @jsakaluk I guess what I was aiming for, and wanted to be sure to back up the argument, is that if you are willing to overlook certain things that many applied statisticians take for granted all the time, i.e., the likelihood principle, then using the frequentist paradigm can be a much simpler alternative to the Bayesian paradigm. However, if you cannot then you will most likely have to find alternatives. Commented Feb 6, 2016 at 8:15
• @RustyStatistician The likelihood principle is a central tenet for likelihoodists. Likelihoodists are not Bayesian at all. I posted links in my answer. Your claim "if you do believe in the likelihood principle, then (I believe) you most certainly have to espouse the Bayesian paradigm" is false.
– stan
Commented Feb 7, 2016 at 13:24
• @Stan I agree with you that yes likelihoodists believe in the likelihood principle sure. But I would find it extremely hard to believe that if you ask any Bayesian if they believe in adhering to the likelihood principle that they would say no they don't (that's just my opinion you don't have to agree). Commented Feb 7, 2016 at 16:13
• The roles of likelihood principle (LP), conditionality principle (CP) and sufficiency principle (SP) in inference are not simple..this is because these principles relate to evidence (as presented by the data), whereas inference involves going beyond the evidence. This is always risky, but necessary to make progress. See Birnbaums Theorem (discussed here...I don't necessarily agree with the rest of the paper): arxiv.org/abs/1302.5468
– user75138
Commented Apr 12, 2016 at 13:29

Personally I'm having difficulty thinking of a situation where the frequentist answer would be preferred over a Bayesian one. My thinking is detailed here and in other blog articles on fharrell.com about problems with p-values and null hypothesis testing. Frequentists tend to ignore a few fundamental problems. Here is just a sample:

• Outside of the Gaussian linear model with constant variance and a few other cases, the p-values that are computed are of unknown accuracy for your dataset and model
• When the experiment is sequential or adaptive, it is often the case that a p-value can't even be computed and one can only set an overall $$\alpha$$ level to achieve
• Frequentists seem happy to not let the type I error go below, say, 0.05 no matter now the sample size grows
• There is no frequentist prescription for how multiplicity corrections are formed, leading to an ad hoc hodge-podge of methods

Regarding the first point, one commonly used model is the binary logistic model. Its log likelihood is very non-quadratic, and the vast majority of confidence limits and p-values computed for such models are not very accurate. Contrast that with the Bayesian logistic model, which provides exact inference.

Others have mentioned error control as a reason for using frequentist inference. I do not think this is logical, because the error to which they refer is the long-run error, envisioning a process in which thousands of statistical tests are run. A judge who said "the long run false conviction probability in my courtroom is only 0.03" should be disbarred. She is charged with having the highest probability of making the correct decision for the current defendent. On the other hand one minus the posterior probability of an effect is the probabiity of zero or backwards effect and is the error probability we actually need.

• "There is no frequentist prescription for how multiplicity corrections are formed, leading to an ad hoc hodge-podge of methods." On the other hand, I've never seen a Bayesian do multiplicity corrections at all. Andrew Gelman even proudly declares that he never uses them. For instance, I've seen people report marginal 95% credible intervals for $\theta_1, \ldots, \theta_k$, but the joint credibility of those $k$ intervals is not 95%. Nor is it obvious how best to address this. Do you have any advice or examples? Commented Feb 20, 2019 at 20:13
• There are many, many reasons for not doing multiplicity corrections in general. And you only need to do them when they affect the likelihood function. The evidence for A should not depend on the evidence you have for B unless A and B are correlated. What works for the majority of cases is pre-specifying the reporting order when you have multiple hypotheses, in addition to forming priors before looking at the data. Commented May 5 at 12:31
• Thank you, but I'm not sure I follow. Let's say $\theta_1$ and $\theta_2$ are uncorrelated & independent. Say you have 95% posterior belief that $\theta_1 \in [1,3]$ and you also have 95% posterior belief that $\theta_2 \in [10,15]$. If I ask what's your posterior belief that both events $\theta_1 \in [1,3]$ AND $\theta_2 \in [10,15]$ both hold simultaneously, will you still say 95%? Commented May 5 at 19:01
• That’s a very fine posterior probability to compute, and a joint prob of 0.85 dictates that the individual probs are $\geq 0.95$. Commented May 6 at 13:55
• Why would the joint probability be 0.85, not 0.95^2 = 0.9025 since the events are independent? But regardless of that detail, this seems to me exactly analogous to multiple comparisons. I concede that quantifying "the evidence for both A and B" is not always of interest... but when it is, a 95% joint credible region is not the same as two 95% credible intervals reported simultaneously. Either the two intervals change (typically become wider) so their joint coverage stays 95%, or the joint coverage changes (typically is less) from the 95% coverage for each marginal interval. Commented May 6 at 20:32

You and I are both scientists, and as scientists, are chiefly interested in questions of evidence. For that reason, I think Bayesian approaches, when feasible, are preferable.

Bayesian approaches answer our question: What is the strength of evidence for one hypothesis over another? Frequentist approaches, on the other hand, do not: They report only whether the data are weird given one hypothesis.

That said, Andrew Gelman, notable Bayesian, seems to espouse the use of p-values (or p-value-like graphical checks) as a check for errors in model specification. You can see an allusion to this approach in this blog post.

His approach, as I understand it, is something like a two-step process: First, he asks the Bayesian question of what is the evidence for one model over the other. Second, he asks the Frequentist question of whether the preferred model actually looks at all plausible given the data. It seems like a reasonable hybrid approach to me.

• Although the link to the Gelman blog should remain valid, it won't be "today's" after midnight. Edited accordingly. Commented Feb 4, 2016 at 17:51
• I strongly disagree with the notation that frequentist approaches do not measure evidence, and that this is solely in the Bayesian world. You are leaving out the origin of hypothesis testing, such as the LR test, measures the evidence of one hypothesis against the evidence for the other. Commented Feb 5, 2016 at 6:15
• (+1) to @CliffAB - for everyone thinking about "frequentist" statistics, please, please look up "likelihood ratio", "Birnbaum's Theorem", and perhaps read a bit of Royall....don't jump to straw-man arguments involving NHST -- which, by the way, hasn't seemed to dampen scientific progress despite its supposedly catastrophic flaws....that's because statisticians are not carbon-based MINITAB programs...they THINK [yes, doing statistics is actually a profession, just like medicine, or economics, or auto-mechanics,...you can't just read a book, try a formula, and expect truth to land in your lap].
– user75138
Commented Apr 12, 2016 at 13:12
• @Bey: Personally, I do believe that p-values have done some dampening to the scientific process (in that biologists are forced to become part time statisticians to publish papers, reducing the time they get to be biologists), but I don't don't think the alternatives to p-values in any way reduce this issue! I feel that the issue of p-values is not their theoretic background, but their ease of use by non-statisticians. Posterior probabilities, (for example) I think make that particular issue worse, rather than better. Commented Apr 12, 2016 at 19:56
• @CliffAB couldn't agree more...didn't think of it from that side..but that's just the nature of publishing I guess...unless research departments can afford to have staff statisticians. Any statistical tool can be misused by one not knowledgeable in its use...pity statistical tools seem so easy to use...
– user75138
Commented Apr 12, 2016 at 19:58

Many people do not seem aware of a third philosophical school: likelihoodism. AWF Edwards's book, Likelihood, is probably the best place to read up on it. Here is a short article he wrote.
Likelihoodism eschews p-values, like Bayesianism, but also eschews the Bayesian's often dubious prior. There is an intro treatment here as well.

• There's algorithmic probability approach by Vovk, developed from Kolmogorov's ideas. Commented Feb 4, 2016 at 19:18
• "Many people do not seem aware of a third philosophical school: likelihoodism" I do not think that this sentence is true in 2016...
– Tim
Commented Feb 4, 2016 at 21:28
• @Tim, although everybody I know is familiar with frequentism and Bayesianism, I have never met anyone who had heard of likelihoodism. The original questioner seems to be like my colleagues who were trained in frequentism and are becoming increasingly interested in Bayesianism. Perhaps most people who read my answer above think I am referring to maximum likelihood estimation or testing hypotheses using likelihood ratios. Nope! I suggest Yudi Pawitan and this lecture
– stan
Commented Feb 5, 2016 at 8:06
• None of those approaches is religion, so there is not much to believe, they are just helpful for certain kind of problems, and some of the approaches are better suited for some problems and other for others :)
– Tim
Commented Feb 5, 2016 at 8:50
• (+1) for mentioning likelihood school and for the comment regarding Pawitan. Pawitan's book "In All Likelihood" dramatically widened and enhanced by statistical practice...I was also only aware of Bayes vs Frequentism. He tackles a lot of philosophical and methodological aspects of Bayes, "classical" frequentism, and, of course, covers the pure likelihood school. Just a great book for becoming a more sophisticated user of stats...regardless of your philosophical inclinations.
– user75138
Commented Apr 12, 2016 at 13:19

Frequentist tests focus on falsifying the null hypothesis. However, Null Hypothesis Significance Testing (NHST) can also be done from a Bayesian perspective, because in all cases NHST is simply a calculation of P( Observed Effect | Effect = 0 ). So, it's hard to identify a time when it would be necessary to conduct NHST from a frequentist perspective.

That being said, the best argument for conducting NHST using a frequentist approach is ease and accessibility. People are taught frequentist statistics. So, it's easier to run a frequentist NHST, because there are many more statistical packages that make it simple to do this. Similarly, it is easier to communicate the results of a frequentist NHST, because people are familiar with this form of NHST. So, I see that as the best argument for frequentist approaches: accessibility to stats programs that will run them and ease of communication of results to colleagues. This is just cultural, though, so this argument could change if frequentist approaches lose their hegemony.

• The comments about what Fisher thought seem overwrought here unless you can provide exact quotations. The null hypothesis is a device as a part of a significance test to try to discourage scientists from over-interpreting results from small samples. Fisher was as keen as anybody else that scientists should use statistics to do good science; he was himself a very serious contributor to genetics. Commented Feb 4, 2016 at 17:56
• I agree completely, and so I edited the answer to remove the speculation about Fisher's mental state. Commented Feb 4, 2016 at 18:38

One of the biggest disadvantages of frequentist approaches to model building has always been, as TrynnaDoStats notes in his first point, the challenges involved with inverting big closed-form solutions. Closed-form matrix inversion requires that the entire matrix be resident in RAM, a significant limitation on single CPU platforms with either large amounts of data or massively categorical features. Bayesian methods have been able to work around this challenge by simulating random draws from a specified prior. This has always been one of the biggest selling points of Bayesian solutions, although answers are obtained only at a significant cost in CPU.

Andrew Ainslie and Ken Train, in a paper from about 10 years ago that I have lost the reference to, compared finite mixture (which are frequentist or closed form) with Bayesian approaches to model-building and found that across a wide range of functional forms and performance metrics, the two methods delivered essentially equivalent results. Where Bayesian solutions had an edge or possessed greater flexibility were in those instances where the information was both sparse and very high-dimensional.

However, that paper was written before "divide and conquer" algorithms were developed that leverage massively parallel platforms, e.g., see Chen and Minge's paper for more about this http://dimacs.rutgers.edu/TechnicalReports/TechReports/2012/2012-01.pdf

The advent of D&C approaches has meant that, even for the hairiest, sparsest, most high dimensional problems, Bayesian approaches no longer have an advantage over frequentist methods. The two methods are at parity.

This relatively recent development is worth noting in any debate about the practical advantages or limitations of either method.

• I think this is a nice addition to the discussion (+1) but I find it hard to follow. It really, really, really postpones its punch-line... Maybe you could reorganise it a bit? :) Commented Feb 5, 2016 at 18:35
• @user11852 You don't say that the post fails to communicate something useful whereas you do find the development of the logic not up to journalistic standards. Since this thread has gone "community," I'm not too inclined (motivated?) to work on reorganizing it around your suggestion. It can stand as is. But thank you anyway for the upvote and comment. Commented Feb 5, 2016 at 19:17
• 1.) Matrix inversion is often used for MLE estimation (which is only one of many frequentist methods), but not always. My work in MLE estimation involves optimization over often up to $n$ parameters (i.e. parameter space can grow linearly with sample size) and matrix inversion is absolutely not an option...but I still optimize the likelihood! 2.) Matrix inversion still happens all the time in Bayesian statistics, such as a block updater sampler. Commented Feb 7, 2016 at 17:24
• @CliffAB I was thinking of ANOVA-type inversion of the matrix of cross-products. Commented Feb 16, 2016 at 15:13
• @DJohnson: I see. But my point was that matrix inversion is orthogonal to frequentist vs bayesian methods; both camps use tools that do something very similar (at least in terms of computational costs) in many of their methods. Commented Feb 16, 2016 at 17:37

• The fundamental difference between the bayesian and frequentist statistician is that the bayesian is willing to extend the tools of probability to situations where the frequentist wouldn't.

• More specifically, the bayesian is willing to use probability to model the uncertainty in her own mind over various parameters. To the frequentist, these parameters are scalars (albeit scalars where the statistician does not know the true value). To the Bayesian, various parameters are represented as random variables! This is extremely different. The Bayesian's uncertainty over parameters valeus is represented by a prior.
• In Bayesian statistics, the hope is that after observing data, the posterior overwhelms the prior, that the prior doesn't matter. But this often isn't the case: results may be sensitive to the choice of prior! Different Bayesians with different priors need not agree on the posterior.

A key point to keep in mind is that statements of the frequentist statistician are statements that any two Bayesians can agree on, regardless of their prior beliefs!

The frequentist does not comment on priors or posteriors, merely the likelihood.

The statements of the frequentist statistician in some sense are less ambitious, but the bolder statements of the Bayesian can significantly rely on the assignment of a prior. In situations where priors matter and where there is disagreement on priors, the more limited, conditional statements of frequentist statistics may stand on firmer ground.

The goal of much research is not to reach a final conclusion, but just to obtain a little more evidence to incrementally push the community's sense of a question in one direction.

Bayesian statistics are indispensable when what you need is to evaluate a decision or conclusion in light of the available evidence. Quality control would be impossible without Bayesian statistics. Any procedure where you need to take some data and then act on it (robotics, machine learning, business decision making) benefits from Bayesian statistics.

But a lot of researchers are not doing that. They are running some experiments, collecting some data, and then saying "The data points this way", without really worrying too much about whether that's the best conclusion given all the evidence others have gathered so far. Science can be a slow process, and a statement like "The probability that this model is correct is 72%!" is often premature or unnecessary.

This is appropriate in a simple mathematical way, too, because frequentist statistics often turn out to be mathematically the same as the update-step of a Bayesian statistic. In other words, while Bayesian statistics is (Prior Model, Evidence) → New Model, frequentist statistics is just Evidence, and leaves it to others to fill in the other two parts.

• Although much of this post is interesting, it consists of many unsupported opinions. Please consult our help center concerning what kinds of answers are expected on this site.
– whuber
Commented Feb 5, 2016 at 15:03
• @whuber I see. I have added one citation I can remember off the top of my head, but the rest I don't have citations for, so if it seems too unsupported I can delete it.
– Owen
Commented Feb 5, 2016 at 15:08
• I'm surprised that you mentioned quality control, since it seems like an area where the frequentist interpretation of probability (relative frequency over many trials) would be very natural: given that the factory is working correctly, how likely are we to see this many (or more) broken widgets? Could I push you to elaborate on what makes Bayesian statistics particularly useful for QC? Commented Feb 5, 2016 at 20:35
• The idea that frequentist statistics cannot not combine information from successive studies seems to ignore the field of meta-analyses. Commented Jun 8, 2016 at 23:03
• But note that it was David Spiegelhalter who documented the anti-conservatism of confidence intervals from the most widely used statistical method for doing meta-analysis (DerSimoneon & Laird), and the accuracy of Bayesian hierarchical models. Also, meta-analysis requires symmetric information whereas Bayes can used a wider variety of information sharing. Commented May 7 at 11:58

The actual execution of a Bayesian method is more technical than that of a Frequentist. By "more technical" I mean things like: 1) choosing priors, 2) programming your model in a BUGS/JAGS/STAN, and 3) thinking about sampling and convergence.

Obviously, #1 is pretty much not optional, by definition of Bayesian. Though with some problems and procedures, there can be reasonable defaults, somewhat hiding the issue from the user. (Though this can also cause problems!)

Whether #2 is an issue depends on the software you use. Bayesian statistics has a bent towards more general solutions than frequentist statistical methods, and tools like BUGS, JAGS, and STAN are a natural expression of this. However, there are Bayesian functions in various software packages that appear to work like the typical frequentist procedure, so this is not always an issue. (And recent solutions like the R packages rstanarm and brms are bridging this gap.) Still, using these tools is very similar to programming in a new language.

Item #3 is usually applicable, since the majority of real-world Bayesian applications are going to use MCMC sampling. (On the other hand, frequentist MLE-based procedures use optimization which may converge to a local minima or not converge at all, and I wonder how many users should be checking this and don't?)

As I said in a comment, I'm not sure that freedom from priors is actually a scientific benefit. It's certainly convenient in several ways and at several points in the publication process, but I'm not sure it actually makes for better science. (And in the big picture, we all have to be aware of our priors as scientists, or we'll suffer from all kinds of biases in our investigations, regardless of what statistical methods we use.)

• In regards to (3), many classical stats models (i.e. glm's) have concave log-likelihoods, so it's very rare that standard algorithms should fail, outside of extreme corner cases. In regards to non-concave problems (i.e. NNs), while these do require heavy concern about improper convergence (which is typically understood by users), these are (not coincidentially) also problems in which classic MCMC algorithms will horribly fail if only run for, say, one human's lifetime. However, it's generally less of a stretch to fix up the MCMC than the optimization algorithm! Commented Oct 5, 2017 at 21:27

Conceptually: I don't know. I believe Bayesian statistics is the most logical way to think but I coudn't justify why.

The advantage of frequentist is that it is easier for most people at elementary level. But for me it was strange. It took years until I could really clarify intellectually what a confidence interval is. But when I started facing practical situations, frequentist ideas appeared to be simple and highly relevant.

Empirically

The most important question I try to focus on nowadays is more about practical efficiency: personal work time, precision, and computation speed.

Personal work time: For basic questions, I actually almost never use Bayesian methods: I use basic frequentist tools and will always prefer a t-test over a Bayesian equivalent that would just give me a headache. When I want to know if I'm significantly better at tictactoe than my girlfriend, I do a chi-squared :-). Actually, even in serious work as a computer scientist, frequentist basic tools are just invaluable to investigate problems and avoid false conclusions due to random.

Precision: In machine learning where prediction matters more than analysis, there is not an absolute boundary between Bayesian and frequentist. MLE is a frequentist approcah: just an estimator. But regularized MLE (MAP) is a partially Bayesian approach: you find the mode of the posterior and you don't care for the rest of the posterior. I don't know of a frequentist justification of why use regularization. Practically, regularization is sometimes just inevitable because the raw MLE estimate is so overfitted that 0 would be a better predictor. If regularization is agreed to be a truly Bayesian method, then this alone justifies that Bayes can learn with less data.

Computation speed: frequentist methods are most often computationally faster and simpler to implement. And somehow regularization provides a cheap way to introduce a bit of Bayes in them. It might be because Bayesian methods are still not as optimized as they could. For example, some LDA implementations are fast nowadays. But they required very hard work. For entropy estimations, the first advanced methods were Bayesian. They worked great but soon frequentist methods were discovered and take much less computation time... For computation time frequentist methods are generally clearly superior. It is not absurd, if your are a Bayesian, to think of frequentist methods as approximations of Bayesian methods.

• "I don't know of a frequentist justification of why [to] use regularization". That's easy; under repeated trials, it has shown to lower out-of-sample error. Commented Feb 3, 2019 at 4:20

One type of problem in which a particular Frequentist based approach has essentially dominated any Bayesian is that of prediction in the M-open case.

What does M-open mean?

M-open implies that the true model that generates the data does not appear in the set of models we are considering. For example, if the true mean of $$y$$ is quadratic as a function of $$x$$, yet we only consider models with the mean a linear function of $$x$$, we are in the M-open case. In other words, model miss-specification results in an M-open case.

In most cases, this is a huge problem for Bayesian analyses; pretty much all theory that I know about relies on the model being correctly specified. Of course, as critical statisticians, we should think that our model is always misspecified. This is quite an issue; most of our theory is based on the model being correct, yet we know it never is. Basically, we're just crossing our fingers hoping that our model is not too incorrect.

Why do Frequentist methods handle this better?

Not all do. For example, if we use standard MLE tools for creating the standard errors or building prediction intervals, we're not better off than using Bayesian methods.

However, there is one particular Frequentist tool that is very specifically intended for exactly this purpose: cross validation. Here, in order to estimate how well our model will predict on new data, we simply leave of some of the data when fitting the model and measure how well our model predicts the unseen data.

Note that this method is completely ambivalent to model miss-specification, it merely provides a method for us to estimate for how well a model will predict on new data, regardless of whether the model is "correct" or not.

I don't think it's too hard to argue that this really changes the approach to predictive modeling that's hard to justify from a Bayesian perspective (prior is supposed to represent prior knowledge before seeing data, likelihood function is the model, etc.) to one that's very easy to justify from a Frequentist perspective (we chose the model + regularization parameters that, over repeated sampling, leads to the best out of sample errors).

This has completely revolutionized how predictive inference is done. I don't think any statistician would (or at least, should) seriously consider a predictive model that wasn't built or checked with cross-validation, when it's available (i.e., we can reasonable assume observations are independent, not trying to account for sampling bias, etc.).

• Cross validation doesn't seem like an inherent problem for Bayesian inference. Rather cross validation is computationally expensive so doing an MCMC for every fold would be very expensive. These days I see people using PSIS and WAIC, which circumvent doing CV. Commented May 26 at 4:36
• @Galen: Cross validation is by definition a frequentist method. You are correct that you could use a Frequentist method to evaluate a Bayesian model but that doesn't make CV Bayesian. Commented May 26 at 20:30
• Feel free to spell it out explicitly for me; could be that my brain just needs more coffee. It doesn't seem like it is required to assume asymptotic relative frequency to evaluate a Bayesian model against another sample. Does out-of-sample evaluation really need to mean infinite/asymptotic generalization? Commented May 26 at 20:35
• (Note I am not trying to say that CV is Bayesian; only unsure that it is inherently frequentist) Commented May 26 at 20:41
• Cross validation makes no sense from a pure Bayesian viewpoint: anything that isn't the Bayesian posterior predictive density is naive. CV judges a model/estimating procedure only by the distribution of it's estimations relative to true labels, which is essentially the definition of frequentist methods. Commented May 27 at 4:36

Unfortunately I'm late to the party, and I see some good answers here already (particularly the accepted one by Cliff AB), but I feel myself drawn into writing another answer because (a) there are currently important aspects missing and (b) I am somewhat disconcerted by the large number of answers that seem to imply that Bayesian is in principle always better except that a frequentist solution in some cases may be faster and simpler.

When I think about these things, I always start from acknowledging that mathematical and in particular statistical models are idealisations that are not true in reality (as of course George Box said long ago). Still, we should understand how exactly the models, as far as we use them as tools for thinking, are connected with the reality we want to model.

A first thing to note is that there is a distinction between epistemic probability (i.e., probability modelling somebody's state of belief/knowledge) and aleatory probability (i.e., probability modelling a real process that generates data). Furthermore, within epistemic probability there is a distinction between subjectivists (models refer to belief/knowledge of a specific individual) and objectivists (models refer to scientifically secured knowledge only, and what beliefs rationally follow from it). Usually Bayesian statistics are thought to use epistemic probability, whereas frequentists are thought to use aleatory probability, but this is a simplification that is to some extent misleading. Personally I think that terminology is used better and clearer saying that frequentism refers to a specific aleatory interpretation of probability (as with epistemic probability, there are alternatives), which does not necessarily imply that frequentists have to use methods traditionally called "frequentist" such as significance tests and confidence intervals - I call these "CFI" for "classical frequentist inference" in the following). In fact people can apply Bayesian methods even implying a frequentist interpretation of probability, where prior distributions do not refer to belief or formalised scientific knowledge, but rather to a potentially real (frequentist) distribution of situations of similar kinds. Andrew Gelman has advertised this in several places, and there is a body of research concerned with the frequentist properties of Bayesian methods. In this sense, there isn't a contradiction or even a tension between frequentism and Bayesian statistics, although there is between epistemic and aleatory probability, and between Bayesian statistics and CFI.

The classical philosophical foundations of Bayesian statistics (due to de Finetti and Ramsey in particular, but also Jeffreys and Jaynes on the objectivist side), which many Bayesians allude to, are however epistemic. De Finetti said explicitly that (frequentist) "probability does not exist", and meant this as a motivation for epistemic probability - the argument is that frequentist probability is based on the idea that experiments can be infinitely and identically repeated, which in reality isn't possible. I agree with this (models are idealisations), however I do not think that epistemic probability does better in this respect, as in reality the assignment/elicitation of epistemic probabilities is very problematic and arguably does never correctly express a "true" state of belief/knowledge (see my paper linked above for more on this). Also the divide into subjectivist and objectivist epistemic probability highlights an essential problem with it, namely that (a) in science we want to be objective and/or rely on secured knowledge only (as far as possible), but (b) very obviously in any real situation existing knowledge and perceptions are so complex (and not necessarily consistent) that there isn't a unique automatic/objective way to assign probabilities, so that the subjective component is irreducible, and in several situations arguably attempts to reduce it can very well do more harm than good (whereas at the same time of course we should strive to do it in such a way that strong evidence, secured knowledge and good arguments get us as far as possible).

Reading literature where Bayesian statistics is applied to real problems, authors are rarely explicit about their interpretation of probability, and priors are very often very weakly if at all motivated, with very limited if any sensitivity analysis. This is no coincidence, given how hard it is to do it well. Of course one could state that "ideally the Bayesian approach would be optimal" where it not for the practical (and computational) difficulty, but I think this belies the essential philosophical difficulty and tension over what a prior distribution is supposed to do and can do. Effectively, saying something like "if existing knowledge could be unambiguously and uniquely expressed by a prior, then Bayesian statistics would be great" is no different in my view from saying "if we were dealing with real i.i.d. random processes that come with the possibility of infinite replication, then frequentist statistics would be great". But this is not how the world is.

But then much of the Bayesian literature seems to imply that it could secure statements such as "the probability that the true parameter is between 1 and 2 is 0.86". This talking implies that there is a "true parameter" (in many cases implicitly apparently understood to be frequentist), which there isn't (given that "all models are wrong"). Bayesians who do not imply true frequentist models should know this (although in practical statistics this is rarely explicitly said), and interpret their results accordingly, particularly acknowledging their subjective choices and how things could come out differently given other choices that may well look just as plausible (sensitivity analysis). In an "ideal Bayesian world" this is possible, and I have seen excellent presentations of Bayesian analyses where this is done appropriately. These presentations however will give the audience a good impression what the issues are, how much of a pain it can be to convincingly set up a prior and defend it, and why a frequentist may be happy avoiding this.

Now of course CFI doesn't make this problem fully go away, as frequentist models are idealisations, too, and not really true. However, in Bayesian statistics there is one additional layer of probability modelling coming in that we need to care for and where we may wonder about justification and misspecification (apart from the complexity issue mentioned in any other answers), on top of the "sampling model" that Bayesians of course also have to choose. And it doesn't help that many people who apply Bayesian statistics apparently tend to think about the sampling model in frequentist terms whereas the prior is taken to be epistemic (and if they cite philosophical justifications they tend to refer to de Finetti, Jaynes and the like, i.e., epistemic probability). This produces a problematic hybrid that is neither fully compatible with epistemic nor with aleatory probability, and my impression is that most people who use probability statements for supposedly true parameters don't exactly understand what these probabilities mean and refer to, not because it were impossible, but rather because they don't go into the philosophical depths far enough to realise what the problem is. (Bayesians sometimes say that these probability statements "give the users what they really want", but whereas many users may want such probability statements indeed, they very well may not want them to depend on prior choices that could well be made very differently, and that are as hard to justify as any prior choice is if you take it seriously rather than just saying "we use an informationless prior from this-or-that famous author as we don't have time to think for longer".)

I characterise the major difference between Bayesian statistics and CFI by the terms "inverse probability logic" and "compatibility logic". Inverse probability logic takes the data to redistribute probability in a pre-modelled space with prior probability assignment. Compatibility logic checks whether and to what extent data are compatible with certain models (usually defined by specific parameters).

Here are two reasons why I tend to prefer compatibility logic more often than not (even though I'm really a pluralist and I do think that there are situations in which a Bayesian approach can help a lot).

1. In compatibility logic it is clearer and easier to handle that the job of models isn't to be "true". In order to say that normal distributions with mean between 11.2 and 12.5 are compatible with my data at 95% confidence level in the sense of the data having the corresponding t-statistic values (they may of course be incompatible using other ways of checking compatibility such as whether all data points are integer numbers), I do not need to assume that the normal model is indeed true. (Of course I should be interested in what alternative different looking models could also "explain" the data, and how whatever model I have chosen will be helpful in finding out what I want to know.) Unfortunately this is rarely emphasised when teaching frequentism/CFI. Instead people are taught to "make sure that model assumptions are fulfilled", which of course is never possible. I'd say inverse probability logic has this problem in a worse way. Any Bayesian probability statement is only valid within the model, which seems to increase the reliance on the model (empirically I see fewer Bayesians questioning their model, and I think that this isn't a coincidence, despite it surely being possible for Bayesians, too - note also that using epistemic probability the model concerns your a priori thinking rather than the data, so the data are not a valid basis to challenge the model, unless you use Bayesian statistics with a frequentist interpretation of probability). Changing models based on the data will violate Bayesian coherence, i.e., the Bayesian approach formally assumes that the prior model is correct, whereas compatibility logic doesn't (although in fact that very same problem comes up in CFI as well as data-based model selection will bias CFI; this can however at least be analysed and handled transparently in frequentist terms, also there are nonparametric and robust methods).

2. Given that neither frequentist nor epistemic models should be thought of as potentially true, and that in science we are interested to find out about reality rather than our own thinking, I prefer to model reality directly instead of going via the epistemic detour modelling my (or even "agreed objective scientific") thinking about reality. This comes with the proviso that occasionally using a prior to incorporate useful knowledge that can help the analysis in a desirable way is certainly a good thing. In some cases the Bayesian can argue that there is some knowledge that should be modelled and can be introducing a prior but not otherwise. However in many data analytic situations I don't believe that this is the case, so I'm happy to model a data generating process only, rather than a parameter generating process about which normally the data in hand don't tell me anything (there are exceptions). So I'm happy to run methods that don't force me to do this, which I am required to do when running a Bayesian (inverse probability logic) analysis.

PS: The simplest answer to the title question probably is: "Bayesian is worse if the prior isn't any good", and making sure it's good enough to be worthwhile is not exactly a walk in the park. Even though my earlier discussion implies that it isn't quite as easy as that as there are further things that make a difference.

• I recall in McElreath's Statistical Rethinking that he makes use of a metaphor that models are golems (inspired by The Golem I think). Commented May 26 at 15:17
• I often wear different hats depending on the analysis (Bayes', Freq', neither), but I'll confess it would be nice to integrate different approaches into one unified hat. Commented May 26 at 15:28
• "...people can apply Bayesian methods even implying a frequentist interpretation of probability". I've thought we can do this with all Bayesian approaches. In particular, we probably want to calibrate a prior to the requirement that "Of the set of declarations one gives a $p$ prior probability to be true, approximately $p$ proportion of them should be in true in the long run". If that statement does not hold for you (or at least is an acceptably close approximation), why should I care about your posterior belief rather than what I get from a standard Frequentist analysis? Commented May 28 at 1:43
• @CliffAB I don't disagree with this, except that when people run Bayesian analyses in practice, you rarely find this kind of argument (it occasionally happens though), and particularly, very often either you don't have any hard evidence (data) that could tell you how to set up the prior so that it works in this way, or you have a reference class problem, i.e., it's hard to decide which data to include and which not to (e.g., "prior chosen according to results of similar already know studies" - what qualifies as "similar"?) Commented May 28 at 9:43