# Same results from bayesian and frequentist hypothesis testing? References needed

Are there any articles or books on the sameness the Bayesian and frequentist inference?

I'm looking for more than just a statement that 'in large samples and for uninformative priors the results are the same, period'. Like when p-value approach leads to the same conclusions as Bayes Factor (as a way of doing bayesian hypothesis testing) in a concrete, chosen by an author circumstance.

Have you read something? Please share, especially if you wrote some :-)

EDIT. I see my question might cause some misunderstanding, so let me explain my understanding of 'the same results'.
The same results of Bayesian and p-value approach is that the Bayes Factor and p-value both indicate the same decision: accept/reject null hypothesis, i.e. BF is above 1 or 3 (depending which scale has been chosen) and p-value is above 0.05.

• This doesn’t directly address your question, but I just wanted to point out that Bayes’ factors don’t take into account the prior, only the relative likelihood of the hypotheses given the evidence. – LmnICE Nov 6 '17 at 15:09
• With uniform prior $\pi(\theta) \propto 1$ and likelihood $f(x|\theta)$, posterior is $f(x|\theta)\;\pi(\theta) = f(x|\theta) \times 1$ so MAP it is the same as if you optimlized the likelihood function. – Tim Nov 6 '17 at 15:39
• @Tim yeah, right. sure, understood. But... I'm looking for the same results in hypothesis testing, not estimators :-) – Lil'Lobster Nov 6 '17 at 15:50
• @LmnICE I saw that some people equal these two things but in my understanding of Bayes Factor there are in the numerator and denominator $\int f(x\mid\theta)\cdot \pi(\theta)$, where $\pi(\theta)$ is a prior. – Lil'Lobster Nov 6 '17 at 15:53
• @Lil the relative likelihood of the hypotheses are all estimated using the same prior, so the priors cancel out. – Mark White Nov 6 '17 at 15:58

## 2 Answers

With flat priors and a large sample size (and many times without both of these things), (a) Bayesian and frequentist point-estimates will be virtually the same, and (b) the credible and confidence intervals will virtually cover the same range, leading to the same "hypothesis test" decision of "reject" or "fail to reject" the null. (Tim's comment shows this).

However, I will push back on this sentence:

'in large samples and for uninformative priors the results are the same, period'

There is no situation where the "results are the same." Period. Whether or not the confidence/credible interval includes zero is not the totality of the "results." It is one aspect of the results, and it is many times a trivial aspect.

Let's say you estimate a regression coefficient of b = 0.5. Frequentist p-values will tell you the probability of observing your data—or more extreme data—given that the null hypothesis is true. However, using MCMC, you can sample from the posterior to calculate the probability that the coefficient is greater than zero; that is, you have the probability of an alternative hypothesis, given the data.

Using Bayesian estimation, you can also compare probabilities—given the data—of different hypotheses (i.e., "Bayes factors"). You cannot do this in the frequentist paradigm.

As Tim mentioned, the mode of the posterior will be virtually the same thing as the point-estimate in a maximum likelihood, frequentist paradigm. However, in Bayesian estimation, we can look at the median or mean of the posterior, as well, as often our posteriors are not (and should not) be symmetric.

The argument you want to make, however, is a straw-person argument, as it does not make sense to select uniform priors. For example, nobody would choose a flat prior from $-\infty$ to $+\infty$ for a variance, because variances cannot be negative. Similarly, if all of our variables are standardized, we would expect there to be very few cases where coefficients are < -1 or > +1.

There are certain situations where—even if you have uniform priors and large sample sizes—maximum likelihood approaches will not converge. In this case, giving even a weakly-informative prior will allow you to get some type of estimation, instead of your maximum likelihood procedure just saying, "Could not converge." I have run into this with multilevel models.

In short: Having virtually the same point estimates and 95% credible/confidence interval upper and lower bounds does not mean that you have "the same results." We might make certain inductive, big-picture conclusions in the exact same way, but the totality of the results are not the same. I think the wealth of information you get from MCMC posteriors (i.e., you can estimate how uncertain you are about variances of coefficients!) is often overlooked when comparing frequentist and Bayesian methods.

This paper of Altham compares Bayesian and non-Bayesian (Fisher's exact) tests for equal proportions between two groups.

It turns out Fisher’s exact test can be understood as taking the prior Beta(1, 0) and Beta(0, 1) which is a surprising because using the uniform prior Beta(1,1) would have been more intuitive. Altham notes, “[Fisher’s exact test] seems to correspond to a strong prior belief in negative association of rows and columns.” She also shows that Fisher’s exact test is strictly conservative relative to the Bayesian analysis with uniform priors.