Expressing one-sided p values of directional hypothesis tests as Bayes factors

Question

Assume we want to test the directional hypothesis that $µ<0$. From a frequentist angle we use a one-tailed $t$-test and imagine we obtain a 1-sided $p$ value of say 0.07, which then would imply that our mean is not significantly smaller than 0 at the 0.05 level (some might still refer to it as "marginally significant").

Now I was reading the paper "Three Insights from a Bayesian Interpretation of the One-Sided P Value" by Marsman & Wagenmakers (2017), which notes that one-sided $p$ values of 1-tailed tests (i.e. of directional tests) can readily be converted to Bayes factors $K$ of a Bayesian test for direction under a symmetric prior around $µ$ based on the formula

$K = (1-p)/p$

In this example, this would give us a Bayes Factor $K$ of $µ$ being < 0 as opposed to it being >= 0 of (1-0.07)/0.07 = 13, which would imply that there would be 13 times more support for $µ$ being < 0 than $µ$ being >= 0. Following Jeffreys (1998) as well as Kass and Raftery (1995), this would be regarded as "strong evidence" for there being a negative effect as opposed to a positive one as this $K > 10$. In fact their cutoff for a strong effect $(K=10)$ would then correspond with a one-sided $p$ value of 1/11 = 0.09. Decisive evidence they consider to correspond to $K > 100$, which would then imply a one-sided $p$ value of 1/101 = 0.001. Am I correct stating this, and does it illustrate the arbitrariness and lack of concordance of such proposed cutoffs in the frequentist and Bayesian literature? That is, in this case implying a nonsignificant result from a frequentist angle, but a strong effect from a Bayesian one (under a noninformative prior, which I would be happy with in the context of my analysis)?

I was a little surprised by the lower bar that Bayesians would put on inferring a strong effect than frequentists, given that I've also regularly seen the argument that $p$ values around 0.05 would correspond to quite modest Bayes factors, and that the $p$ value cutoff for significance should therefore be taken as more stringent than 0.05. For example, in this paper, "Evidence From Marginally Significant t Statistics" (Johnson 2019). Am I correct that this difference in conclusion is due to the fact that Marsman & Wagenmakers (2017) consider directional tests (H0: effect in one direction, Ha: no effect or effect in the opposite direction), whereas Johnson (2019) consider tests for the existence of an effect (H0: there is no effect, Ha: there is an effect)? [This is what I gathered based on my reading of Held & Ott, 2018, "On p-Values and Bayes Factors" at least]

I was also wondering, given that maximum likelihood estimates coincide with the MAP estimate in a Bayesian analysis under a non-informative prior (and that Bayesian 95% credible intervals are numerically very close to 95% MLE confidence intervals derived under a non-informative prior), whether $p$ values deriving from maximum likelihood analyses (and most frequentist tests can be stated in MLE terms) can not always be restated as Bayes factors derived under a non-informative prior? If so, is MLE then not merely a special case of Bayesian analysis, and what's the big deal of pitting those two school of thoughts against each other (aside from their choice of a different prior, which in MLE is taken as non-informative based on the lack of information you have about this, and in Bayesian analysis can be specified in a somewhat more subjective way, taking into account your subjective beliefs)?

Answer 1

If so, is MLE then not merely a special case of Bayesian analysis, and what's the big deal of pitting those two school of thoughts against each other (aside from their choice of a different prior,

There is a difference but it is only not appearing when we consider the location parameter. That is an exception and not a general rule or principle that makes confidence intervals and credible intervals the same. In the case of a location parameter then the likelihood function (which equals the posterior for flat prior) $\frac{\partial F(x|\theta)}{\partial x}$ and the fiducial distribution $\frac{\partial F(x|\theta)}{\partial \theta}$ coincide.

If you consider other parameters then there is a difference between confidence/fiducial intervals and credible intervals, also when you consider a uniform prior.

The reason for the difference is that the fiducial distribution (and related confidence intervals) and posterior distribution (and related credible intervals) relate to different type of probabilities. The confidence intervals condition on the true parameter value, the credible intervals condition on the observation.

---

In this example, this would give us a Bayes Factor K of µ being < 0 as opposed to it being >= 0 of (1-0.07)/0.07 = 13

I guess that part of the difference here is not so much the use of a Bayesian approach versus a frequentist approach. After all, the credible intervals and confidence intervals coincide here.

I see two other issues.

• The use of Bayes factor for a composite hypothesis. Those K values and characterizations for strength of evidence are different when you consider densities or when you compare cumulative probabilities. Imagine a posterior distribution that is uniform between -0.05 to 0.95 that makes a K value of 19 for hypothesis $\theta > 0$ versus $\theta \leq 0$. However if we compare two point hypotheses in those ranges, e.g say -0.025 and +0.025 then the K value is 1.
• The p-value and choice of low significance cutoff value is similar to assigning a higher prior to the null hypothesis. If the Bayesian technique would use a prior with more density close to zero, then the Bayes factor would be smaller as well. The discrepancy between the p-value 0.07 that is not deemed significant and the Bayes factor of 13 that is considered strong evidence is in the prior information. If prior information places little weight on either hypothesis, then not so much evidence from data is necessary to get a strong factor.