Bayes factor (B) vs p-values: sensitive (H0/H1) vs insensitive data

Question

The question of a beginner in Bayesian stats. As far as I understand, it is claimed (e.g. Dienes 2014) that B-based inference allows us to either confidently reject/accept the null, OR declare the data to be inconclusive/insensitive based on conventional cut-off values for B; while a value of p>.05 can never really distinguish between the null being true and the data just not being sensitive enough.

However, I don't really understand how analysing the data in terms of B gives us more confidence in rejecting/accepting H0 vs declaring the data to be inconclusive. How are the conventions for B (e.g. data is insensitive if B is between 1/3 and 3) any less arbitrary than if one were to define similar "inconclusiveness" conventions also for p-values, such as declaring as inconclusive data that gives p-values only a bit above alpha (e.g. between 0.05 and 0.10), i.e. essentially playing with the false-discovery (Type I error) rate?

Also, can one really always drive B out of the "insensitive range" (e.g. 1/3 to 3) simply by making the data less insensitive i.e. by collecting more data i.e. by increasing N? I understand this is alright to do with B-based stats but not with p-based stats, although I don't understand why - assume you initially find the data to be insensitive using B-values, then add more data, then are able to reject/accept H0 - say you then added even more data, and this leads to a new B-value whereby the opposite conclusion is drawn about H0. How is that acceptable/good practice?

Answer 1

"data is insensitive if B is between 1/3 and 3"

This is indeed an artificial convention or conventional artifice and as such it is as arbitrary as the $0.05$ used as a beacon for $p$-value based decisions. What is almost always missing from the Bayesian toolbox is a calibration of the Bayes factor, namely to see how it behaves under the null predictive and the alternative predictive distributions, because this provides a range of possible (likely?) values under both models or hypotheses and hence a numerical reference to judge the strength of the observed Bayes factor. If the ranges of the Bayes factors under both models intersect too much, we can then conclude by the inconclusive decision! This Bayesian calibration is more of an idea that came to me when answering the OP question than a standard in Bayesian practice, the closest notion being the posterior predictive calibration in Gelman et al. (2013) Bayesian Data Analysis. (For instance, I do not mention this notion in my books.)

"can one really always drive B out of the "insensitive range""

There are fairly generic results that, when the sample size grows to infinity, under some minimal requirements on the regularity of the distribution of the said sample, the Bayes factor goes to zero or to infinity depending on which proposed model is closest in the Kullback-Leibler sense to the true model behind the data. It is therefore the case that more data drives the Bayes factor away from the vicinity of $1$.

"add more data, then are able to reject/accept H0 - say you then added even more data, and this leads to a new B-value whereby the opposite conclusion"

On the one hand, this is cherry picking: if you keep collecting data until the Bayes factor is above a certain bound, it is no different from collecting data until the $p$ value is below a certain bound and you always end up rejecting whatever hypothesis is to be rejected (or die in the attempt if the data never strongly opposes the said hypothesis). On the other hand, the Bayes factor as such is not intended for this decision rule. It compares two marginal likelihoods as measures of evidence for the two models under comparison. While it a.s. converges to one of the limits $0$ or $\infty$, the theory does not say anything about a decision rule that compares the Bayes factor to a fixed or sequential bound.