5
$\begingroup$

I have calculated a Bayes Factor BF10 for the probability of the data under H1 vs. H0. I get very large numbers (the data are very clear, statistics are barely necessary here), in the order of 10^30.

What are the guidelines to report BF10 in these cases? Should I just follow guidelines by eg. Jeffreys (1961) and report the BF > 100? Or report the actual number?

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ I do not see any advantage to discarding the information that BF is much larger than 100. $\endgroup$
    – Christoph Hanck
    Nov 16, 2016 at 9:57

2 Answers 2

Reset to default
1
$\begingroup$

A similar problem was raised in another question. Namely, how to make a decision based on a Bayes factor. My personal position is that Jeffreys' scale is quite arbitrary and that the decision should either reflect the final impact of a wrong decision or be calibrated by posterior predictive calibration of the Bayes factor under both hypotheses.

With regard to reporting the raw figure instead of its position in Jeffreys' scale, I definitely support reporting the raw figure.

Improve this answer
$\endgroup$
4
  • $\begingroup$ It's not immediately obvious to me how your answer is relevant to the question. Are you saying that reporting the "actual number" may be more appropriate, or that "greater than..." is fine, but perhaps a number other than 10 would be more appropriate? $\endgroup$
    – Ian_Fin
    Nov 16, 2016 at 10:44
  • $\begingroup$ I am stating that the figure by itself has no absolute meaning. No more than the actual value of a p-value has an absolute meaning. To make a decision on whether or not accepting the null hypothesis, those quantities should be calibrated by posterior predictive tools. $\endgroup$
    – Xi'an
    Nov 16, 2016 at 10:48
  • 3
    $\begingroup$ But it's the figure that the OP wants to report. I think you make an important point, but I also feel the answer could be improved by making clearer the relevance of your point to the specific question. $\endgroup$
    – Ian_Fin
    Nov 16, 2016 at 11:07
  • 1
    $\begingroup$ I like your answer: Jeffrey's scale is arbitrary, and computing expected loss for the two hypotheses would make much more sense. The choice of the loss function also contains a degree of arbitrariness, but at least it's much more problem-specific than a "one-size-fits-all" scale. However, what do you mean with "posterior predictive calibration of the Bayes factor"? It sounds interesting, could you explain or add a reference? $\endgroup$
    – DeltaIV
    Nov 16, 2016 at 21:40
1
$\begingroup$

Kass and Raftery (1995) propose to use $2\log_e B_{10}$, i.e. twice the natural logarithm of the Bayes factor (BF), since it is on the same scale as the likelihood ratio test statistic. The interpretation is as follows:

 0-2:  Not worth more than a bare mention
 2-6:  Positive
6-10:  Strong
 >10:  Very strong.

Update: However, as Xi'an pointed out, be aware that this categories are not a calibration of the Bayes factor, but a quick descriptive measure of the evidence.

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1 for citing Kass & Raftery - I'm kind of self-taught in bayesian stats (well, actually even in frequentist stats :D so I don't know if it's a standard reference, but I believe it's great. What do you mean by "calibration of the Bayes factor"? $\endgroup$
    – DeltaIV
    Nov 16, 2016 at 21:31
  • 1
    $\begingroup$ Thanks! By calibration I mean the the scales do not take into account the sampling properties of the BF. See here www3.stat.sinica.edu.tw/statistica/oldpdf/A15n24.pdf. $\endgroup$
    – utobi
    Nov 17, 2016 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.