4
$\begingroup$

I have calculated a Bayes Factor BF10 for the probability of the data under H1 vs. H0. I get very large numbers (the data are very clear, statistics are barely necessary here), in the order of 10^30.

What are the guidelines to report BF10 in these cases? Should I just follow guidelines by eg. Jeffreys (1961) and report the BF > 100? Or report the actual number?

$\endgroup$
  • 3
    $\begingroup$ I do not see any advantage to discarding the information that BF is much larger than 100. $\endgroup$ – Christoph Hanck Nov 16 '16 at 9:57
1
$\begingroup$

A similar problem was raised in another question. Namely, how to make a decision based on a Bayes factor. My personal position is that Jeffreys' scale is quite arbitrary and that the decision should either reflect the final impact of a wrong decision or be calibrated by posterior predictive calibration of the Bayes factor under both hypotheses.

With regard to reporting the raw figure instead of its position in Jeffreys' scale, I definitely support reporting the raw figure.

$\endgroup$
  • $\begingroup$ It's not immediately obvious to me how your answer is relevant to the question. Are you saying that reporting the "actual number" may be more appropriate, or that "greater than..." is fine, but perhaps a number other than 10 would be more appropriate? $\endgroup$ – Ian_Fin Nov 16 '16 at 10:44
  • $\begingroup$ I am stating that the figure by itself has no absolute meaning. No more than the actual value of a p-value has an absolute meaning. To make a decision on whether or not accepting the null hypothesis, those quantities should be calibrated by posterior predictive tools. $\endgroup$ – Xi'an Nov 16 '16 at 10:48
  • 3
    $\begingroup$ But it's the figure that the OP wants to report. I think you make an important point, but I also feel the answer could be improved by making clearer the relevance of your point to the specific question. $\endgroup$ – Ian_Fin Nov 16 '16 at 11:07
  • 1
    $\begingroup$ I like your answer: Jeffrey's scale is arbitrary, and computing expected loss for the two hypotheses would make much more sense. The choice of the loss function also contains a degree of arbitrariness, but at least it's much more problem-specific than a "one-size-fits-all" scale. However, what do you mean with "posterior predictive calibration of the Bayes factor"? It sounds interesting, could you explain or add a reference? $\endgroup$ – DeltaIV Nov 16 '16 at 21:40
1
$\begingroup$

Kass and Raftery (1995) propose to use $2\log_e B_{10}$, i.e. twice the natural logarithm of the Bayes factor (BF), since it is on the same scale as the likelihood ratio test statistic. The interpretation is as follows:

 0-2:  Not worth more than a bare mention
 2-6:  Positive
6-10:  Strong
 >10:  Very strong.

Update: However, as Xi'an pointed out, be aware that this categories are not a calibration of the Bayes factor, but a quick descriptive measure of the evidence.

$\endgroup$
  • $\begingroup$ +1 for citing Kass & Raftery - I'm kind of self-taught in bayesian stats (well, actually even in frequentist stats :D so I don't know if it's a standard reference, but I believe it's great. What do you mean by "calibration of the Bayes factor"? $\endgroup$ – DeltaIV Nov 16 '16 at 21:31
  • 1
    $\begingroup$ Thanks! By calibration I mean the the scales do not take into account the sampling properties of the BF. See here www3.stat.sinica.edu.tw/statistica/oldpdf/A15n24.pdf. $\endgroup$ – utobi Nov 17 '16 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.