# Bayes Factors for more than 2 hypotheses

Naïve question:

I would like to use Bayesian framework for model selection. I have more than 10 models with the same number of parameters (just different assumptions on underlying parameters of the distribution). How to apply, for example, Bayes factors in this case? One-vs-all? So

$H_0 = H_0$,

$H_A = H_1 \cup H_2 \ldots \cup H_n$,

calculate Bayes factors for $H_0$ and $H_A$ for all hypothesis and than choose the model with the highest Bayes factor as the most appropriate (or decide that none of the models are suitable)?

(I would use BIC/AIC if I would have models with different sets of parameters, but I do not know any method for this problem. Maximum likelihood does not use all the information so it is not a solution).

Here is the example (without priors). I would say that Posterior Probability (pp) of 0.58 is not big enough to make a decision in favour of second model and I would increase the sample size, but using ML I would choose third model (while second is true). Is the suggested procedure correct?

 > dataset <- rnorm(10, mean=1.0)
>
> first_mu = 0.0
> second_mu = 1.0
> third_mu = 2.0
> first_likelihood = prod((dnorm(dataset, first_mu)))
> second_likelihood = prod((dnorm(dataset, second_mu)))
> third_likelihood = prod((dnorm(dataset, third_mu)))
> first_likelihood
[1] 1.370522e-10
> second_likelihood
[1] 2.122489e-06
> third_likelihood
[1] 1.492313e-06
> pp1 = first_likelihood / (first_likelihood + second_likelihood + third_likelihood)
> pp2 = second_likelihood / (first_likelihood + second_likelihood + third_likelihood)
> pp3 = third_likelihood / (first_likelihood + second_likelihood + third_likelihood)
> pp1
[1] 3.791275e-05
> pp2
[1] 0.5871438
> pp3
[1] 0.4128183


If I would have dataset from dnorm(n, mean=0.5), I would choose one model using likelihood, but I can choose none of models using BFs, that's why I don't want to use ML methods.

UPD: the similar situation is described here, but I still can not figure out how to switch to BFs for several models...Can not understand how posterior probability be interpreted (again, some arbitrary threshold on posterior probability? like level of significance for hypothesis testing?)

• AIC uses the number of parameters AND the maximum likelihood (ML), so it is the ML that will be the tie breaker. Why not use it? What do you mean by "Maximum likelihood does not use all the information so it is not a solution"? – Salmonstrikes Feb 22 '16 at 12:24
• The model can have the highest likelihood, but BF of the model can be slightly bigger than 1 (so I should increase the sample size in this case). Also I have prior probabilities of different hypothesis, do not know how to use AIC with priors. – German Demidov Feb 22 '16 at 12:29

If you have $m$ models to compare, you can always produce $m$ evidences or marginal likelihoods for those models $$\mathfrak{e}_i=\int_{\Theta_i} f(x|\theta)\text{d}\pi(\theta)$$ and rank those models according to the evidences $\mathfrak{e}_i$.
Unless you have a fundamental reason or loss function to rank the models a priori and a preference to select the first one over the second, and so on, I see no reason in opposing one model $\mathfrak{M}_1$ to the conjunction of all others. And keeping this rule to evaluate all models is incoherent as the denominator should always be the same.
• Thank you for the answer. Yes, I can rank likelihoods, but there are several issues that I have described: likelihood tells me that a hypothesis is the most appropriate among alternatives, but only likelihood ratio can tell me is one of the hypothesis is strongly preferred than any other. The rule is just an application of Bayes factors: we have parameter space and two hypothesis that covers all space and we need to answer the question: how big is the evidence in favour of $H_i$ against all alternatives $H_A$? – German Demidov Feb 22 '16 at 13:55