I would like to use Bayesian framework for model selection. I have more than 10 models with the same number of parameters (just different assumptions on underlying parameters of the distribution). How to apply, for example, Bayes factors in this case? One-vs-all? So
$H_0 = H_0$,
$H_A = H_1 \cup H_2 \ldots \cup H_n$,
calculate Bayes factors for $H_0$ and $H_A$ for all hypothesis and than choose the model with the highest Bayes factor as the most appropriate (or decide that none of the models are suitable)?
(I would use BIC/AIC if I would have models with different sets of parameters, but I do not know any method for this problem. Maximum likelihood does not use all the information so it is not a solution).
Here is the example (without priors). I would say that Posterior Probability (pp) of 0.58 is not big enough to make a decision in favour of second model and I would increase the sample size, but using ML I would choose third model (while second is true). Is the suggested procedure correct?
> dataset <- rnorm(10, mean=1.0) > > first_mu = 0.0 > second_mu = 1.0 > third_mu = 2.0 > first_likelihood = prod((dnorm(dataset, first_mu))) > second_likelihood = prod((dnorm(dataset, second_mu))) > third_likelihood = prod((dnorm(dataset, third_mu))) > first_likelihood  1.370522e-10 > second_likelihood  2.122489e-06 > third_likelihood  1.492313e-06 > pp1 = first_likelihood / (first_likelihood + second_likelihood + third_likelihood) > pp2 = second_likelihood / (first_likelihood + second_likelihood + third_likelihood) > pp3 = third_likelihood / (first_likelihood + second_likelihood + third_likelihood) > pp1  3.791275e-05 > pp2  0.5871438 > pp3  0.4128183
If I would have dataset from dnorm(n, mean=0.5), I would choose one model using likelihood, but I can choose none of models using BFs, that's why I don't want to use ML methods.
UPD: the similar situation is described here, but I still can not figure out how to switch to BFs for several models...Can not understand how posterior probability be interpreted (again, some arbitrary threshold on posterior probability? like level of significance for hypothesis testing?)
UPD2: If there are more than two models to compare then we choose one of them as a reference model and calculate Bayes factors relative to that reference. - that is even more strange...