Choosing a Model Selection Criterion I am trying to decide which among for model selection criteria to use for a Bayesian nonparametric model. The candidates are:


*

*The L-criterion, as defined by Laud & Ibrahim (1995);

*Bayes factors;

*Bayesian Information Criterion; and

*Kullback-Leibler divergence.


Is one criterion better than the others? Can you point me to some references that discuss the advantages and disadvantages of each?
Thanks in advance.
 A: Recently, according to my studies of this book and lectures, the tendency when it comes to information criterion for model comparison and selection, is to use the Widely Applicable Information Criterion (WAIC, which I just found out has a very dry Wikipedia entry).
Now, in a Bayesian perspective, an Information Criterion is attempting to estimate out-of-sample deviance, which it self, is a proxy of the distance from the estimated parameters/model distributions to the true parameters/model distribution. (my explanation might be a little of here, since I am self taught.)
This is all 'built' around the Kullback-Leibler divergence. The AIC also has this Bayesian/informational theoretical interpretation. There is also the DIC, which has this type of interpretation. 
It is also hard to tell how a criterion will perform when very skewed posteriors come up from you model specification (WAIC exceeds in that).
All that being said I would follow one the 2 choices:


*

*If I have time, I would device a set of simulation data based on how I believe the structure of the model is (with varying number of parameters), train the models an compare the information criterion with respect to which one better indicates the best model.

*Choose a small set of criteria that I understand how they work, and use them to perform model selection. 
Also keep in mind that model selection is nice, but in a Bayesian perspective, model ensamble is awesome, and you should do it.
PS: If you think you might be inclined towards a more rigorous approach to my answer, I can easily provided it.
