I have a question about model selection when using AIC/BIC. So, if two model structures are totally different, can I still directly apply AIC and BIC? Also, for a hierarchical model, how to compute the total number of parameters when computing AIC and BIC? Thanks.
$\begingroup$
$\endgroup$
-
$\begingroup$ can you explain what you mean by model structures? $\endgroup$ – DataD'oh May 17 '14 at 22:34
-
$\begingroup$ For Example, one is Multinomial Logistic Regression, another one is Hierarchical Gamma. $\endgroup$ – Cosmozhang May 18 '14 at 0:24
$\begingroup$
$\endgroup$
My understanding of AIC and BIC is that they let you compare the fits of non-nested models. So as long both models are fit to the same data set, a comparison seems justified even if they are "totally different". As for the number of free parameters in a hierarchical model, I think you should use the number of parameters that are being estimated, which means (1) the fixed effects, (2) the random effects variance components, and (3) the covariances of the random effects, if any. This thread provides a more in-depth discussion: https://stat.ethz.ch/pipermail/r-sig-mixed-models/2012q3/019121.html
-
1$\begingroup$ Just to add some extra bits - you need to ensure normalising constants are included in the maximised log - likelihood. Also the likelihood used in multi level modeling should be the integrated/marginal likelihood (ie random effects integrated out). $\endgroup$ – probabilityislogic Jun 29 '14 at 23:36
