I have a question about model selection when using AIC/BIC. So, if two model structures are totally different, can I still directly apply AIC and BIC? Also, for a hierarchical model, how to compute the total number of parameters when computing AIC and BIC? Thanks.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ can you explain what you mean by model structures? $\endgroup$– KrrrCommented May 17, 2014 at 22:34
-
$\begingroup$ For Example, one is Multinomial Logistic Regression, another one is Hierarchical Gamma. $\endgroup$– CosmozhangCommented May 18, 2014 at 0:24
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
My understanding of AIC and BIC is that they let you compare the fits of non-nested models. So as long both models are fit to the same data set, a comparison seems justified even if they are "totally different". As for the number of free parameters in a hierarchical model, I think you should use the number of parameters that are being estimated, which means (1) the fixed effects, (2) the random effects variance components, and (3) the covariances of the random effects, if any. This thread provides a more in-depth discussion: https://stat.ethz.ch/pipermail/r-sig-mixed-models/2012q3/019121.html
-
1$\begingroup$ Just to add some extra bits - you need to ensure normalising constants are included in the maximised log - likelihood. Also the likelihood used in multi level modeling should be the integrated/marginal likelihood (ie random effects integrated out). $\endgroup$ Commented Jun 29, 2014 at 23:36