I have a question about model selection when using AIC/BIC. So, if two model structures are totally different, can I still directly apply AIC and BIC? Also, for a hierarchical model, how to compute the total number of parameters when computing AIC and BIC? Thanks.

  • $\begingroup$ can you explain what you mean by model structures? $\endgroup$ – DataD'oh May 17 '14 at 22:34
  • $\begingroup$ For Example, one is Multinomial Logistic Regression, another one is Hierarchical Gamma. $\endgroup$ – Cosmozhang May 18 '14 at 0:24

My understanding of AIC and BIC is that they let you compare the fits of non-nested models. So as long both models are fit to the same data set, a comparison seems justified even if they are "totally different". As for the number of free parameters in a hierarchical model, I think you should use the number of parameters that are being estimated, which means (1) the fixed effects, (2) the random effects variance components, and (3) the covariances of the random effects, if any. This thread provides a more in-depth discussion: https://stat.ethz.ch/pipermail/r-sig-mixed-models/2012q3/019121.html

  • 1
    $\begingroup$ Just to add some extra bits - you need to ensure normalising constants are included in the maximised log - likelihood. Also the likelihood used in multi level modeling should be the integrated/marginal likelihood (ie random effects integrated out). $\endgroup$ – probabilityislogic Jun 29 '14 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.