Let's say I have measurements $Y$ which are all positive, and the distribution seems to be somewhat skewed. I'm modelling $Y$ in GLM framework. Now I could set my GLM using different distributional assumptions for $Y$, i.e. I could set family as normal or gamma, or I could log-transform $Y$ and use normal distribution for that. The problem is, how do I compare these models and figure out which is the best? My thinking is that I cannot use AIC or BIC because of different distributions (or in log-transformation case different response variable values). S what wold be the correct way? Only thing I have come up is checking the histograms of residuals and seeing what they look.

Edit: Thinking more, if likelihood is the probability of the data given the model, and if all models have same number of parameters, I can directly compare the log-likehoods, which must contain all the constants also. Can somebody confirm this?

  • 2
    $\begingroup$ Speaking for myself, the main issue would seem to be the implied variance function. If the variance isn't right, concerns about the distributional shape are usually secondary. $\endgroup$ – Glen_b Jan 10 '14 at 10:26
  • $\begingroup$ @Glen_b So you suggest that I should focus on checking if the pearson residuals have constant variance (visually), and make the decision based on that? $\endgroup$ – Jouni Jan 10 '14 at 10:53
  • $\begingroup$ You can plot the predictions of each model against the actual values and against each other and see what the differences are. $\endgroup$ – Peter Flom Jan 10 '14 at 12:00
  • 2
    $\begingroup$ Hemmo, I don't know that I'd necessarily go so far as to especially advocate that as a method of model selection, but if you are looking for a basis for choosing, it's typically more important to get that part right. If you have the ability to do sample splitting/cross validation, you might consider using it in those circumstances. $\endgroup$ – Glen_b Jan 10 '14 at 12:30
  • $\begingroup$ Thanks, I think I'm able to try some cross validation also. $\endgroup$ – Jouni Jan 11 '14 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.