2
$\begingroup$

Suppose I have two models created by calling glm() on the same data but with different formulas and/or families. Now I want to compare which model is better by predicting on an unknown data. Something like this:

mod1 <- glm(formula1, family1, data)
mod2 <- glm(formula2, family2, data)
mu1 <- predict(mod1, newdata, type = "response")
mu2 <- predict(mod2, newdata, type = "response")
  1. How can I tell which of the predictions mu1 or mu2 is better?
  2. Is there some simple command to compute the log likelihood of a prediction?
$\endgroup$
  • $\begingroup$ Define 'better' for your purposes. $\endgroup$ – Glen_b May 15 '14 at 19:53
  • $\begingroup$ this could be a long answer without more details. you mean something like? ncbi.nlm.nih.gov/pmc/articles/PMC3575184 $\endgroup$ – charles May 15 '14 at 20:07
  • $\begingroup$ and why not extractAIC(mode1) or AIC(mod1) / BIC(mod1) ? $\endgroup$ – charles May 15 '14 at 21:13
1
$\begingroup$
  1. You can compare the predicted values for each model with the actual observed values to see which model is performing better. This means that you are just looking at the in-sample-performance of the models. You can also double check the out-of-sample performance of those models. Or if the models are nested and you want to compare them you can use anova(m1,m2, test="Chisq").
  2. There is a function (logLik) that you can use to obtain the log likelihood of your model (not the prediction). But obtaining that likelihood depends heavily on your model that you are fitting, most of the time you can but not always (see e.g. model m3 below where there is no log likelihood to extract).

The Code:

> counts <- c(18,17,15,20,10,20,25,13,12)
> outcome <- gl(3,1,9)
> treatment <- gl(3,3)
> m1<- glm(counts ~ outcome + treatment, family = poisson)
> logLik(m1)
'log Lik.' -23.38066 (df=5)
> m2<- glm(counts ~ outcome + treatment, family = gaussian)
> logLik(m2)
'log Lik.' -22.78576 (df=6)
> m3<- glm(counts ~ outcome + treatment, family = quasipoisson)
> logLik(m3)
'log Lik.' NA (df=5)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.