Comparing model efficiency

Question

I hope you all don't mind me asking this question.

I have two models :

1. general linear mixed effects model

   library(lme4)
   
   d = read.csv('http://www.bodowinter.com/uploads/1/2/9/3/129362560/politeness_data.csv')
   
   res1 = lmer(frequency ~ scenario + gender + attitude + (1+ scenario|subject), data=d)
   summary(res1)
   

2. general estimating equation model

   library(geepack)\
   res2 = geeglm(frequency ~ scenario + gender + attitude , id=subject, family=gaussian, corstr="ex", data=d)\
   summary(res2)
   

I am not sure how to compare the model fit between these two models. The mixed effects model provides AIC and LogLikelihood values, but I don't see that with the GEE model. Any suggestions are much appreciated. Thanks.

Answer 1

One option would be to evaluate model performance through cross-validation methods. For example, simulate 1000 data sets from your original data, each time randomly withholding 20% for testing and fitting each of the two models on the remaining 80%. Then you could report back root-mean-square error, bias, etc. for the two models and compare their performance with respect to these metrics.

Because GEE isn't a likelihood-based method, it is not possible to calculate AIC, etc. (and you won't see them returned as part of any model summary, as you mention in your post).

Depending on your application, the difference in interpretation between GEEs and GLMs can be enough to determine which model to go with. The comments and answers to this CV question are a good resource and contain links to other sources for a deeper-dive into this topic.