I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable).

TEMPC <- geeglm(BINARYACTIVITY ~ TEMPC, family=binomial(link="logit"), data=Animal, id=NEW_PERIOD2, corstr="ar1", std.err="san.se")

I intend for the GEE to 'account for' repeated measures; activity was assessed multiple times each hour, during sampling periods that lasted days (hence, I used "id=NEW_PERIOD2" to designate sampling periods).

I am interested in evaluating the predictive accuracy of the resulting model. Normally, for data that are not autocorrelated, I would use a logistic regression model, and assess its predictive accuracy with calculation of AUC from a ROC. However, I am uncertain how to calculate AUC for a GEE, which accounts for autocorrelation. Is there a way to calculate AUC for a GEE? If so, can I apply the method to interaction models, such as the one I pasted below?

TEMPC_COSHOUR <- geeglm(BINARYACTIVITY ~ TEMPC*COSHOUR, family=binomial(link="logit"), data=Animal, id=NEW_PERIOD2, corstr="ar1", std.err="san.se") #COSHOUR is a a transformed Time variable

Thank you for any advice you can provide!


I hate to say this, but I think you might be barking up the wrong tree here.

Both GEE's and GLMM's can account for correlation between observations. GLMM's attempt to model associations at the individual level (i.e. model an individual's mean and slope), while GEE's just look at overall population level trends, while still accounting for the fact that the observations are correlated.

As a result, GEE's are intended to be used for more robust analyses: it's much easier to model what happens at a population level rather than at the individual level. Because of this, GEE's are more robust to deviations from the model.

The trade off is that by not modeling things on the individual level, you really essentially lose the ability to make individual level predictions. If that's what you're interested in, you should really be switching to a GLMM.

  • $\begingroup$ Thanks, Cliff. I appreciate your ideas here. However, I have investigated R packages for GLMMs (nlme and lme4) and neither models autocorrelated data with a binary response (lme4 models G- but not R-sided covariance structures*). Because of this, I moved to GEE, which I think will be suitable for my purposes. *From the lme4 manual: "lme4 does not currently implement nlme’s features for modeling heteroscedasticity and correlation of residuals." $\endgroup$
    – Nick
    Jun 4 '15 at 14:05
  • $\begingroup$ While I agree with @CliffAB, it would not be possible to make individual level predictions with GEE. I could see an argument of assigning a group level prediction to each individual in that group, but you would expect a different kind of error for assuming individuals respond according to the group. That said, it still seems "wrong." I was only able to find one example of ROC curve areas (c statistics) for GEE based models in the medical literature, and I don't put much stock in their statistical analysis. $\endgroup$ Oct 8 '16 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.