How does one compare the Goodness of Fit for different models of some given binomial data using Generalized Linear Mixed Models.

Specifically we want to know whether a model with a logit link gives a better fit than a model with a probit link.

Is it legitimate to compare -2loglikelihood values for the 2 models using the different link functions?

If not, what would be a good Goodness of Fit comparison measure?

  • $\begingroup$ Logit and probit are so close, you need a big data set to see any differences $\endgroup$ – kjetil b halvorsen Sep 18 '18 at 21:22

Models with different link functions are not nested, and therefore you cannot use standard statistical tests (e.g., a likelihood ratio test) to compare them. You could consider using information criteria, such as the AIC and the BIC, though given that there is no other difference between the two models, it boils down to comparing the likelihoods.

Two other alternatives could be (1) to see which model predicts better, e.g., using ROC analysis or by calculating the squared prediction error, and (2) go for a Bayesian approach and use the idea of posterior predictive checks.

| cite | improve this answer | |
  • $\begingroup$ thanks. 1. we have tried predictive error as (p_oberved-p_predicted)^2/p_predicted. can you suggest a reference to justify this? 2. is there a simple equation to obtain posterior predictive probability from the loglikihood of the F values associated with each predictive hypothesis? $\endgroup$ – dekdek Sep 19 '18 at 10:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.