6
$\begingroup$

I would like to know whether I can use AIC, or if the models have the same number of predictors, the log-likelihood, to compare logit vs probit vs cloglog models (fitted for instance with glmer or glmmTMB in R). The question could also be formulated: do the various link functions of these models scale the likelihood similarly (irrespectively of the software-dependent constant included in the likelihood)?

I feel that this question has been asked under various forms but not always answered directly (for instance here: Can I use AIC value for comparing logit and probit model where for each model the number of covariates are equal?) and I can find contrasted answers (for instance here someone suggests you can't: https://www.researchgate.net/post/AIC_of_glmer). So I am allowing myself to post this question here. Answers would certainly be helpful to me and hopefully others in the future. Thanks.

Improve this question
$\endgroup$
8
  • 3
    $\begingroup$ (+1) Generalized linear models that differ merely in the link function employed don't have incommensurable likelihoods (see Likelihood comparable across different distributions). But your parenthetic "fitted for instance with glmer or glmmTMB" deserves more emphasis (& translation from software-specific jargon): you're talking about mixed models & may not be comparing maximized likelihoods anyway. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Jan 9, 2019 at 15:21
  • 2
    $\begingroup$ Not too long ago, someone on this website made the point that the choice of link function should be made primarily based on whether we care about explanation or prediction. If we care about being able to interpret model coefficients, then using a logit link, for example, would be preferred. If we care about prediction, then we could consider links like cloglog. Joseph Hilbe may cover this point in one of his books. $\endgroup$
    – Isabella Ghement
    Commented Jan 9, 2019 at 16:20
  • 2
    $\begingroup$ A poor choice of link function may result in over-dispersion (see mun.ca/biology/dschneider/b7932/B7932Final10Dec2008.pdf), so you may counter that poor choice by allowing for over-dispersion in your modelling. $\endgroup$
    – Isabella Ghement
    Commented Jan 9, 2019 at 16:27
  • 1
    $\begingroup$ @Isabella true, there are many discussions about this on cross validated and elsewhere. Here I am only focusing on the mathematical/statistical validity of the comparison. the issue of the link between link function choice and overdispersion is less discussed though. Thanks for bringing it in. $\endgroup$
    – Jehol
    Commented Jan 10, 2019 at 9:04
  • 1
    $\begingroup$ @André.B: If they're fitting their models by maximum-likelihood, then I can't see a problem with comparing likelihoods/AIC (standard terms & conditions apply). If they're fitting their models by REML (or whatever the equivalent of REML is for GLMMs), then I don't know - you could ask a q. here giving more detail & see what the mixed-model experts have to say. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Apr 29, 2019 at 8:44

1 Answer 1

Reset to default
1
$\begingroup$

I would say yes, I can't see any reason why they wouldn't. We're talking about evaluating the log-likelihood for the same conditional probability distribution, with the same probability density (i.e. we don't have to account for changes in the scale due to transformation). So we're comparing the likelihoods for

$$ y_i \sim \textrm{Distrib}(g_1^{-1}((\mathbf X \boldsymbol \beta)_i), \phi) $$

with

$$ y_i \sim \textrm{Distrib}(g_2^{-1}((\mathbf X \boldsymbol \beta)_i), \phi) $$

where $\textrm{Distrib}$ is the response distribution, $g_1$ and $g_2$ are the alternative link functions (the rest is as usual for GLMs: $\mathbf X$=model matrix, $\boldsymbol \beta$=coefficient vector, $\phi$=scale parameter). This is just comparing two different nonlinear specifications for the location of the distribution. As long as you're OK with using AIC to compare non-nested functions (which almost everyone is), this should be fine.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.