# Which link function in binomial regression is better?

Concerning the choice of the link function in binomial regression (e.g. logit versus probit or cauchit), I wonder what the recommended comparison criterion might be. Note that I am not interested in hypothesis testing, but just in a relative criterion for "goodness".

Obvious candidates for comparing link functions are the AIC, the Brier score, or the test statistic of the Hosmer-Lemeshow test, but the latter depends on the somewhat arbitrary choice of binning quantiles.

Is one of these recommended, or are they even equivalent? Are there other suggestion?

I would suspect, e.g., that AIC and Brier score are asymptotically equivalent, because the Brier score is a measure for prediction accuracy and the AIC is asymptotically equivalent to using leave-one-out MSE as a criterion.

• Dec 5, 2021 at 20:34
• The link @kjetilbhalvorsen provides has some excellent context. One aspect not mentioned is logit is recommended if the sample size is small. I do not have a reference for this, but presumably this has to do with the higher kurtosis of the logistic compared to the normal distribution. Dec 5, 2021 at 20:56
• @single-malt Are you referring to Hahn & Soyer (2005)? home.gwu.edu/~soyer/mv1h.pdf A comment in the thread mentioned by kjetil b halvorsen gives this reference: their finding was not that logistic was better for small sample size, but that probit was only better for medium and large sample sizes. Incidentally, this paper also gives a possible answer to my question: they used "DIC" (deviance information criterion). They used it because "s similar in interpretation and in spirit to [...] AIC". I thus guess that using the AIC is an acceptable appraoch. Dec 6, 2021 at 8:25
• @kjetil-b-halvorsen The evaluation of parametrized link functions by Koenker (2007) is interesting. Unfortunately, the evaluation is done in Monte Carlo simulations with known true model and the distance to the true model is measured (criteria $d_1, d_2, d_\infty$ in Tbl. 4). These criteria cannot be computed from empirical data stemming from real observations. Dec 6, 2021 at 8:40
• @cdalitz have been unable to substantiate the preference for logit over probit in small sample sizes. Dec 6, 2021 at 19:04

• As Geoffrey so well summarized, only the logit link provides an interpretable scale. I would also base the choice on which link function minimizes the likelihood ratio $\chi^2$ statistic for the chunk test of all two-way interactions combined. I.e., possibly choose the link that minimizes the need for interactions. Dec 6, 2021 at 14:54