# Is the logit function always the best for regression modeling of binary data?

I've been thinking about this problem. The usual logistic function for modeling binary data is: $$\log\left(\frac{p}{1-p}\right)=\beta_0+\beta_1X_1+\beta_2X_2+\ldots$$ However is the logit function, which is an S-shaped curve, always the best for modeling the data? Maybe you have reason to believe your data does not follow the normal S-shaped curve but a different type of curve with domain $(0,1)$.

Is there any research into this? Maybe you can model it as a probit function or something similar, but what if it is something else entirely? Could this lead to better estimation of the effects? Just a thought I had, and I wonder if there is any research into this.

• possible duplicate of Difference between logit and probit models Jan 19, 2013 at 5:38
• @macro I don't think it's an exact duplicate. That question is about just logit and probit; this one asks for other alternatives as well. Jan 19, 2013 at 13:01
• I'm voting to leave this open. The main difference I see is that this Q is asking for research in statistics on the topic of different possible link functions. It's a subtle difference, but it may be enough. @Glen, you may want to review the other Q, if you haven't seen it already. In my answer I talk about different possible links. If you think this Q isn't really different, flag it & the mods can close it; if you can think of a way to make the distinction b/t what you're asking & that Q clearer, you may want to edit to do so. Jan 19, 2013 at 15:12
• I know it's not an exact duplicate of the logit vs. probit question but I thought gung's answer, which went above and beyond what was asked by the linked question, addresses most of what was asked here, which is why I closed as a duplicate. There are probably other closely related threads but that is the first that came to mind. Jan 19, 2013 at 20:50
• Thanks for the comments. I believe my question is different from the previous question. I am very familiar with the probit and log-log transformations, and the discussion from the previous question was very informative for me. However, I am interested in other link functions (possibly non-parametric?) that are possible, in a situation you which you may or may not have knowledge that the probability curve follows a different distribution. I think when interactions are involved among the covariates this could play an important role. @David J. Harris answer is helpful as well...
– Glen
Jan 20, 2013 at 5:01

People use all sorts of functions to keep their data between 0 and 1. The log-odds fall out naturally from the math when you derive the model (it's called the "canonical link function"), but you're absolutely free to experiment with other alternatives.

As Macro alluded to in his comment on your question, one common choice is a probit model, which uses the quantile function of a Gaussian instead of the logistic function. I've also heard good things about using the quantile function of a Student's $t$ distribution, although I've never tried it.

They all have the same basic S-shape, but they differ in how quickly they saturate at each end. Probit models approach 0 and 1 very quickly, which can be dangerous if the probabilities tend to be less extreme. $t$-based models can go either way, depending on how many degrees of freedom the $t$ distribution has. Andrew Gelman says (in a mostly unrelated context) that $t_7$ is roughly like the logistic curve. Lowering the degrees of freedom gives you fatter tails and a broader range of intermediate values in your regression. When the degrees of freedom go to infinity, you're back to the probit model.

Hope this helps.

Edited to add: The discussion @Macro linked to is really excellent. I'd highly recommend reading through it if you're interested in more detail.

• The question is specifically about "binary data" — not about data that is between 0 and 1. The probit model has no theoretical justification in the case of binary data. Jan 19, 2013 at 16:03
• @NeilG, one reason to use the probit model is that it gives a convenient way of modeling multivariate binary data (e.g. with a mixed model) as thresholded normals. In that case, the correlation matrix of the underlying variables is statistically indentifiable, whereas it is not in the logistic case. There's a bit longer discussion here. Jan 19, 2013 at 20:45
• @Macro: Oh, I see. That's very interesting, thanks. Jan 19, 2013 at 20:52
• @David J.Harris: Do you mean quintile (or maybe quantile has the same meaning), i.e., breaking down the distribution into chunks of fifths: 20%, 40%,..,100% ?
– MSIS
Aug 26, 2016 at 17:20
• @MSIS a quintile divides into fifths, a percentile divides into 100ths, and a quartile divides into arbitrary units See en.wikipedia.org/wiki/Quantile#Specialized_quantiles Aug 26, 2016 at 19:43

I see no reason, a-priori, why the appropriate link function for a given dataset has to be the logit (although the universe does seem to be rather kind to us in general). I don't know if these are quite what you're looking for, but here are some papers that discuss more exotic link functions:

Disclosure: I don't know this material well. I tried dabbling with the Cauchit and Scobit a couple of years ago, but my code kept crashing (probably because I'm not a great programmer), and it didn't seem relevant for the project I was working on, so I dropped it.

Most of this stuff has to do with differing tail behavior than the prototypical links (i.e., the function 'turns the corner' early and doesn't asymptote to 0 & 1 very fast), or are skewed (i.e., like the cloglog, they approach one limit faster than the other). You should also be able to replicate these behaviors, I believe, by fitting a spline function of $X$ with a logistic link.

The best strategy is to model the data in light of what is going on (No surprise!)

• Probit models originate with LD50 studies - you want the dose of insecticide that kills half the bugs. The binary response is whether the bug lives or dies (at a given dose). The bugs that are susceptible at one dose will be susceptible at lower doses as well, which is where the idea of modeling to the cumulative Normal comes in.
• If the binary observations come in clusters, you can use a beta-binomial model. Ben Bolker has a good introduction in the documentation of his bbmle package (in R) which implements this in simple cases. These models allow more control over the variation of the data than what you get in a binomial distribution.
• Multivariate binary data -- the sort that rolls up into multi-dimensional contingency tables - can be analysed using a log-linear model. The link function is the log rather than the log odds. Some people refer to this as Poisson regression.

There probably isn't research on these models as such, although there has been plenty of research on any one of these models, and on the comparisons between them, and on different ways of estimating them. What you find in the literature is that there is a lot of activity for a while, as researchers consider a number of options for a particular class of problems, and then one method emerges as superior.

• +1 for beta-binomial. That's a great tool to have in one's toolbox. Jan 21, 2013 at 4:20

Logit is a model such that the inputs are a product of experts each of which is a Bernoulli distribution. In other words, if you consider all of the inputs to be independent Bernoulli distributions with probabilities $p_i$ whose evidence is combined, you will find that you are adding the logistic function applied to each of the $p_i$s. (Another way of saying the same thing is that conversion from the expectation parametrization to the natural parametrization of the Bernoulli distribution is the logistic function.)