Is the logit function always the best for regression modeling of binary data?

Question

I've been thinking about this problem. The usual logistic function for modeling binary data is: $$ \log\left(\frac{p}{1-p}\right)=\beta_0+\beta_1X_1+\beta_2X_2+\ldots $$ However is the logit function, which is an S-shaped curve, always the best for modeling the data? Maybe you have reason to believe your data does not follow the normal S-shaped curve but a different type of curve with domain $(0,1)$.

Is there any research into this? Maybe you can model it as a probit function or something similar, but what if it is something else entirely? Could this lead to better estimation of the effects? Just a thought I had, and I wonder if there is any research into this.

Answer 1

People use all sorts of functions to keep their data between 0 and 1. The log-odds fall out naturally from the math when you derive the model (it's called the "canonical link function"), but you're absolutely free to experiment with other alternatives.

As Macro alluded to in his comment on your question, one common choice is a probit model, which uses the quantile function of a Gaussian instead of the logistic function. I've also heard good things about using the quantile function of a Student's $t$ distribution, although I've never tried it.

They all have the same basic S-shape, but they differ in how quickly they saturate at each end. Probit models approach 0 and 1 very quickly, which can be dangerous if the probabilities tend to be less extreme. $t$-based models can go either way, depending on how many degrees of freedom the $t$ distribution has. Andrew Gelman says (in a mostly unrelated context) that $t_7$ is roughly like the logistic curve. Lowering the degrees of freedom gives you fatter tails and a broader range of intermediate values in your regression. When the degrees of freedom go to infinity, you're back to the probit model.

Hope this helps.

Edited to add: The discussion @Macro linked to is really excellent. I'd highly recommend reading through it if you're interested in more detail.

Answer 2

I see no reason, a-priori, why the appropriate link function for a given dataset has to be the logit (although the universe does seem to be rather kind to us in general). I don't know if these are quite what you're looking for, but here are some papers that discuss more exotic link functions:

• Cauchit (etc.):

  Koenker, R., & Yoon, J. (2009). Parametric links for binary choice models: A Fisherian-Bayesian colloquy. Journal of Econometrics, 152, 2, pp. 120-130.

  Koenker, R. (2006). Parametric links for binary choice models. Rnews, 6, 4, pp. 32-34.

• Scobit:

  Nagler, J. (1994). Scobit: An alternative estimator to logit and probit. American Journal of Political Science, 38, 1, pp. 230-255.

• Skew-Probit:

  Bazan, J.L., Bolfarine, H., & Branco, M.D. (2010). A framework for skew-probit links in binary regression. Communications in Statistics--Theory and Methods, 39, pp. 678-697.

• (This seems like a good overview of skewed links within a Bayesian framework):

  Chen, M.H. (2004). Skewed link models for categorical response data. In Skew-Elliptical Distributions and Their Applications: A Journey Beyond Normality, Marc Genton, editor. Chapman and Hall.

Disclosure: I don't know this material well. I tried dabbling with the Cauchit and Scobit a couple of years ago, but my code kept crashing (probably because I'm not a great programmer), and it didn't seem relevant for the project I was working on, so I dropped it.

Most of this stuff has to do with differing tail behavior than the prototypical links (i.e., the function 'turns the corner' early and doesn't asymptote to 0 & 1 very fast), or are skewed (i.e., like the cloglog, they approach one limit faster than the other). You should also be able to replicate these behaviors, I believe, by fitting a spline function of $X$ with a logistic link.

Answer 3

The best strategy is to model the data in light of what is going on (No surprise!)

• Probit models originate with LD50 studies - you want the dose of insecticide that kills half the bugs. The binary response is whether the bug lives or dies (at a given dose). The bugs that are susceptible at one dose will be susceptible at lower doses as well, which is where the idea of modeling to the cumulative Normal comes in.
• If the binary observations come in clusters, you can use a beta-binomial model. Ben Bolker has a good introduction in the documentation of his bbmle package (in R) which implements this in simple cases. These models allow more control over the variation of the data than what you get in a binomial distribution.
• Multivariate binary data -- the sort that rolls up into multi-dimensional contingency tables - can be analysed using a log-linear model. The link function is the log rather than the log odds. Some people refer to this as Poisson regression.

There probably isn't research on these models as such, although there has been plenty of research on any one of these models, and on the comparisons between them, and on different ways of estimating them. What you find in the literature is that there is a lot of activity for a while, as researchers consider a number of options for a particular class of problems, and then one method emerges as superior.

Answer 4

Logit is a model such that the inputs are a product of experts each of which is a Bernoulli distribution. In other words, if you consider all of the inputs to be independent Bernoulli distributions with probabilities $p_i$ whose evidence is combined, you will find that you are adding the logistic function applied to each of the $p_i$s. (Another way of saying the same thing is that conversion from the expectation parametrization to the natural parametrization of the Bernoulli distribution is the logistic function.)