5
$\begingroup$

I am running a logistic model. In SAS Entreprise Miner, I noticed there's a link function that has three possible options: logit, probit and cll (complementary log-log).

Can you please shed light on the following questions:

  1. Can we use any of these link function to carry out a logistic regression?
  2. Are there situations where one would be better than others?
  3. Is it intuitively possible to get some insight about which kind of function could be useful in which situation? (By just looking at the formula, complementary log-log function might be good for normalization of data when data does not depart (too much) from a normal distribution.)

Any additional pointers would be greatly appreciated.

$\endgroup$
  • 1
    $\begingroup$ 1, please consider improving the way you ask questions. I am asking solely because you have asked several questions already which were poorly written. They got edited to make them better, but you cannot expect others to fix your mistakes forever. $\endgroup$ – mpiktas May 10 '11 at 18:44
  • 3
    $\begingroup$ I'll second @mpiktas's comment, and I would recommend you take some time to think about your question(s) and how to improve them (I'm not speaking about english typos, but about their clarity). I also noticed that you only accepted few answers so far. Any reason why? It you feel users missed important points when answering your past questions, you should notify them; otherwise, if one of those responses looks acceptable from your point of view, you should mark it as is. $\endgroup$ – chl May 10 '11 at 20:55
  • $\begingroup$ @chl-- I have written earlier that I was not able to get user id and password for my earlier id which was required for accepting the answers..but you would notice (if you ) that I consistently say Thanks or any other happy gestures to express my satisfaction..About the clarity of questions..i really didnt understand if I was very vague or not expressing the question..As far as I see, somebody comes and edits the way to phrase the question. Thats english then..aint it ? still I will consider asking the questions the best way I can. $\endgroup$ – ayush biyani May 11 '11 at 5:01
  • $\begingroup$ I have merged your two accounts (and deleted our recent comments). I leave the rest as a reminder. Marking a response as accepted is different from thanking the answerer, because then it means the problem has been solved and the thread might be "freezed" in some sense (at least temporarily, because sometimes there's no definitive answer). (b) Yes, some users take some time to try to improve or reword questions to make them more likely to get a good response. If you're not happy with changes that were suggested, you can roll back to the previous version, at any time. $\endgroup$ – chl May 11 '11 at 7:19
  • $\begingroup$ @chl - fair enough. Lemme see to the guidelines now. Thanks $\endgroup$ – ayush biyani May 12 '11 at 15:02
4
$\begingroup$

I don't know of SAS, so i'll just answer based on the statistics side of the question. About the software you mays ask at the sister site, stackoverflow.

  1. If the link function is different (logistic, probit or Clog-log), than you will get different results. For logistic, use logistic.

  2. About the real differences of these link functions.

Logistic and probit are pretty much the same. To see why they are pretty much the same, remember that in linear regression the link function is the identity. In logistic regression, the link function is the logistic and in the probit, the normal. Formally, you can see this by noting that, in case your dependent variable is binary, you can think of it as following a Bernoulli distribution with a given probability of success. $Y \sim Bernoulli(p_{i})$

$p_{i} = f(\mu)$

$\mu = XB$

Here, thew probabitliy $p_{i}$ is a function of predictor, just like in linear regression. The real difference is the link function. In linear regression, the link function is just the identity, i.e., $f(\mu) = \mu$, so you can just plug-in the linear predictors.In the logistic regression, the link function is the cumulative logistic distribution, given by $1/(1+exp(-x)). In the probit regression, the link function is the (inverse) cumulative Normal distribution function. And in the Clog-log regression, the link function is the complementary log log distribution.

I never used the Cloglog, so i'll abstein of coments about it here.

You can see that Normal and Logist are very similar in this blog post by John Cook, of Endeavour http://www.johndcook.com/blog/2010/05/18/normal-approximation-to-logistic/.

In general I use the logistic because the coefficients are easier to interpret than in a probit regression. In some specific context I use probit (ideal point estimation or when I have to code my own Gibbs Sampler), but I guess they are not relevant to you. So, my advice is, whenever in doubt about probit or logistic, use logistic!

$\endgroup$
2
$\begingroup$

I have a question/comment. I thought that by definition, logistic regression uses the logit link. If you are using the probit or complementary log-log link, then I do not think that is logistic regression.

What you are doing is fitting generalized linear models on a binary outcome, which is assumed to follow a Bernoulli. The 3 usual choices of link functions are the logit, probit, and complementary log-log. If you are using the logit link, that is logistic regression.

$\endgroup$
  • $\begingroup$ No, we can use different functions under proc logistic. Yes,the interpretation can be different. Like logit says it is log(odds)..probit is related to pth quantile probability distribution. CLL has a different formula. we have link= option in proc logistic which allows us to do so. $\endgroup$ – ayush biyani May 11 '11 at 18:28
  • $\begingroup$ The same applies with R, Stata, and any decent statistical software. Logistic regression belongs to GLMs (where G=generalized), whatever the link function (which relates the outcome to a linear combination of explanatory variables) you choose. $\endgroup$ – chl May 11 '11 at 21:00
  • 5
    $\begingroup$ I think that is what I am trying to say: that logistic regression belongs to GLMs, rather than GLMs belonging to logistic regression. Just because SAS has those options for its proc logistic does not mean that it is logistic regression. $\endgroup$ – jkd May 13 '11 at 18:32
  • $\begingroup$ +1 since this is exactly what I found myself thinking while reading this question... $\endgroup$ – Macro Jun 26 '12 at 22:33
2
$\begingroup$

All 3 link functions are s-shaped and are not going to be that different. Li and Duan showed that if the predictor variables are well behaved (elliptically symmetric predictors are a subset of the well behaved group) then changing the link function will change the coefficients by a multiplicitive constant. Even if the predictors are not perfectly well behaved the differences between similar link functions are unlikely to change the overall inference (the exact coefficients will change, but what is important or significant will still be under a different link function).

The logit allows you to interpret individual coefficients as log-odds, so it tends to be the most popular these days.

$\endgroup$
  • $\begingroup$ Can you add the link to Li & Duan's paper? $\endgroup$ – chl May 11 '11 at 21:02
  • $\begingroup$ @all -- can we take this post as an opportunity to discuss the interpretation of probit and CLL in some what detail..I'll state the CLL formula here and start it. log(-log(1-pi)). $\endgroup$ – ayush biyani May 12 '11 at 7:13
1
$\begingroup$

This is an excellent question that sits at the nexus of mathematics and science. As someone who teaches a linear models course that touches on "logistic regression" and its several possible link functions, I feel compelled to answer.

First, I believe that SAS is fitting a generalized linear model (GLM) and estimating the parameters using MLE (or qMLE) in its "logistic" function. As such, any appropriate link function that transforms (0, 1) into (-\inf, \inf) is appropriate. Of that infinite class of functions, the logit, the probit, and the complementary log-log are members... so are all quantile functions.

Second, there is little appreciable difference between the logit and the probit link functions. While the coefficient estimates will tend to differ by a factor of about 3.8, the predictions will be very similar.

Third, the logit and probit functions are symmetric about (0, 0.5), while the complementary log-log function is not symmetric. This constitutes the primary difference between the logit/probit functions and the complementary log-log function.

Recall that the dependent variable is the probability of a success and the independent variable is the linear predictor. For the logit/probit links, the function value approaches 0 at the same rate as it does 1. For the complementary log-log function, however, that is not true. The cloglog function approaches 1 more sharply than it approaches 0. [Side note: the log-log function is the complement of the cloglog. It approaches 0 more sharply than 1.]

Fourth... I'm not sure what that actually means in terms of your last question. My experience is that the science has not advanced enough to suggest a "correct" link function. As a result, I instruct my students to fit their model using several link functions. If the coefficient results differ by "a lot," then there is something wrong with their model. Otherwise, the model is robust to the selection of the link function.

While this is an answer to ayush biyani, I think #4 could drive an interesting discussion about link functions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.