Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Given the same set of covariates and distribution family, how can I compare models having different link functions?

I think the correct answer here is "AIC/BIC", but I am not 100% sure.

Is it possible to have nested models if they have a different link?

share|improve this question
    
Note that "AIC/BIC" is one of the possible answers but, in principle, any (appropriate) model-selection technique could be employed. –  user10525 Jan 2 '13 at 16:20
2  
No, they are not nested. Also, recall that when using AIC/BIC, the normalising constants matter as well. –  user10525 Jan 2 '13 at 16:31
5  
Information criteria, like the AIC & BIC, simply adjust the deviance of the model for its complexity (ie, the number of parameters). If you have the same number of covariates (not even necessarily the same covariates themselves), that adjustment will be irrelevant. You can check them by comparing the deviances directly. You may find it helpful to read my answer here: difference-between-logit-and-probit-models, which touches on this issue. –  gung Jan 2 '13 at 16:32
3  
Another possibility for comparing models, which is very general, but requires more from you, is to use the Parametric Bootstrap Cross-Fitting method. You can find a pdf here. –  gung Jan 2 '13 at 16:37
2  
Is there a particular family you have in mind with 2 possible link functions? i.e. binomial family, logit vs log link? –  Placidia Jan 2 '13 at 19:02
show 3 more comments

2 Answers

up vote 7 down vote accepted

For this problem you can also use so-called "goodness of link tests", the canonical treatment of which was published by Daryl Pregibon in Applied Statistics in 1980. You might want to read the paper here.

There has also been some more recent work on that front, noteably by Cheng and Wu in their 1994 JASA paper.

As stated by @gung using the deviance is also possible, see e.g., this paper if you don't want to take it at face value.

share|improve this answer
2  
+1, it's nice to have the canonical paper. BTW, I suspect you meant recent work, not "reason" work. –  gung Jan 16 '13 at 1:16
add comment

(I'm just copying the information from the comments here so that this question doesn't show up as officially unanswered.)

You can compare the two models by comparing the deviances. All the AIC and BIC do is adjust the deviances for the number of parameters in the model. Since that number is the same, it won't make any difference. In general, it will be very difficult to differentiate between different link functions unless they differ in shape; it is often better to use theoretical knowledge to determine the appropriate link function. For example, the logit and the probit links barely differ in shape at all but do differ in how you are thinking about the data generating process (as I discuss here).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.