# Robust logistic regression vs logistic regression

It's been a while since I've thought about or used a robust logistic regression model. However, I ran a few logits yesterday and realized that my probability curve was being affected by some 'extreme' values, and particularly low ones. However, when I went to run a robust logit model, I got the same results as I did in my logit model.

Under what circumstances should a robust logit produce different results from a traditional logit model? (in terms of coefficients)

R Code:

> library(Design)
> me = lrm(factor(dlstatus) ~ dlour_bid, data=dlmydat)
> me

Logistic Regression Model

lrm(formula = factor(dlstatus) ~ dlour_bid, data = dlmydat)

Frequencies of Responses
1   2
906 154

Obs  Max Deriv Model L.R.       d.f.          P          C        Dxy      Gamma      Tau-a         R2      Brier
1060      3e-05     170.11          1          0       0.81      0.619      0.621      0.154      0.263      0.105

Coef      S.E.      Wald Z P
Intercept -5.233549 0.3731235 -14.03 0
dlour_bid  0.005367 0.0004925  10.90 0

> library(car)
> dlmod = glm(factor(dlstatus) ~ dlour_bid, data=dlmydat, family=binomial(link="logit"))
> summary(dlmod)

Call:
glm(formula = factor(dlstatus) ~ dlour_bid, family = binomial(link = "logit"),
data = dlmydat)

Deviance Residuals:
Min       1Q   Median       3Q      Max
-1.2345  -0.5687  -0.3059  -0.1739   2.6999

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -5.2335492  0.3731235  -14.03   <2e-16 ***
dlour_bid    0.0053667  0.0004925   10.90   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 878.61  on 1059  degrees of freedom
Residual deviance: 708.50  on 1058  degrees of freedom
AIC: 712.5

Number of Fisher Scoring iterations: 6

• glm() is not robust, and a quick look at lrm() doesn't tell me that it's robust either. Am I missing something? Thanks. – djma Jan 14 '12 at 3:35

First I would ask what do you mean by robust logistic regression (it could mean a couple of different things ...). Nevertheless, assuming that you are using "robust" in the sense that you want to control for heteroscedasticity in binary outcome models what I know is the following:

1) You should read in detail the 15th chapter of the Wooldridge 2001 Econometrics of Cross Section and panel data book (or any other equivalent book that talks about binary outcome models in detail).

2) Heteroscedasticity in binary outcome models will affect both the "Betas" and their standard errors.

2a) BETAS: Heteroscedasticity in binary outcome models has functional form implications. Basically F(XB) != LOGISTIC( XB ) if you have it, then your estimation will be inconsistent regardless of what you do ...

If you are absolutely sure about the type of heteroskedasticity you are having, this is, how your error changes as X changes, then you can correct your covariates accordingly to control for this. Since that is unlikely there is nothing you can do about it.

Now the fact that the estimation of Betas is inconsistent might not be very relevant anyway since the partial effects may still be a good approximation of the real partial effects. The whole point here is that heteroscedasticity in binary outcome models implies functional form mispecification and should be treated accordingly.

As an example think about probit vs logit. For your data, only one of these models can be the correct data generation process (if any). So when you estimate both of them you must know that at least one of the models will surely have inconsistent betas. But if go and look at their partial effects you won't see much of a difference ...

Go and test for heteroscedasticity first to see if this can be an issue. There are several tests arround ....

2 b) Standard Errors: Under heteroscedasiticty your standard errors will also be miscalculated by the "normal" way of estimating these models. In R what i would suggest is that you use bootstrap since i am not sure if there are any packages available that correct for kinds of misspecification and/or that allow for intragroup correlation ....

If this has nothing to do with what you asked and as Rolando2 pointed out in the comment you are trying to penalize outliers in the regression then you should know that your use of the lrm function is not correct: you are calling it with the default parameters in which case, quoting from the documentation: The default is penalty=0 implying that ordinary unpenalized maximum likelihood estimation is used

Hope this helped somehow, Miguel

• Could you please clarify why you believe heteroscedasticity is an issue here (isn't the problem instead one of influential points, or leverage?) and why you are calling this a "latent variable model." – rolando2 Jan 14 '12 at 14:26
• Rolando, I assumed that, maybe errouneously because, "robust" inference is a term often used to designate heteroscedasticity-consistent standard errors or Huber–White standard errors, Eicker–White or Eicker–Huber–White ... If the problem is one of outliers then, in the logit model, think (although i never used this) there must be some specification of how you will penalize these observations in the regression. If that is what you want you are not using the "lrm" function properly since you should specify the penalizing matrix ! – mmgm Jan 14 '12 at 15:36
• Actually if you go and read lrm documentation you will see that the default penalty is "no penalty": "The default is penalty=0 implying that ordinary unpenalized maximum likelihood estimation is used" – mmgm Jan 14 '12 at 15:45
• Regarding the "latent variable" part of your comment i just corrected it. I am used to thinking on probit and logit models as the outcome of "utility building process" which is unobserved. My bad since i absolutely have no idea in what context this is being used. – mmgm Jan 14 '12 at 15:59