Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.
  1. Are there any particular assumptions regarding the errors for logistic regression such as the constant variance of the error terms and the normality of the residuals?

  2. Also typically when you have points that have a Cook's distance larger than 4/n, do you remove them? If you do remove them, how can you tell if the model with the removed points is better?

share|improve this question
    
Does anyone have any comments about the above question? –  lord12 Apr 23 '12 at 14:38
add comment

2 Answers

up vote 5 down vote accepted

I don't know if I can give you a complete answer, but I can give you some thoughts that may be helpful. First, all statistical models / tests have assumptions. However, logistic regression very much does not assume the residuals are normally distributed nor that the variance is constant. Rather, it is assumed that the data are distributed as a binomial, $\mathcal{B}(n_{x_i},p_{x_i})$, that is, with the number of Bernoulli trials equal to the number of observations at that exact set of covariate values and with the probability associated with that set of covariate values. Remember that the variance of a binomial is $np(1-p)$. Thus, if the $n$'s vary at different levels of the covariate, the variances will as well. Further, if any of the covariates are at all related to the response variable, then the probabilities will vary, and thus, so will the variances. These are important facts about logistic regression.

Second, model comparisons are usually performed between models with different specifications (for example, with different sets of covariates included), not over different subsets of the data. To be honest, I am not sure how that would properly be done. With a linear model, you could look at the 2 $R^2$s to see how much better the fit is with the aberrant data excluded, but this would only be descriptive, and you should know that $R^2$ would have to go up. With logistic regression, the standard $R^2$ cannot be used, however. There are various 'pseudo-$R^2$s' that have been developed to provide similar information, but they are often considered to be flawed and are not often used. For an overview of the different pseudo-$R^2$s that exist, see here. For some discussion, and criticism, of them, see here. Another possibility might be to jackknife the betas with and without the outliers included to see how excluding them contributes to stabilizing their sampling distributions. Once again, this would only be descriptive (i.e., it wouldn't constitute a test to tell you which model--er, subset of your data--to prefer) and the variance would have to go down. These things are true, for both pseudo-$R^2$s and the jackknifed distributions, because you selected those data to exclude based on the fact that they appear extreme.

share|improve this answer
add comment

1) Are there any particular assumptions regarding the errors for logistic regression such as the constant variance of the error terms and the normality of the residuals?

Logistic regression models don't have "errors" in the traditional sense. It is both counter-intuitive and methodologically inconsistent. The model outputs are fitted probabilities or risks whereas the observed outcomes are 0/1 event indicators. Methodologically, you would tend to under-emphasize domains of very high or very low fitted probabilities (contributing very small amounts to the residual distance) whereas the model fitting algorithm places considerably higher importance on such regions. Squared distance is generally a poor way of calibrating a logistic regression model.

An alternative goodness of fit test is the Hosmer-Lemeshow test in which the fitted values are used to create binned partitions based on deciles of fitted risk. You can read about this test in Alan Agresti's Categorical Data Analysis or the book Logistic Regression by Hosmer and Lemeshow.

2) Also typically when you have points that have a Cook's distance larger than 4/n, do you remove them? If you do remove them, how can you tell if the model with the removed points is better?

I never remove points based on sensitivity analyses. If I do a random sample of 100 people and their income and 1 person happens to be a billionaire, then my safest assumption is that that 1 billionaire represents 1/100th of the population.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.