I'm modeling the effect of a categorical predictor on a binary dependent variable using logistic regression. I'm comparing models with/without the predictor using a likelihood-ratio test.

Two categories of the predictor are associated with values of 1 only (no 0s) for the dependent variable. Regression coefficients for these categories (expressed as changes in log(odds) compared to a reference category) are very large and highly suspicious, as this reference category is always associated with response values of 1 (but for one case), and I would thus expect regression coefficients close to 0 for these two categories. Comparisons between the reference category and other categories having more balanced distribution of 1 and 0s matches what I'm expecting from visual inspection of the data. Removing cases associated with these two 'problematic' categories does not change the logLikelihood of the models, but because it changes the number of parameters it affects the results of the likelihood ratio test.

Models are fitted using the glm function with binomial family and logit link in R.

My question therefore is: what model (or procedure) should I use to:

(1) test the global significance of the effect of the predictor on the dependent variable? Should I keep data from the 'problematic' categories in the model or not before conducted the likelihood ratio test?

(2) compare these two 'problematic' categories with others?

Any hint appreciated,

  • $\begingroup$ I would vote this as an answer (solutions are provided in the linked document) - but I can't because it is 'just' a comment. $\endgroup$ – Jehol Jan 24 '13 at 14:18
  • $\begingroup$ start by checking that you have no exact separation $\endgroup$ – user603 Feb 23 '13 at 19:06

I'm not certain but it sounds like you have a problem of quasi-complete separation. Paul Allison wrote an excellent paper dealing with this. He uses SAS but the paper really applies to any software package.

Per @Rolando2 's request, Allison's basic points (without his introduction and theory and so on) are that, for quasi-complete separation:

  1. You can delete the variable that causes the separation
  2. You can combine categories in the problem variable, if there is a sensible way to do so
  3. You can ignore the problem and report likelihood ratio chi squares for other variables in the model (this one surprised me)
  4. If it is a small N, you can use exact inference.
  5. You can use Bayesian methods
  6. You can use penalized maximum likelihood

but that for complete separation, only the first and last options are available.

  • 1
    $\begingroup$ Since links can rot, care to give an account of his explanation? $\endgroup$ – rolando2 Feb 23 '13 at 18:40
  • $\begingroup$ #1: shouldn't you remove all but the variable causing separation? If you have perfect separation along a subset of your variables, why use any other variables at all? $\endgroup$ – user603 Feb 23 '13 at 19:08
  • $\begingroup$ OK,yes, I guess technically I should say "variable or variables". Often it is only one variable. $\endgroup$ – Peter Flom Feb 23 '13 at 19:16
  • $\begingroup$ @PeterFlom: no i meant, shouldn't #1 be; delete the variable not causing separation...if a variable causes separation, the other ones are useless...why keep them? $\endgroup$ – user603 Feb 23 '13 at 23:25
  • $\begingroup$ The could be very useful! Why would they be useless? Separation COULD be due to a very strong relationship, but it is more often due to either a model building error, sample problems, too many categories in a classification variable and so on. If a continuous variable caused perfect celebration that would be excellent. $\endgroup$ – Peter Flom Feb 24 '13 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.