# Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what?

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run glm in R, I get:

Warning messages:
1: glm.fit: algorithm did not converge
2: glm.fit: fitted probabilities numerically 0 or 1 occurred


I've read the other responses that suggest perfect separation might be occurring, but I'm confident that isn't the case in my data (though quasi-complete separation could exist; how can I test to see if that's the case?). If I remove some variables, the "did not converge" error might go away. But that's not always what happens.

I tried using the same variables in a bayesglm function and got the same errors.

What steps would you take to figure out exactly what's going on here? How do you figure out which variables are causing the problems?

• Why are you confident that separation isn't occurring? In the bayesglm paper, they argue that separation is "a common problem, even when the sample size is large and the number of predictors is small" – David J. Harris Dec 13 '12 at 5:26
• Another thought: bayesglm tries to avoid separation by adding a prior, but with 24,000 rows, the prior is probably getting swamped by the likelihood. Try shrinking prior.scale, possibly by a large amount. Also consider increasing the prior's degrees of freedom, which will help rule out large values associated with separation. – David J. Harris Dec 13 '12 at 5:31
• Thanks for the suggestions David. I don't think separation is occurring because when I sort each of the explanatory variables, the dependent variable isn't always true or false for high or low values of the explanatory variables. Unless this is considered separation: the dependent variable is true for all x7 > 32 but x7 is only > 32 in 10 cases. Is there a way to verify the separation outside of a logistic regression? Or see which variable is causing the separation? I tried your bayesglm suggestions (I set prior.scale to 1 and prior.df to Inf) and still got the Hauck Donner errors. – Dcook Dec 13 '12 at 7:27
• – user603 Dec 13 '12 at 18:23
• "How do you figure out which variables are causing the problems?" Binary-search is always a good fallback. You only have 50 variables, so if it's perfectly separated by one individual variable, 6 iterations will find the culprit. If it's two variables, at most 49+6=55 iterations will find it, worst-case. – smci Feb 2 '17 at 9:03

With such a large design space ($\mathbb{R}^{50}$!) it is possible to get perfect separation without having separation in any of the variable taken individually. I would even second David J. Harris's comment in saying that this is likely.