2
$\begingroup$

This question already has an answer here:

I'm fitting a logistic regression model in R. Following is the structure of my data set.

I've used the glm function in R.

lgM9 <- glm(V35~ ., family=binomial(link='logit'), data=traindata9,maxit=50)   
summary(lgM9)

Then I'm getting following warning message.

glm.fit: fitted probabilities numerically 0 or 1 occurred 

The resulted model is like below.

Deviance Residuals:    
Min          1Q      Median          3Q         Max
-6.173e-06   2.110e-08   2.110e-08   2.110e-08   4.800e-06`      

Coefficients:   
            Estimate    Std. Error    z value    Pr(>|z|)    
(Intercept) -1.787e+02  3.827e+06       0        1     
V1           1.890e+02  3.408e+06       0        1    
V3           1.118e+01  3.575e+06       0        1  
V4          -7.848e+01  3.548e+06       0        1  
V5           5.907e+01  3.463e+06       0        1  
V6           2.714e+01  1.358e+06       0        1    
V7          -2.702e+00  1.815e+06       0        1    
V8           3.982e+01  2.114e+06       0        1        
V9          -2.734e+01  4.579e+06       0        1        
V10          3.256e+01  8.839e+05       0        1        
V11         -2.947e+01  2.858e+06       0        1    
V12         -4.938e+01  2.722e+06       0        1    
V13         -4.126e+01  3.017e+06       0        1    
V14          3.966e+01  3.539e+06       0        1
V15          9.373e+01  1.283e+06       0        1
V16          3.142e+01  4.139e+06       0        1
V17          2.281e+01  5.939e+06       0        1
V18          3.875e+01  2.722e+06       0        1
V19         -2.905e+01  1.728e+06       0        1
V20         -6.367e+01  2.395e+06       0        1
V21          1.170e+01  3.323e+06       0        1
V22         -1.078e+02  5.119e+06       0        1
V23         -4.789e+01  7.774e+06       0        1
V24          6.323e+01  1.901e+06       0        1
V25          2.542e+01  5.440e+06       0        1
V26         -2.560e+01  3.597e+06       0        1
V27         -9.350e+01  2.115e+06       0        1
V28          1.159e+02  6.192e+06       0        1
V29          4.443e+01  7.150e+06       0        1
V30          6.238e+01  3.410e+06       0        1
V31          5.388e+01  1.606e+06       0        1
V32         -1.153e+02  3.318e+06       0        1
V33          9.183e-02  2.774e+06       0        1
V34         -1.925e+01  8.054e+06       0        1

(Dispersion parameter for binomial family taken to be 1)    
Null deviance: 1.3381e+02  on 245  degrees of freedom    
Residual deviance: 3.1571e-10  on 212  degrees of freedom    
AIC: 68

Number of Fisher Scoring iterations: 28

Why I'm getting all the z value as 0 and all the pr values as 1. Feel something is not right here and I can't figure this out. Can someone help me on this?

$\endgroup$

marked as duplicate by kjetil b halvorsen, mdewey, Peter Flom Oct 4 '16 at 11:22

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

migrated from stackoverflow.com Oct 4 '16 at 7:55

This question came from our site for professional and enthusiast programmers.

  • 5
    $\begingroup$ this is most likely complete separation and the Hauck-Donner effect (you can Google both of them ...) $\endgroup$ – Ben Bolker Sep 24 '16 at 13:20
  • 3
    $\begingroup$ A first glance, it looks like a lit of predictors for a logistic regression. Are you over-fitting, which can be the case when you have fewer data points than parameters in your model. Without knowing any more about your dataset, I advocate using a dimension reduction technique like Principal Components (PCA) to reduce the number of predictor variables and then fit the model. $\endgroup$ – Sun Bee Sep 24 '16 at 13:52
  • 1
    $\begingroup$ As Ben and Sun said: You have nowhere near enough degrees of freedom for so many predictor variables, and categories in some predictors only contain 0 (or 1) as outcome. If category A only has 0 as outcome, then the probability for any observation in category A is exactly 0, which gives you that warning. This in combination with your many predictors, gives inflation of the variance of your coefficients. I.e. the variance is far larger than the estimate, which gives you the p-value of 1. So your problem is the model, not the code. $\endgroup$ – Joris Meys Sep 24 '16 at 14:06