NA in glm model

Question

I have the data here.But When I tried to build the logistic regression model using glm function its shows NA in TotalVisits. I have found similar question on stack overflow but that is answered for linear model.

 str(quality)
'data.frame':   131 obs. of  14 variables:
 $ MemberID            : int  1 2 3 4 5 6 7 8 9 10 ...
 $ InpatientDays       : int  0 1 0 0 8 2 16 2 2 4 ...
 $ ERVisits            : int  0 1 0 1 2 0 1 0 1 2 ...
 $ OfficeVisits        : int  18 6 5 19 19 9 8 8 4 0 ...
 $ Narcotics           : int  1 1 3 0 3 2 1 0 3 2 ...
 $ DaysSinceLastERVisit: num  731 411 731 158 449 ...
 $ Pain                : int  10 0 10 34 10 6 4 5 5 2 ...
 $ TotalVisits         : int  18 8 5 20 29 11 25 10 7 6 ...
 $ ProviderCount       : int  21 27 16 14 24 40 19 11 28 21 ...
 $ MedicalClaims       : int  93 19 27 59 51 53 40 28 20 17 ...
 $ ClaimLines          : int  222 115 148 242 204 156 261 87 98 66 ...
 $ StartedOnCombination: logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
 $ AcuteDrugGapSmall   : int  0 1 5 0 0 4 0 0 0 0 ...
 $ PoorCare            : int  0 0 0 0 0 1 0 0 1 0 ...



table(is.na(quality))
FALSE 
1834

My data does not contain any NA values.

set.seed(100)
split <- sample.split(quality$PoorCare, SplitRatio = .5)
train <-subset(quality, split ==TRUE)
test <- subset(quality, split ==FALSE)

Building the model using all variable

log.Quality <- glm(PoorCare ~ ., data = train, family = 'binomial')

summary(log.Quality)      
Call:
glm(formula = PoorCare ~ ., family = "binomial", data = train)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.5679  -0.6384  -0.3604  -0.1154   2.1298  

Coefficients: (1 not defined because of singularities)
                          Estimate Std. Error z value Pr(>|z|)  
(Intercept)              -3.583178   1.807020  -1.983   0.0474 *
MemberID                 -0.008742   0.010988  -0.796   0.4263  
InpatientDays            -0.106578   0.095632  -1.114   0.2651  
ERVisits                  0.275225   0.310364   0.887   0.3752  
OfficeVisits              0.126433   0.066140   1.912   0.0559 .
Narcotics                 0.190862   0.106890   1.786   0.0742 .
DaysSinceLastERVisit     -0.001221   0.002026  -0.603   0.5467  
Pain                     -0.020104   0.023057  -0.872   0.3832  
TotalVisits                     NA         NA      NA       NA  
ProviderCount             0.046297   0.040637   1.139   0.2546  
MedicalClaims             0.025123   0.030564   0.822   0.4111  
ClaimLines               -0.010384   0.012746  -0.815   0.4152  
StartedOnCombinationTRUE  2.205058   1.724923   1.278   0.2011  
AcuteDrugGapSmall         0.217813   0.139890   1.557   0.1195  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 72.549  on 64  degrees of freedom
Residual deviance: 49.213  on 52  degrees of freedom
AIC: 75.213

Number of Fisher Scoring iterations: 6

Can anyone provide me a good explanation why this is happening ?

Answer 1

Your one or more variable is perfectly correlated with each other. When I looked through the correlation matrix I found that the correlation b/w TotalVisits and OfficeVisits is very high i.e. 0.80388329. Removing one of them solve the problem.

log.Quality <- glm(PoorCare ~ . - OfficeVisits , data = train, family = 'binomial')
summary(log.Quality)

Call:
glm(formula = PoorCare ~ . - OfficeVisits, family = "binomial", 
    data = train)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.5679  -0.6384  -0.3604  -0.1154   2.1298  

Coefficients:
                          Estimate Std. Error z value Pr(>|z|)  
(Intercept)              -3.583178   1.807020  -1.983   0.0474 *
MemberID                 -0.008742   0.010988  -0.796   0.4263  
InpatientDays            -0.233011   0.124301  -1.875   0.0609 .
ERVisits                  0.148792   0.302291   0.492   0.6226  
Narcotics                 0.190862   0.106890   1.786   0.0742 .
DaysSinceLastERVisit     -0.001221   0.002026  -0.603   0.5467  
Pain                     -0.020104   0.023057  -0.872   0.3832  
TotalVisits               0.126433   0.066140   1.912   0.0559 .
ProviderCount             0.046297   0.040637   1.139   0.2546  
MedicalClaims             0.025123   0.030564   0.822   0.4111  
ClaimLines               -0.010384   0.012746  -0.815   0.4152  
StartedOnCombinationTRUE  2.205058   1.724923   1.278   0.2011  
AcuteDrugGapSmall         0.217813   0.139890   1.557   0.1195  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 72.549  on 64  degrees of freedom
Residual deviance: 49.213  on 52  degrees of freedom
AIC: 75.213

Number of Fisher Scoring iterations: 6

Hope this will help you to understand why it is happening.