2
$\begingroup$

I am relatively new to R programming and I am using a dataset on Alzheimers disease and trying to predict Normal/Abnormal outcomes using several predictor variables and a logistic regression that divides outcomes to Normal/Abnormal (I grouped Alzheimers, mild cognitive impairment, impairment into the Abnormal category). When I run the logistic regression, I get: Warning messages: 1: In predict.lm(object, newdata, se.fit, scale = 1, type = if (type == : prediction from a rank-deficient fit may be misleading How do you deal with this? Also, my confusion matrix is showing that the model is just predicting almost everything as well, leading to way too many false positives.

Any insights on how I've done and what I could do to improve would be greatly appreciated. A link to my code: https://github.com/asimonia/NACC.git All you need to do is change to the correct file path in the argument in the read.csv function.

Here are my results:

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.4612  -0.3743  -0.2637  -0.1828   3.5832  

Coefficients: (2 not defined because of singularities)
                                             Estimate Std. Error z value Pr(>|z|)    
(Intercept)                                 -9.479607   0.404849 -23.415  < 2e-16 ***
SEXFemale                                   -0.330718   0.062027  -5.332 9.72e-08 ***
`EDUChigh school or GRE`                     0.083953   0.161738   0.519 0.603714    
`EDUCbachelor’s degree`                      0.120916   0.163368   0.740 0.459212    
`EDUCmaster's degree`                        0.052916   0.166313   0.318 0.750358    
EDUCdoctorate                               -0.432873   0.189831  -2.280 0.022589 *  
NACCBMI                                     -0.022288   0.006471  -3.444 0.000573 ***
SMOKYRS                                      0.002659   0.002755   0.965 0.334581    
`PACKSPER1 cigarette to less than 1/2 pack` -0.116233   0.097550  -1.192 0.233448    
`PACKSPER½ pack to less than 1 pack`        -0.037703   0.104980  -0.359 0.719484    
`PACKSPER1 pack to 1½ packs`                -0.074125   0.133189  -0.557 0.577842    
`PACKSPER1½ packs to 2 packs`               -0.341374   0.175910  -1.941 0.052305 .  
`PACKSPERMore than two packs`                0.017249   0.158228   0.109 0.913191    
ALCOHOLActive                                0.649711   0.357180   1.819 0.068911 .  
ALCOHOLInactive                              0.393085   0.133058   2.954 0.003134 ** 
CVHATTActive                                 0.332094   0.261834   1.268 0.204679    
CVHATTInactive                               0.116788   0.111267   1.050 0.293893    
CBSTROKEActive                               0.521873   0.250214   2.086 0.037006 *  
CBSTROKEInactive                             0.635339   0.118670   5.354 8.61e-08 ***
`HYPERTENRecent/Active`                     -0.047228   0.062215  -0.759 0.447787    
`HYPERTENRemote/Inactive`                    0.478778   0.118430   4.043 5.28e-05 ***
DEP2YRSYES                                   1.024557   0.063537  16.125  < 2e-16 ***
NACCAPOEe3e4                                 0.457120   0.066714   6.852 7.29e-12 ***
NACCAPOEe3e2                                -0.192786   0.092721  -2.079 0.037600 *  
NACCAPOEe4e4                                 0.839686   0.176109   4.768 1.86e-06 ***
NACCAPOEe4e2                                 0.547205   0.162910   3.359 0.000782 ***
NACCAPOEe2e2                                -0.005128   0.376799  -0.014 0.989141    
`NACCNE4S1 copy of e4 allele`                      NA         NA      NA       NA    
`NACCNE4S2 copies of e4 allele`                    NA         NA      NA       NA    
AGE                                          0.090927   0.003481  26.118  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 10969.4  on 24027  degrees of freedom
Residual deviance:  9630.1  on 24000  degrees of freedom
AIC: 9686.1

Number of Fisher Scoring iterations: 6

Confusion Matrix and Statistics

          Reference
Prediction Normal Abnormal
  Normal     7519      479
  Abnormal      5        5

               Accuracy : 0.9396          
                 95% CI : (0.9341, 0.9447)
    No Information Rate : 0.9396          
    P-Value [Acc > NIR] : 0.5121          

                  Kappa : 0.0178
Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ R is telling you that you are over-fitting. In fact, you are over-fitting so much that not all coefficients can be estimated. $\endgroup$
    – Roland
    Commented Nov 27, 2019 at 7:01
  • 1
    $\begingroup$ I am voting to leave this open. This is not an R problem, it is a statistics problem. @Roland has identified that problem. $\endgroup$
    – Peter Flom
    Commented Nov 27, 2019 at 11:25

1 Answer 1

Reset to default
2
$\begingroup$

If you look at a crosstab of the copy of allele variable I expect you will find the problem: There aren't enough people in some of the cells and you cannot estimate the parameters for those levels of that variable.

Also, it looks like you are using lm (which fits linear regression) but your DV (normal/abnormal) looks like it is categorical, which would indicate using logistic regression.

Looking at your variables, it also seems likely that you have some coliearity. Have you checked that?

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for your help, Peter! I am using a binomial family GLM which I think is logistic regression and would be applicable to a binary encoded RV. Is this not correct? Also how do I measure and deal with coliearity? $\endgroup$
    – Alex
    Commented Nov 27, 2019 at 16:06
  • $\begingroup$ Your code sample show lm..... You can search on here for collinearity to see many ways ofdealing with it. $\endgroup$
    – Peter Flom
    Commented Nov 27, 2019 at 19:18
  • $\begingroup$ Hmm interesting, the warning shows lm, maybe that is what you mean, but the R script on my github is glmModel <- train(NACCUDSD ~ .,data = training,trControl = trCntl,method="glm",family = "binomial") $\endgroup$
    – Alex
    Commented Nov 27, 2019 at 19:26
  • $\begingroup$ That is very strange. I use R a little, but I am not expert. This part might be a question for an R list. $\endgroup$
    – Peter Flom
    Commented Nov 27, 2019 at 21:11
  • 4
    $\begingroup$ predict.glm calls predict.lm internally. $\endgroup$
    – Roland
    Commented Nov 28, 2019 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.