0
$\begingroup$

I've produced a logistic regression model using 70% of my data and tested it using the remaining 30%.

These are the results of my confusion matrix:

Confusion Matrix and Statistics

Confusion Matrix (wouldn't copy over for some reason)

           Accuracy : 0.8161          
             95% CI : (0.7866, 0.8431)
No Information Rate : 0.8175          
P-Value [Acc > NIR] : 0.56            
                                      
              Kappa : 0.0065          
                                      

Mcnemar's Test P-Value : <2e-16

        Sensitivity : 0.996764        
        Specificity : 0.007246        
     Pos Pred Value : 0.818061        
     Neg Pred Value : 0.333333        
         Prevalence : 0.817460        
     Detection Rate : 0.814815        

Detection Prevalence : 0.996032
Balanced Accuracy : 0.502005

   'Positive' Class : 0   

I'm confused as it states 86% accuracy. From my interpretation, the confusion matrix highlights that the logistic regression model I have produced does not predict very well unless I've got that wrong. Confused about these results, could anyone help interpret them?

$\endgroup$
3
  • $\begingroup$ Please provide additional details about your aims and input data. It would be easier for us to provide you guidance. $\endgroup$ Commented May 22 at 18:27
  • $\begingroup$ There is 82% 0 data to begin with and 18% 1 data. Could the above be due to an imbalance of data? $\endgroup$
    – Laura
    Commented May 22 at 18:33
  • $\begingroup$ You may find these links on "imbalanced data" helpful. $\endgroup$ Commented May 22 at 19:01

1 Answer 1

1
$\begingroup$

The accuracy of your model is 0.82, but this isn't very good, as there is 82% of the data in one class. In other words, if you made no model at all, you could do just as well as your model. You can also see this in the sensitivity and specificity results -- one is 0.99, the other is 0.01.

Try playing with the cost function or the cutoff values for your logistic reg.

$\endgroup$
4
  • $\begingroup$ What are cost functions and cutoff values? Apologies, I've never come across this problem before. $\endgroup$
    – Laura
    Commented May 22 at 19:01
  • $\begingroup$ Look at the links that Stephen provided in his comment. $\endgroup$
    – Peter Flom
    Commented May 22 at 19:02
  • 2
    $\begingroup$ Laura: your logistic regression gives you predictions of class membership probabilities. To get "hard" 0-1 predictions (which are the only ones that can be "True" or "False"), you need to compare these probabilities to a threshold. This threshold is usually silently set to 0.5, which I would argue does not usually make a lot of sense, see here. I would say that the first step should be to understand whether you need "hard" or probabilistic predictions, and if the former, to think about the threshold - as Peter says. $\endgroup$ Commented May 22 at 19:05
  • $\begingroup$ I've used upsampling to resolve the imbalance of data - is this valid? $\endgroup$
    – Laura
    Commented May 23 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.