5
$\begingroup$

I have built a model to predict Upsell probability. When I use the function confusionMatirx from caret package, I get the following results:

> confusionMatrix(data = predict_svm_test_5, test_td$UpSell_Ind)
Confusion Matrix and Statistics

             Reference
Prediction    0    1
          0 7976 2886
          1  217  644

           Accuracy : 0.7353          
             95% CI : (0.7272, 0.7433)
No Information Rate : 0.6989          
P-Value [Acc > NIR] : < 2.2e-16       

              Kappa : 0.1987          
Mcnemar's Test P-Value : < 2.2e-16       

        Sensitivity : 0.9735          
        Specificity : 0.1824          
     Pos Pred Value : 0.7343          
     Neg Pred Value : 0.7480          
         Prevalence : 0.6989          
     Detection Rate : 0.6804          
Detection Prevalence : 0.9266          
  Balanced Accuracy : 0.5780          

   'Positive' Class : 0  

However, I expected to see the confusion matrix as follows:

             Reference
Prediction   1      0
      1     644    217
      0    2886   7976
Specificity(TPR): 0.9735
Sensitivity(TNR): 0.1824

1 meaning there was an Upsell (Event) and 0 meaning no Upsell (No Event) based on the PDF of Caret Package. Link is here Page 24, 25

Now my question: How do I interpret the results of confusionMatrix? The values given by the function are different from values that I calculate.

Thanks in advance for the help.

$\endgroup$
3
  • 1
    $\begingroup$ this question isn't clear - to me at least. The caret values and the values you calculated are identical. The difference is confusion matrix layout is cosmetic and not substantial. $\endgroup$
    – charles
    Jun 9, 2015 at 0:32
  • $\begingroup$ Thanks @charles for the response. OK, I can understand that the layout is cosmetic. But the thing that confuses me is the values that I get for Specificity and Sensitivity from my calculation and the values that are output from caret confusionMatrix function. The value of my calculated Specificity = Caret's Sensitivity. How do I interpret? Does it mean that True Positive rate of 0(from caret) = True Negative Rate of 1( from my calculation). This is where I am confused. $\endgroup$
    – EsBee
    Jun 9, 2015 at 15:35
  • 2
    $\begingroup$ this is just a coding issue I think. Sen/spec require a definition of "positive". Here caret has automatically chosen a value different from the one you want. I don't use the caret package but something like...confusionMatrix((pred, ref, positive=1) might work. you want to use the positive= option. $\endgroup$
    – charles
    Jun 10, 2015 at 0:21

1 Answer 1

4
$\begingroup$

Thanks @charles for pointing me to "positive". Though positive = 1 did not work as the argument positive takes only character value in the function. But I was able to get what I wanted using the following:

levels(test_td$UpSell_Ind)

[1] "0" "1"
confusionMatrix(data = predict_glm_vif_test, test_td$UpSell_Ind, positive = levels(test_td$UpSell_Ind)[2])

Confusion Matrix and Statistics

          Reference
Prediction    0    1
         0 8104 3241
         1   89  289

           Accuracy : 0.7159          
             95% CI : (0.7077, 0.7241)
No Information Rate : 0.6989          
P-Value [Acc > NIR] : 2.701e-05       

              Kappa : 0.0952          
 Mcnemar's Test P-Value : < 2.2e-16       

        Sensitivity : 0.08187         
        Specificity : 0.98914         
     Pos Pred Value : 0.76455         
     Neg Pred Value : 0.71432         
         Prevalence : 0.30112         
     Detection Rate : 0.02465         
 Detection Prevalence : 0.03224         
  Balanced Accuracy : 0.53550         

   'Positive' Class : 1  
$\endgroup$
3
  • $\begingroup$ I have the same problem, and I tried to print levels(df$colY) but I get NULL as output instead of factors ? $\endgroup$ Mar 4, 2017 at 14:38
  • 2
    $\begingroup$ @MurlidharFichadia : could you check the class of your column? if it is not factor, it will give NULL when you do print levels(df$colY) $\endgroup$
    – EsBee
    Mar 6, 2017 at 19:25
  • $\begingroup$ problem resolved. it was giving error because of it being character rather than a factor. Thanks $\endgroup$ Mar 6, 2017 at 23:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.