2
$\begingroup$

I used Sklearn logistic regression for multiclass classifier to classify as Male , Female and Infant on abalone data set Below is my sample Logistic regression for multi classifier

x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.20,random_state=False)
log_reg=LogisticRegression()
log_model=log_reg.fit(x_train,y_train)
pred=log_model.predict(x_test)
confusion_matrix(y_test,pred)

Below is my confusion Matrix

        M   F   I           --- predicted
M   [[  64, 46, 39],
F   [   12, 237,42],
I   [   52, 79, 165]] actual vs Predicted 

Consider a case of 2X2 where I classified patient as HIV positive --1

        1       0       --- predicted
1   [[  1--TP,  0--FN],
0   [   1--FP,  0--TN ]]  Act vs Predicted 

unlike 2 x 2 I am unable to extrapolate it to N X N only I can make out is 64 I predicted as Male and which is Actually male as True Positive My question is how can I identify True Negative , False Positive , false Negative .

$\endgroup$
  • $\begingroup$ Couldn't infants be male & female? I'm not a biologist, but I could swear that was the case... I wonder if two binary LR models, or a multinomial LR w/ 4 categories (male infant, female infant, male non-infant, female non-infant), might make more sense here. $\endgroup$ – gung Dec 14 '17 at 15:32
  • $\begingroup$ Business wanted me to classify it as Infant rather I agree to your later point multinomial LR w/ 4 categories would have made more sense than just classifying it as Infant $\endgroup$ – nithin Dec 15 '17 at 2:48
3
$\begingroup$

Based on the 3x3 confusion matrix in your example (assuming I'm understanding the labels correctly) the columns are the predictions and the rows must therefore be the actual values. The main diagonal (64, 237, 165) gives the correct predictions. That is, the cases where the actual values and the model predictions are the same.

The first row are the actual males. The model predicted 64 of these correctly and incorrectly predicted 46 of the males to be female and 139 of the males to be infants.

Looking at the male column, of the 128 males predicted by the model (sum of column M), 64 were actually males, while 12 were females incorrectly predicted to be males and 52 were infants incorrectly predicted to be males.

Analogous interpretations apply to the other columns and rows.

          Predicted
          M    F    I 
Actual M 64   46  139
       F 12  237   42
       I 52   79  165

If this is a multinomial logistic regression model, then the model output would be predicted probabilities that each observation belongs to a particular class, rather than predicted classes. The links in @StephenKolassa's answer discuss the issue of scoring rules and you may want to consider what scoring rule will result in classifications that minimize a loss function tailored to your specific needs.

$\endgroup$
  • $\begingroup$ I agree with you on predicted probabilities but with sklearn you can also get predicted classes as well so in above confusion matrix It is predicted classes rather than probability. $\endgroup$ – nithin Dec 14 '17 at 9:55
  • $\begingroup$ Yes, but the predicted class membership will depend on where you set the probability thresholds and that in turn depends on the scoring rule/loss function. $\endgroup$ – eipi10 Dec 14 '17 at 17:32
2
$\begingroup$

True Positive, False Positive and similar counts and rates only make sense if there is a notion of "positive" and "negative" classes in your data. That is, only if you have exactly two classes. You have three classes, not two.

In your case, you can more or less reasonably discuss analogues, like "True Male" numbers: take the number of cases you correctly (!) classify as male and divide by the total number of males in the test sample.

Note that TPR, FPR, accuracy and similar KPIs have major problems if used to evaluate classification models.

$\endgroup$
  • $\begingroup$ Thank you I got your point , It helped me to understand things $\endgroup$ – nithin Dec 14 '17 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.