7
$\begingroup$

Matthews correlation coefficient ($\textrm{MCC}$) is a measurement to measure the quality of a binary classification ([Wikipedia][1]). $\textrm{MCC}$ formulation is given for binary classification utilizing true positives ($TP$), false positives ($FP$), false negatives ($FN$), and true negatives ($TN$) values as given below:

$$\textrm {MCC} = \frac{TP\times TN - FP\times FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}$$

I have a case where I need to classify three different classes, $A$, $B$, and $C$. Can I apply the above formulation to calculate $\textrm{MCC}$ for multi-class case after calculating $TP$, $TN$, $FP$, and $FN$ values for each class as shown below? $$ TP = TP_A + TP_B + TP_C;\\ TN = TN_A + TN_B + TN_C;\\ FP = FP_A + FP_B + FP_C;\\ FN = FN_A + FN_B + FN_C; $$

$\endgroup$
4
$\begingroup$

Yes, in general, you can. This approach you want to use is sometimes called "Micro-Averaging": first, sum all TNs, FPs, etc for each class, and then calculate the statistic of interest.

Another way to combine the statistics for individual classes is to use so-called "Macro-Averaging": here you first calculate the statistics for individual classes (A vs not A, B vs not B, etc), and then calculate the average of them.

You may have a look here for some extra details. The page talks about Precision and Recall, but I believe it applies to Matthew's coefficient as well as other statistics based on contingency tables.

$\endgroup$
4
$\begingroup$

Macro averaging technique works well for precision, sensitivity, and specificity. But when I tried it for MCC it did not give proper results. For more details on multiclass MCC calculations see:

  1. Jurman G, Riccadonna S, Furlanello C (2012) "A Comparison of MCC and CEN Error Measures in Multi-Class Prediction". PLoS ONE 7(8): e41882. doi:10.1371/journal.pone.0041882
  2. Jurman, Giuseppe, and Cesare Furlanello. "A unifying view for performance measures in multi-class prediction." arXiv preprint arXiv:1008.2908 (2010).

The following code worked for me:

% the confusion matrix at input is given by matrix cm_svm_array
mcc_numerator=0;count=1;
% limits klm=1 TO n SUM(ckk.cml - clk.ckm)
for k = 1:1:length(cm_svm_array)
    for l=1:1:length(cm_svm_array)
        for m=1:1:length(cm_svm_array)
          mcc_numerator1(count) = (cm_svm_array(k,k) *cm_svm_array(m,l))-
                                  (cm_svm_array(l,k)*cm_svm_array(k,m))
          mcc_numerator=mcc_numerator+mcc_numerator1(count)
          count=count+1;
        end
    end
end

mcc_denominator_1=0 ; count=1;
for k=1:1:length(cm_svm_array)
     mcc_den_1_part1=0;
    for l=1:1:length(cm_svm_array)
        mcc_den_1_part1= mcc_den_1_part1+cm_svm_array(l,k);
    end
    mcc_den_1_part2=0;
    for f=1:1:length(cm_svm_array)
        if f ~=k
          for g=1:1:length(cm_svm_array)
            mcc_den_1_part2= mcc_den_1_part2+cm_svm_array(g,f);
          end
        end
    end
    mcc_denominator_1=(mcc_denominator_1+(mcc_den_1_part1*mcc_den_1_part2));
end

mcc_denominator_2=0; count=1;
for k=1:1:length(cm_svm_array)
     mcc_den_2_part1=0;
    for l=1:1:length(cm_svm_array)
        mcc_den_2_part1= mcc_den_2_part1+cm_svm_array(k,l);
    end
    mcc_den_2_part2=0;
    for f=1:1:length(cm_svm_array)
        if f ~=k
          for g=1:1:length(cm_svm_array)
            mcc_den_2_part2= mcc_den_2_part2+cm_svm_array(f,g);
          end
        end
    end
    mcc_denominator_2=(mcc_denominator_2+(mcc_den_2_part1*mcc_den_2_part2));
end

mcc = (mcc_numerator)/((mcc_denominator_1^0.5)*(mcc_denominator_2^0.5))
$\endgroup$
0
$\begingroup$

MCC is designed for binary classification.

If you want get a similar measurement of a classifier, you could try Cohen's Kappa, it can be applied to multi-class confusion matrix.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.