Matthews correlation coefficient with multi-class

Question

Matthews correlation coefficient ($\textrm{MCC}$) is a measurement to measure the quality of a binary classification ([Wikipedia][1]). $\textrm{MCC}$ formulation is given for binary classification utilizing true positives ($TP$), false positives ($FP$), false negatives ($FN$), and true negatives ($TN$) values as given below:

$$\textrm {MCC} = \frac{TP\times TN - FP\times FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}$$

I have a case where I need to classify three different classes, $A$, $B$, and $C$. Can I apply the above formulation to calculate $\textrm{MCC}$ for multi-class case after calculating $TP$, $TN$, $FP$, and $FN$ values for each class as shown below? $$ TP = TP_A + TP_B + TP_C;\\ TN = TN_A + TN_B + TN_C;\\ FP = FP_A + FP_B + FP_C;\\ FN = FN_A + FN_B + FN_C; $$

Answer 1

Yes, in general, you can. This approach you want to use is sometimes called "Micro-Averaging": first, sum all TNs, FPs, etc for each class, and then calculate the statistic of interest.

Another way to combine the statistics for individual classes is to use so-called "Macro-Averaging": here you first calculate the statistics for individual classes (A vs not A, B vs not B, etc), and then calculate the average of them.

You may have a look here for some extra details. The page talks about Precision and Recall, but I believe it applies to Matthew's coefficient as well as other statistics based on contingency tables.

Answer 2

Macro averaging technique works well for precision, sensitivity, and specificity. But when I tried it for MCC it did not give proper results. For more details on multiclass MCC calculations see:

1. Jurman G, Riccadonna S, Furlanello C (2012) "A Comparison of MCC and CEN Error Measures in Multi-Class Prediction". PLoS ONE 7(8): e41882. doi:10.1371/journal.pone.0041882
2. Jurman, Giuseppe, and Cesare Furlanello. "A unifying view for performance measures in multi-class prediction." arXiv preprint arXiv:1008.2908 (2010).

The following code worked for me:

% the confusion matrix at input is given by matrix cm_svm_array
mcc_numerator=0;count=1;
% limits klm=1 TO n SUM(ckk.cml - clk.ckm)
for k = 1:1:length(cm_svm_array)
    for l=1:1:length(cm_svm_array)
        for m=1:1:length(cm_svm_array)
          mcc_numerator1(count) = (cm_svm_array(k,k) *cm_svm_array(m,l))-
                                  (cm_svm_array(l,k)*cm_svm_array(k,m))
          mcc_numerator=mcc_numerator+mcc_numerator1(count)
          count=count+1;
        end
    end
end

mcc_denominator_1=0 ; count=1;
for k=1:1:length(cm_svm_array)
     mcc_den_1_part1=0;
    for l=1:1:length(cm_svm_array)
        mcc_den_1_part1= mcc_den_1_part1+cm_svm_array(l,k);
    end
    mcc_den_1_part2=0;
    for f=1:1:length(cm_svm_array)
        if f ~=k
          for g=1:1:length(cm_svm_array)
            mcc_den_1_part2= mcc_den_1_part2+cm_svm_array(g,f);
          end
        end
    end
    mcc_denominator_1=(mcc_denominator_1+(mcc_den_1_part1*mcc_den_1_part2));
end

mcc_denominator_2=0; count=1;
for k=1:1:length(cm_svm_array)
     mcc_den_2_part1=0;
    for l=1:1:length(cm_svm_array)
        mcc_den_2_part1= mcc_den_2_part1+cm_svm_array(k,l);
    end
    mcc_den_2_part2=0;
    for f=1:1:length(cm_svm_array)
        if f ~=k
          for g=1:1:length(cm_svm_array)
            mcc_den_2_part2= mcc_den_2_part2+cm_svm_array(f,g);
          end
        end
    end
    mcc_denominator_2=(mcc_denominator_2+(mcc_den_2_part1*mcc_den_2_part2));
end

mcc = (mcc_numerator)/((mcc_denominator_1^0.5)*(mcc_denominator_2^0.5))

Answer 3

MCC can be used for binary and multiclass classification Wikipedia and it's implemented in sci-kit learn for binary and multiclass labels.

Answer 4

MCC is designed for binary classification.

If you want get a similar measurement of a classifier, you could try Cohen's Kappa, it can be applied to multi-class confusion matrix.