Matthews correlation coefficient with multi-class

Matthews correlation coefficient ($$\textrm{MCC}$$) is a measurement to measure the quality of a binary classification ([Wikipedia][1]). $$\textrm{MCC}$$ formulation is given for binary classification utilizing true positives ($$TP$$), false positives ($$FP$$), false negatives ($$FN$$), and true negatives ($$TN$$) values as given below:

$$\textrm {MCC} = \frac{TP\times TN - FP\times FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}$$

I have a case where I need to classify three different classes, $$A$$, $$B$$, and $$C$$. Can I apply the above formulation to calculate $$\textrm{MCC}$$ for multi-class case after calculating $$TP$$, $$TN$$, $$FP$$, and $$FN$$ values for each class as shown below? $$TP = TP_A + TP_B + TP_C;\\ TN = TN_A + TN_B + TN_C;\\ FP = FP_A + FP_B + FP_C;\\ FN = FN_A + FN_B + FN_C;$$

Yes, in general, you can. This approach you want to use is sometimes called "Micro-Averaging": first, sum all TNs, FPs, etc for each class, and then calculate the statistic of interest.

Another way to combine the statistics for individual classes is to use so-called "Macro-Averaging": here you first calculate the statistics for individual classes (A vs not A, B vs not B, etc), and then calculate the average of them.

You may have a look here for some extra details. The page talks about Precision and Recall, but I believe it applies to Matthew's coefficient as well as other statistics based on contingency tables.

Macro averaging technique works well for precision, sensitivity, and specificity. But when I tried it for MCC it did not give proper results. For more details on multiclass MCC calculations see:

1. Jurman G, Riccadonna S, Furlanello C (2012) "A Comparison of MCC and CEN Error Measures in Multi-Class Prediction". PLoS ONE 7(8): e41882. doi:10.1371/journal.pone.0041882
2. Jurman, Giuseppe, and Cesare Furlanello. "A unifying view for performance measures in multi-class prediction." arXiv preprint arXiv:1008.2908 (2010).

The following code worked for me:

% the confusion matrix at input is given by matrix cm_svm_array
mcc_numerator=0;count=1;
% limits klm=1 TO n SUM(ckk.cml - clk.ckm)
for k = 1:1:length(cm_svm_array)
for l=1:1:length(cm_svm_array)
for m=1:1:length(cm_svm_array)
mcc_numerator1(count) = (cm_svm_array(k,k) *cm_svm_array(m,l))-
(cm_svm_array(l,k)*cm_svm_array(k,m))
mcc_numerator=mcc_numerator+mcc_numerator1(count)
count=count+1;
end
end
end

mcc_denominator_1=0 ; count=1;
for k=1:1:length(cm_svm_array)
mcc_den_1_part1=0;
for l=1:1:length(cm_svm_array)
mcc_den_1_part1= mcc_den_1_part1+cm_svm_array(l,k);
end
mcc_den_1_part2=0;
for f=1:1:length(cm_svm_array)
if f ~=k
for g=1:1:length(cm_svm_array)
mcc_den_1_part2= mcc_den_1_part2+cm_svm_array(g,f);
end
end
end
mcc_denominator_1=(mcc_denominator_1+(mcc_den_1_part1*mcc_den_1_part2));
end

mcc_denominator_2=0; count=1;
for k=1:1:length(cm_svm_array)
mcc_den_2_part1=0;
for l=1:1:length(cm_svm_array)
mcc_den_2_part1= mcc_den_2_part1+cm_svm_array(k,l);
end
mcc_den_2_part2=0;
for f=1:1:length(cm_svm_array)
if f ~=k
for g=1:1:length(cm_svm_array)
mcc_den_2_part2= mcc_den_2_part2+cm_svm_array(f,g);
end
end
end
mcc_denominator_2=(mcc_denominator_2+(mcc_den_2_part1*mcc_den_2_part2));
end

mcc = (mcc_numerator)/((mcc_denominator_1^0.5)*(mcc_denominator_2^0.5))


MCC is designed for binary classification.

If you want get a similar measurement of a classifier, you could try Cohen's Kappa, it can be applied to multi-class confusion matrix.